Homelab upgrades - Adding HBA, SLOG, and SSDs for VM storage

[BackwardsCire]

Hi Everyone, before I make these changes to my server I wanted to write this up to see if there were any other suggestions.

Use Case #1: Home file sharing - Mostly a repository for our personal data, digital photos, and media. Only two users and very light demands when we do (working on documents, editing some photos) and user access is from Windows and Mac. Been using the server for this for a long time and it works very well for this use case. Way over powered honestly.

Use Case #2: VM/Container storage backend - I have a four SFF systems running Proxmox and configured an NFS share on my TrueNAS server. Each physical host is planned to have a couple of containers and VMs on them and will use shared storage rather than the local disks for ease of migration and data centralization. It is this use case that I want make some changes to the server to optimize for to reach decent perfornamce. Early testing was very slow. Not expecting blazing fast but disabling sync writes made the performance on that share decent enough for the VMs, but I know full well what that does to the relative safety of my data.

Current Hardware Config:
- SuperMicro X9SCL
- Intel(R) Xeon(R) CPU E31225 @ 3.10GHz
- 24GB DDR3 ECC RAM (4x DIMMS, all slots full)
- Intel Network Card
- Drives: All running on motherboard SATA headers currently
- - Boot: 256GB Samsung SSD
- - Storage:
- - - MainPool: 5x 4TB WD Purple in RAIDZ2 (Corrected, originally listed RED but they are Purple WDC WD40PURZ-85TTDY0)

TrueNAS: TrueNAS Core TrueNAS-13.0-U5.1

As I said, I've expanded my home lab to include 4 SFF machines that I'm running on Proxmox with NFS configured on TrueNAS and immediately found the NFS performance lacking. Did some reasearch and found the "solution" to turn off sync writes OR to add a SLOG. To test, I did turn off sync writes for the single NFS share but I'm not comfortable with that as a long term solution. So I've purchased some ebay hardware and planned on making the following storage changes:


Proposed Hardware Config:
- SuperMicro X9SCL
- Intel(R) Xeon(R) CPU E31225 @ 3.10GHz
- 24GB DDR3 ECC RAM (4x DIMMS, all slots full)
- Intel Network Card
- Drives:
- - Motherboard:
- - - SSD boot drive
- - HBA: LSI 6Gbps SAS HBA 9200-8I IT Mode 20.0.7.X firmware
- - - MainPool: 5x 4TB WD Purple in RAIDZ2
- - - SLOG: Micron 5100 MAX MTFDDAK480TCC 480 GB (found some people here talking about this as a good SLOG and found them cheap on ebay)
- - - NEWPOOL: 2x SSDs for VM storage

Changes in summary: adding the HBA, recabling the existing disks to the HBA, adding a Micron5100 MAX SSD for a SLOG, and adding two additional SSDs for a NEW pool dedicated to a NFS share for the VM backend. These are brand new 1TB drives and will not be heavily used so I'm not concerned about the SSD lifetime issues. I'll monitor and replace as needed.

Question 1: Any major concerns with the above configuration?
Question 2: What pool config should I use for the SSDs for the VM backend storage? Mirror?
Question 3: Anything special for the SLOG with this device other than https://www.truenas.com/docs/references/slog/?


Thanks for all the great help and advice here!

 

[sretalla]

- - - MainPool: 5x 4TB WD RED in RAIDZ2

Make sure those are RED Plus, not "EFAX" disks. https://www.truenas.com/community/resources/list-of-known-smr-drives.141/

- - - SLOG: Micron 5100 MAX MTFDDAK480TCC 480 GB (found some people here talking about this as a good SLOG and found them cheap on ebay)

It has a surprisingly large TBW figure considering it's TLC NAND, but I don't know if it has PLP (so may actually be no better than sync=disabled in the end)

What pool config should I use for the SSDs for the VM backend storage? Mirror?

Yes, always mirrors for block storage. https://www.truenas.com/community/resources/the-path-to-success-for-block-storage.197/

Anything special for the SLOG with this device other than https://www.truenas.com/docs/references/slog/?

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.truenas.com

Also look for the "finding the best SLOG" thread

 

[BackwardsCire]

Make sure those are RED Plus, not "EFAX" disks. https://www.truenas.com/community/resources/list-of-known-smr-drives.141/

Correction, they are Purple drives. Been running them for a few years. Confirmed CMR. WDC WD40PURZ-85TTDY0

TIL: TBW and PLP! - I did some googling and unfortunately can't find if it was PLP... I had previously found the best SLOG thread and similar and that is where I found this Micron 5100 MAX suggested.


Thanks for the quick reply!

 

[Etorix]

With a SSD mirror as pool, there may be less need for a SLOG.
Otherwise a guaranteed good SLOG is an Optane drive. If you have a free PCIe slot look for, say, an Optane 900p 280 GB while supplies last—or while users get rid of this discontinued hardware.