First FreeNAS build - advice welcome! (FN10)

[albertdole]

Hey all,

Thanks for the great resource that this forum provides. I've been poring through the Hardware Guide, Quick Hardware Guide, and various threads and am about ready to pull the trigger on my purchases. This will be for home-use and my aim is to backup pics/docs and run 1-2 Plex streams. I plan on using a raidz1 configuration for my initial 4 drives and adding another 4 down the road. Here's what I'm considering:

• Case: Fractal Define R5
• Motherboard: Supermicro X11SSH-F-O LGA1151/ Intel C236/ DDR4/ SATA3&USB3.0/ V&2GbE/ MicroATX Server Motherboard
• CPU: Intel Xeon E3-1240 v5 Quad-Core Skylake Processor 3.5GHz 8.0GT/s 8MB LGA 1151 CPU, Retail
• RAM: Micron DDR4-2400 16GB/2Gx72 VLP ECC/REG CL17 Server Memory (2x)
• PSU: Seasonic SSR-650RM 650W 80 PLUS Gold ATX12V/EPS12V Power Supply
• HDD: Western Digital Red WD40EFRX 4TB IntelliPower SATA3/SATA 6.0 GB/s 64MB Hard Drive (3.5 inch) (4x)
• Boot Drive: Intel 600p Series SSDPEKKW128G7X1 128GB M.2 80mm PCI-Express 3.0 x4 Solid State Drive (TLC) XX-Transcend 32GB SATA III 6Gb/s MSA370 mSATA Solid State Drive (TS32GMSA370)-XX
• UPS: CyberPower CP1200AVR AVR Series UPS 1200VA 720W Mini-Tower

Questions:

• Compatibility-wise, does everything look good? Did I forget anything?
• When I add the additional 4 drives down the road, I could create another vdev in the existing zpool or in a new zpool if I didn't mind having to manage 2 separate pools, correct? According to Cyberjock's PPT guide, "If any VDev in a zpool is failed, you will lose the entire zpool with no chance of partial recovery." Then am I correct in thinking a separate zpool would prevent a total failure if one of my vdev's failed?
• I decided to go with the X11SSH-F instead of the X11SSM-F because it includes the M.2 port for my boot drive so I could use all 8 SATA3s for my drives. Is this reasonable? The guide had recommendations for SSD brands, but didn't specifically mention M.2 brands -- I assume the recommendations are the same. I was able to find the Transcend M.2 in 32GB, but not the other brands. Is this one fine? I noticed Supermicro has a Tested M.2 List for the board, but only includes a few large Toshibas. My understanding is that it's very important to have manufacturer-approved RAM; is that not necessary for the M.2 or HDDs (they only list Seagates for the HDDs)?
• Since I'm using an M.2 for my boot drive rather than a USB drive, how do I mirror the contents?

Thanks for your thoughts and I look forward to getting my feet wet in FreeNAS soon.

EDIT 1: Replaced boot drive (thanks, BigDave)

 

[BigDave]

Right off the top of my head, that board is not compatible with registered DIMMs.
RAIDz1 is not recommended for large volume (4TB) hard drives due to long re-silver times.
Be careful that the use of the M.2 interface does not disable SATA ports or PCIe slots,
I'd hate to see you buy that board and find out you can't boot from the M.2 attached SSD.

After additional research, seems that the Transend that you linked to is mSATA and would
not be compatible with the gen3 M.2 slot on that Mobo. In order to boot from an M.2 SSD,
it would need to be NVMe capable, as well as the BIOS/firmware of the motherboard
(must be NVMe compatible) which it is :)

 

[Ericloewe]

Boot Drive: Transcend 32GB SATA III 6Gb/s MSA370 mSATA Solid State Drive (TS32GMSA370)

That's not M.2. Even if it was, the X11SSH-F only supports PCIe, not SATA M.2 devices.

 

[albertdole]

Thanks for the feedback!

Right off the top of my head, that board is not compatible with registered DIMMs.

I'm a little confused by this because I matched the manufacturer number right off of Supermicro's site:

RAIDz1 is not recommended for large volume (4TB) hard drives due to long re-silver times.

Oh, okay. I wasn't aware of that. So raidz2 would be acceptable for the initial 4x4TB setup? Either that or I should drop down to raidz1 with 4x3TB?

Be careful that the use of the M.2 interface does not disable SATA ports or PCIe slots, I'd hate to see you buy that board and find out you can't boot from the M.2 attached SSD.

I hadn't considered that. I found this in the manual:

The X11SSH-F/-LN4F motherboard contains one M.2 socket at J23. M.2 was formerly Next Generation Form Factor (NGFF) and serves to replace mini PCI-E and mSATA. M.2 allows for a greater variety of card sizes, increased functionality, and spatial efficiency. The M.2 socket on the X11SSH-F/-LN4F motherboard supports PCI-E 3.0 x2 M.2 cards in the 22x80mm form factor.

But that doesn't indicate if using it disables a SATA port or PCIe slot. How else could I determine that? Does anyone else who is familiar with the board know?

That's not M.2. Even if it was, the X11SSH-F only supports PCIe, not SATA M.2 devices.

Missed that! This is my first board with M.2 and I wasn't aware there were different interfaces for it.
Looking again, I'm having trouble finding M.2 PCIe drives with low capacities. Do they not exist? How about this one? - Intel 600p Series SSDPEKKW128G7X1 128GB M.2 80mm PCI-Express 3.0 x4 Solid State Drive (TLC)

 

[BigDave]

Missed that! This is my first board with M.2 and I wasn't aware there were different interfaces for it.
Looking again, I'm having trouble finding M.2 PCIe drives with low capacities. Do they not exist? How about this one? - Intel 600p Series SSDPEKKW128G7X1 128GB M.2 80mm PCI-Express 3.0 x4 Solid State Drive (TLC)

Yes, now that one would work fine.

 

[BigDave]

I'm a little confused by this because I matched the manufacturer number right off of Supermicro's site:

That indicates the Micron flash chip part number used on the DIMM, it's still a RDIMM not a UDIMM (unbuffered)
which is the type required by that board. I'll admit it's confusing sometimes

 

[BigDave]

In case you 're not thoroughly confused about this new technology, read this...
http://download.intel.com/support/ssdc/hpssd/sb/nvme_boot_guide_332098001us.pdf

 

[Ericloewe]

I'm a little confused by this because I matched the manufacturer number right off of Supermicro's site:

I'm fairly certain that whatever vendor you're looking at is wrong. The part number seems to be for UDIMMs.

 

[artlessknave]

mirror would be better than raidz2 with 4 disks; raidz2 with 4 disks pretty much defeats the entire purpose of raidz2 since its the exact same storage space as a mirror of 4 disks. raidz1 resilvering has zero redundancy while stressing every single drive to rebuild the redundancy.
currently you CANNOT expand a vdev, so a raidz1/2 of 4 disks will always be a raidz1/2 of 4 disks (supposedly vdev resize is on the way in zfs but afaik no ETA)

it has been argued that mirrors are superior to raidz as there is no parity calculations whatsoever; a resilver consists of straight out cloning to new drives, which with large drives can be alot faster than a parity rebuild (rebuilding a raidz1/2 can vastly increase the chance of another drive or more failing); the number of disks in a mirror vdev can be changed, while they cannot be with raidz (other than misc spares/cache etc), but this comes at the cost of 1/2 usable space
http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

you can expand a raidz based pool by adding another raidz pool, however, this can be really easy to mess up by completely nuking all pool redundancy with a missconfig, and you CANNOT remove a vdev once its added

https://forums.freebsd.org/threads/23127/