NAS/MediaServer - build critique and recommendations is appreciated.

[I4C6XFQM]

Mobo: 1x Supermicro X10SDV-4C+-TLN2F (MBD-X10SDV-4C+-TLN4F-O)
RAM: 2x Samsung DDR4 32GB 2,133GHz (M393A4K40BB0-CPB)
HDD: 6x HGST Deskstar NAS 8TB (0S04012)
Case: 1x Fractal Design Node 304 (FD-CA-NODE-304-BL)
PSU: 1x Corsair RMx Series RM650x (CP-9020091-EU)
Boot: 2x Sandisk Ultra Fit 64GB
Fan: 1x Noctua NF-A14 PWM (NF-A14 PWM)
Fan: 2x Noctua NF-A9x14 PWM (NF-A9x14 PWM)
Etc: 1x Delock USB 2.0 pin header female > USB 3.0 pin header male

Planning to run the 6 drives in RAID-Z2.
Server will be mainly used for storing Movies, Films, backups with CrashPlan and function as a Media Server. The Media Server (Sonarr/Radarr/Emby/etc/etc) will run in docker containers either directly from FreeNAS or in a linux VM.

 

[Ericloewe]

Sounds reasonable.

 

[Jailer]

Looks like a good choice of hardware.

 

[Arwen]

You might consider a cheap M.2 SSD PCIe, (non-SATA, since you need all 6 ports), instead of 2 x USB Flash drives.
The Intel 16GB Optane one is cheap;
https://www.newegg.com/Product/Prod...20167426&cm_re=m.2_ssd-_-20-167-426-_-Product
SSDs tend to be more reliable than USB Flash drives, so you would need only 1. (Helpfull, since that system board only has 1 M.2 slot :).

Of course, I don't know what's available in the Netherlands... (but would not mind visiting!)

 

[Stux]

You want to make sure you have an active CPU fan/heatsink

You will want a USB3 to usb2 motherboard header connector
E.g.: http://m.ebay.com/itm/USB-3-0-20-Pi...-female-Adapter-for-motherboard-/321102147967

Consider upgrading the front intake fans to 92mm (iirc) Noctua fans

Consider upgrading the rear exhaust to a 140mm Noctua.

You will want to run a fan controller script, because the little CPU fan is obnoxious full tilt :)

(I really should post my XeonD/node 304 tuned fan config ;))

Booting off a pair of USBs is fine, but a cheap m2 NVMe will allow you to install/play with ESXi.

 

[I4C6XFQM]

You might consider a cheap M.2 SSD PCIe, (non-SATA, since you need all 6 ports), instead of 2 x USB Flash drives.
The Intel 16GB Optane one is cheap;
https://www.newegg.com/Product/Prod...20167426&cm_re=m.2_ssd-_-20-167-426-_-Product
SSDs tend to be more reliable than USB Flash drives, so you would need only 1. (Helpfull, since that system board only has 1 M.2 slot :).

Of course, I don't know what's available in the Netherlands... (but would not mind visiting!)

Thanks for the tip. Didn't notice the board had m.2.
The 32GB Optane (MEMPEK1W032GAXT) is about the same price as the 2 USB's.

You should visit, we have fields with cows and some mills but thats about it :p

 

[I4C6XFQM]

You want to make sure you have an active CPU fan/heatsink

You will want a USB3 to usb2 motherboard header connector
E.g.: http://m.ebay.com/itm/USB-3-0-20-Pi...-female-Adapter-for-motherboard-/321102147967

Consider upgrading the front intake fans to 92mm (iirc) Noctua fans

Consider upgrading the rear exhaust to a 140mm Noctua.

You will want to run a fan controller script, because the little CPU fan is obnoxious full tilt :)

(I really should post my XeonD/node 304 tuned fan config ;))

Booting off a pair of USBs is fine, but a cheap m2 NVMe will allow you to install/play with ESXi.

Sharp eye, ill surely get an USB3>USB2 header connector.

That mobo (MBD-X10SDV-4C+-TLN4F-O) should come with an active CPU fan according to the supermicro naming scheme, the '+' in '4C+' implies it. Can't find the right supermicro page, but the Q&A on Newegg also says so.
https://www.newegg.com/Product/Product.aspx?Item=9SIA5EM5GH7485

Depending on the noise/heat I was already planning to replace the fans with nocuta fans, if you can post your tuned config that would be awesome.

About ESXi, how would that work ? Second partition on m.2 or run FreeNAS in a VM on ESXi ? Wouldn't i need a HBA for that to work since zfs want direct control over the harddisks ?

 

[Stux]

Sharp eye, ill surely get an USB3>USB2 header connector.

Yeah... I've built your system pretty much ;)

That mobo (MBD-X10SDV-4C+-TLN4F-O) should come with an active CPU fan according to the supermicro naming scheme, the '+' in '4C+' implies it. Can't find the right supermicro page, but the Q&A on Newegg also says so.
https://www.newegg.com/Product/Product.aspx?Item=9SIA5EM5GH7485

Except when they don't follow that scheme ;), you also mentioned this board: X10SDV-4C-TLN2F, which doesn't.

Depending on the noise/heat I was already planning to replace the fans with nocuta fans, if you can post your tuned config that would be awesome.

Yeah, so the built in Fractal fans are non-controllable. And you need to run them all at full speed to cool 6x 7200pm non-he drives.

Or, you can upgrade them to the Noctua's I mentioned, and then you can run the cpu at full speed, and all the HDs can be stress tested without overheating either the CPU or HD. And it will be quieter... and then you can control the hd/exhaust/cpu fan speeds so that its quiet most of the time.

About ESXi, how would that work ? Second partition on m.2 or run FreeNAS in a VM on ESXi ? Wouldn't i need a HBA for that to work since zfs want direct control over the harddisks ?

You install ESXi onto the M2. Then you create a datastore on the M2. And then you create a virtual disk on the datastore, to boot FreeNAS. You can also boot FreeNAS off USBs by passing through the USB devices, and since 9.10 and 11 support USB3, you can even use the ESXi USB3 controller even though ESXi 6.5U1 thinks that FreeBSD doesn't support USB3.

You can pass-through the SATA controller. Yes, it works, and it should be safe. But then you can't use any of your six SATA ports in ESXi, but you probably want 6 HDs in FreeNAS anyway.

And if you want a SLOG, then you can use the PCIe port for that. Theoretically you can use up to 5 M2 drives in the system. A 2280 and 4 full-size 22110 drives on a carrier card in the x16 PCIe slot. I use a single HHHL AIC in the PCIe slot currently.

There are no SLOG suitable NVMe drives (currently) which fit the onboard 2280 M2 slot. So, I use that for booting ESXi. You can then use the virtual disks for swap/l2arc safely.

The point is, you can build your FreeNAS now, and use dual USBs to run it... and later you can play with ESXi quite safey using just that Optane card :)

The ONE trick, is you *must* install FreeNAS in BIOS mode not UEFI mode, if you plan to boot FreeNAS off USB inside ESXi, as ESXi does not support booting guests via UEFI.

 

[Arwen]

Thanks for the tip. Didn't notice the board had m.2.
The 32GB Optane (MEMPEK1W032GAXT) is about the same price as the 2 USB's.
...

You are welcome.

...
You should visit, we have fields with cows and some mills but thats about it :p

You have a city with lots of canals, Amsterdam. Plus, it's airport has 6 runways, (and funny layout), pretty rare for Europe. (But not the U.S... Denver's airport has 6 runways, pinwheel layout with potential expansion to 12.)

 

[I4C6XFQM]

Except when they don't follow that scheme ;), you also mentioned this board: X10SDV-4C-TLN2F, which doesn't.

Sorry about that, i missed the plus in the product name. Did list the correct SKU though, updated it.

Yeah, so the built in Fractal fans are non-controllable. And you need to run them all at full speed to cool 6x 7200pm non-he drives.

Or, you can upgrade them to the Noctua's I mentioned, and then you can run the cpu at full speed, and all the HDs can be stress tested without overheating either the CPU or HD. And it will be quieter... and then you can control the hd/exhaust/cpu fan speeds so that its quiet most of the time.

If that's the case ill just buy the noctua's from the start.

You install ESXi onto the M2. Then you create a datastore on the M2. And then you create a virtual disk on the datastore, to boot FreeNAS. You can also boot FreeNAS off USBs by passing through the USB devices, and since 9.10 and 11 support USB3, you can even use the ESXi USB3 controller even though ESXi 6.5U1 thinks that FreeBSD doesn't support USB3.

You can pass-through the SATA controller. Yes, it works, and it should be safe. But then you can't use any of your six SATA ports in ESXi, but you probably want 6 HDs in FreeNAS anyway.

Ow really ? Most resources I read before hand either said to not virtualize FreeNAS or if you do that you should use a HBA. First time hearing that you can just pass-through the onboard SATA controller.

And if you want a SLOG, then you can use the PCIe port for that. Theoretically you can use up to 5 M2 drives in the system. A 2280 and 4 full-size 22110 drives on a carrier card in the x16 PCIe slot. I use a single HHHL AIC in the PCIe slot currently.

There are no SLOG suitable NVMe drives (currently) which fit the onboard 2280 M2 slot. So, I use that for booting ESXi. You can then use the virtual disks for swap/l2arc safely.

The point is, you can build your FreeNAS now, and use dual USBs to run it... and later you can play with ESXi quite safey using just that Optane card :)

The ONE trick, is you *must* install FreeNAS in BIOS mode not UEFI mode, if you plan to boot FreeNAS off USB inside ESXi, as ESXi does not support booting guests via UEFI.

Think i may just end up installing ESXi from the get go and add SLOG in the case that i need more performance for VM's. ( I can do that right ? Add a SLOG after the volume is created ?)

 

[Stux]

Sorry about that, i missed the plus in the product name. Did list the correct SKU though, updated it.

If that's the case ill just buy the noctua's from the start.

Ow really ? Most recourses I read before hand either said to not virtualize FreeNAS or if you do that you should use a HBA. First time hearing that you can just pass-through the onboard SATA controller.

Think i may just end up installing ESXi from the get go and add SLOG in the case that i need more performance for VM's. ( I can do that right ? Add a SLOG after the volume is created ?)

https://forums.freenas.org/index.php?threads/build-report-node-304-x10sdv-tln4f.57116

Will follow up with my ESXi setup later.

And yes, you can add/remove slog/l2arc as you wish.

 

[I4C6XFQM]

https://forums.freenas.org/index.php?threads/build-report-node-304-x10sdv-tln4f.57116

Will follow up with my ESXi setup later.

And yes, you can add/remove slog/l2arc as you wish.

After reading your build log i think I'll hold of on ESXi and run FreeNAS on bare metal. 5MB/s for VM's without a SLOG, yeah... no thank you. Once I'm happy with the build and got everything setup I'll fork over the money for an M.2 and a Intel DC P3700 since that seems todo it's job pretty well.