SM X10/X11SDV + 10GB Home LAN

[imperiosus]

Hi,

I am a newbie with Freenas, reading a lot as I go, so please don't go harsh on me.
I've set my crosshairs (some parts already purchased) on the following HW:
-X10SDV-4C-TLN2F or X11SDV-4C-TLN2F or X11SDV-4C-TP8F (the latter bcs of 12xSATA and multiple LAN ports - possible to run pfsense as well, but kinda out of budget)
-U-NAS 810A chassis (bought)
-Seasonic FlexPSU 350W (bought)
-64GB ECC DDR4 RDIMM (bought)
-8x WD RED 8TB (bought)
-Samsung 250GB SATA3 SSD for Intent Log (i know its not best choice, had a spare)
-Samsung 970 EVO 250GB NVME for Cache (bought)
-Supermicro 32GB SATADOM for boot (have from before)
-Noctua fans, cables, etc... (bought)
-LSI9300 HBA

I want to make use of 10GbE links, the NAS will be used mainly for storing media files such as movies, music, pictures.
So the questions are the following...
Is the above setup capable of delivering? Should I go with X10 or X11 boards?
How should I setup the pool to get the best performance? Raid-z with striping?

I am just pulling the trigger on the mobo now, and your inputs would help a lot.

Edit: wrong thread..sorry, had multiple tabs opened.

 

[Chris Moore]

possible to run pfsense as well

What are you planning to use for your hypervisor?

-Samsung 250GB SATA3 SSD for Intent Log (i know its not best choice, had a spare)
-Samsung 970 EVO 250GB NVME for Cache (bought)

Neither of these are good choices but you want the faster one for what you are calling a Log which I am thinking you mean SLOG and the slower one for what you are calling Cache which I am thinking you mean L2ARC. You should review these guides for terminology:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://www.ixsystems.com/community...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://www.ixsystems.com/community/threads/terminology-and-abbreviations-primer.28174/

-LSI9300 HBA

You don't need 12GB SAS to drive mechanical drives. You would be perfectly fine using a controller like this:
https://www.ebay.com/itm/Dell-H200-...i-P20-IT-Mode-ZFS-FreeNAS-unRAID/163142320696
or this
https://www.ebay.com/itm/Dell-H310-...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601
as long as they have been modified to use the latest version of the IT mode LSI firmware, which these have.

Is the above setup capable of delivering?

Not likely, because you don't have enough drives, unless you set them up as mirror vdevs. A single RAIDz2 vdev is not going to be fast enough to saturate 10Gb networking.

Should I go with X10 or X11 boards?

Not really part of the question. You can do 10Gb with an X9 generation boar if you disk subsystem is up to the task. The disks are the slow part and having SLOG and L2ARC is not needed if all you are using this for is playing some media files.

Raid-z with striping?

Again, the terminology, I am not sure what you are talking about. Your chassis only has eight drive bays, so you don't have many options. If you are using RAIDz2, you only have a couple options, a single vdev of eight drives or two vdevs of four drives each. Neither of those is likely to saturate a 10Gb network, but you can give it a try. If you set your pool up as four mirror vdevs, you should be able to fully saturate a 10Gb network link, but it would be right at the border of doing it when the pool was empty and the performance degrades as the pool fills up.

 

[imperiosus]

Chris,

Super tnx for the quick reply. Appreciate it.

What are you planning to use for your hypervisor?

ESXi is the one I was planing on.

Neither of these are good choices but you want the faster one for what you are calling a Log which I am thinking you mean SLOG and the slower one for what you are calling Cache which I am thinking you mean L2ARC. You should review these guides for terminology:

Got it, seems I've made some wrong choices here. Will eat through the newbie guides.

You don't need 12GB SAS to drive mechanical drives. You would be perfectly fine using a controller like this:
https://www.ebay.com/itm/Dell-H200-...i-P20-IT-Mode-ZFS-FreeNAS-unRAID/163142320696
or this
https://www.ebay.com/itm/Dell-H310-...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601
as long as they have been modified to use the latest version of the IT mode LSI firmware, which these have.

Was more concerned about the heat dissipation. Thought that 9300 was a newer model, smaller tech ASIC. But then again its tuned up to be faster=more heat. Thanks for the links, already bought it - H310 (I have a thing for higher numbers).

Not really part of the question. You can do 10Gb with an X9 generation boar if you disk subsystem is up to the task. The disks are the slow part and having SLOG and L2ARC is not needed if all you are using this for is playing some media files.

Not just play, but move those files around. I hoard data on various PC's and usually copy in bulks to NAS. That's where I need the performance. Those can usually go to 200GB+ per copy and a SLOG seems a perfect fit. Reads I guess are not that important.

Would it be better to exchange the Reds for Exos (5400RPM vs 7200RPM)? I was considering heat (and power consumption), which can be a pain in this U-NAS case.

For the X11 I like the 12xSATA and the multiple LAN ports, perfect for a pfsense setup, if that even makes sense to be running in a setup primarily intended for NAS.
I might think about a dedicated device for pfsense, but those having 10GbE cost a fortune.

Again, the terminology, I am not sure what you are talking about. Your chassis only has eight drive bays, so you don't have many options. If you are using RAIDz2, you only have a couple options, a single vdev of eight drives or two vdevs of four drives each. Neither of those is likely to saturate a 10Gb network, but you can give it a try. If you set your pool up as four mirror vdevs, you should be able to fully saturate a 10Gb network link, but it would be right at the border of doing it when the pool was empty and the performance degrades as the pool fills up.

Yes, RAIDz... need to dig into that, totally in the dark.
What kind of speeds do you think are reachable with those scenarios? Would something else be better?
For me, these would be the priorities:
-P1 - Size of the volume
-P2 - Speed
-P3 - Redundancy (just striping is a nogo)

Thanks a lot again!

BR

 

[Chris Moore]

Would it be better to exchange the Reds for Exos (5400RPM vs 7200RPM)? I was considering heat (and power consumption), which can be a pain in this U-NAS case.

The 7200 RPM disks can be quite hot. I have a mix in my 48 bay chassis and the 5400 RPM drives tend to be around 30°C where the 7200 RPM drives tend to be around 42°C. I don't think the little performance difference would be worth the heat in the long run. Rotational speed of the drives is a greater influence on the random IO than on sequential IO. If you are using this system to host virtualization, you will want more vdevs but if you are just storing data like media, for sequential IO, you are still going to have a mechanical limitation of the drive throughput holding you back. I have not looked these drives up but the fastest ones I have read specs on lately are rated at 240MB/s of sustained transfer and that only ever works when they are doing large sequential read or write. If you have a random workload where there is track to track seeking by the head, you will cut that in half at best. I see average transfer rate on my systems at work around 110MB/s per drive. The thing about vdevs in a ZFS pool is that each vdev is (very roughly) limited to the performance of a single drive that makes up the vdev. So, if you want to be sure you can always satisfy a 10Gb network, you need to have eight to ten vdevs.

For the X11 I like the 12xSATA and the multiple LAN ports

This only makes sense if you are going to connect those SATA ports to the drives and have no ESXi. If you want to have ESXi, so you can run a virtual machine or two, you need a SAS HBA that you can pass through to the FreeNAS that would be running inside a virtual machine, so the FreeNAS (by way of the HBA) can have direct access to the drives. If your drives are on a SAS HBA, the number of SATA ports on the system board becomes much less important. You could attach a SSD to one SATA port for ESXi to use as a data store which could host the disk file for your PfSense and possibly other VMs.

I might think about a dedicated device for pfsense, but those having 10GbE cost a fortune.

Why do you need 10Gb for PfSense? Does your internet connection run at 10Gb?

Very good reading about implementing ESXi on bare metal and installing multiple systems including FreeNAS inside the virtual environment:

Build Report: Node 304 + X10SDV-TLN4F [ESXi/FreeNAS AIO]
https://www.ixsystems.com/community...node-304-x10sdv-tln4f-esxi-freenas-aio.57116/

"Absolutely must virtualize FreeNAS!" ... a guide to not completely losing your data.
https://www.ixsystems.com/community...ide-to-not-completely-losing-your-data.12714/

Virtually FreeNAS ... an alternative for those seeking virtualization
https://www.ixsystems.com/community...ative-for-those-seeking-virtualization.26095/

FreeNAS 9.10 on VMware ESXi 6.0 Guide
https://b3n.org/freenas-9-3-on-vmware-esxi-6-0-guide/

 

[imperiosus]

Chris tnx again. Highly appreciate all of it, read through most of the given links.

I've stumbled upon multiple A2SDV boards from SM using the Denverton CPU.

Now I am in dilemma what to choose. Those Atoms are way more power efficient, support multiple 2.5/5/10G LAN (saw 4x10GbE model with miniSAS ports) and are set at a similar price range. Trade-off: No turbo and some run low as 1.7GHz (Samba runs single threaded), and missing some instructions, which I don't think I have a need for.
Would those be better for my intended use (NAS + pfSense)?

Thanks in advance.