building new FreeNAS need advice

rlorfing

Cadet
Joined
Jun 10, 2018
Messages
3
hello,
I am some what new to FreeNAS and kind of a noob. I have read over the tuning guides and hardware guides. My hardware is as follows..
SUPERMICRO X10SRL-F
Case:NR40700
E5-2690v3 12 cores 24 threads 2.6ghz
256gb ecc (32gb x 8 pc4-2400t-rb2-11)

Raid1
boot disks/OS FreeNAS
x2 120 ssd kingston

Raid 0
larc2 cache
x2 Dual M.2 PCIE Adapter for SATA or PCIE NVMe SSD PCI-e 3.0 x 4 Host Controller Expansion Card
x2 2tb ssd m.2 read 3400 / write 3000

48 4TB sas Enterprise drives
44 disks in raid 10 - 4 drives as spare

SIL LOG - Disabled


Mellanox
3 10g dual sfp+

9240-8i SAS Controller
 
Joined
Oct 18, 2018
Messages
969
Hi @rlorfing welcome to the forums.

One thing that would be helpful would be if you told us a bit about what you want to do with your NAS. Knowing your plan really helps narrow in the hardware requirements. There are some basics though that I can mention.

Also, as a side note. You may want to check out this terminology primer. I mention it because FreeNAS has no such thing as raid1. This may seem like a nit but it is very important to avoid confusion. In general it is best to keep in mind that FreeNAS exposes pools for storage. A pool is composed of one or more vdevs stripped together. vdevs are composed disks. If a single vdev within a pool fails you lose the pool thus redundancy at the vdev level is important. FreeNAS allows for single-disk vdevs, mirror vdevs, raidz1, raidz2, and raidz3 vdevs all with different redundancy and performance characteristics. Raid1 is most similar to a pool with 1 mirror vdev. Raidz1 is most similar to raid5 but reads more similar to raid1.

Anyway, on to the equipment list.

Raid1
boot disks/OS FreeNAS
x2 120 ssd kingston
This is a great idea. Mirror boot devices on SSDs is quite common.

Raid 0
larc2 cache
x2 Dual M.2 PCIE Adapter for SATA or PCIE NVMe SSD PCI-e 3.0 x 4 Host Controller Expansion Card
x2 2tb ssd m.2 read 3400 / write 3000
My typical suggestion with respect to L2ARC is to first see if you even need it. You can always add an L2ARC after the build. If you don't need it, save the money. My other specific suggestion here is to consider using SATA SSDs rather than NVMe. The reason is that if you're going to use up PCIe lanes for storage you will want to use that for SLOG storage if that becomes necessary.

48 4TB sas Enterprise drives
44 disks in raid 10 - 4 drives as spare
I assume by RAID10 you mean 1 pool composed of 20 mirror vdevs? Stripping mirror vdevs is a common configuration and can improve IOPs. Regarding spares, the way I look at them is this. Most disk replacements are not due to outright disk failures. In those cases a spare won't help you. In large arrays you want to replace disks right when they show signs of trouble because otherwise you put extra risk on losing the pool. Spares do becomes useful though if you suspect there is a change you're not going to be able to replace a fully failed disk for a while and want to rely on the spare. I would guess a single spare is more than enough. What I do rather than use spares is do a full burn in of replacement disks, then put them back in static bags and store them safely for use when a disk shows any trouble. I replace disks before the outright fail anyway. All that said, the spares won't hurt you.

SIL LOG - Disabled
When you say disabled do you mean you do not intended to use a SLOG or that you intended to disable sync writes?

Mellanox
3 10g dual sfp+
The general advice is to stick with Chelsio or Intel NICs in the 10g category. I don't know much about Mellanox though. It may be worth double-checking that FreeNAS has drivers and see what reliability and performance others have experienced. Just because drivers exist doesn't mean the support is good. It would be a bummer to build a best of a machine like that and end up with poor network performance.

9240-8i SAS Controller
You'll want to be sure to flash this to IT mode. Hardware raid and software raid don't mix well. You could save money likely and pick up an HBA that doesn't have any RAID functionality to begin with. In order to make full use of the 12Gb/s speed of that card or a similar one you'll want to be sure you have a 12Gb/s backplane. I believe the case you listed does not. That matters because if all 40 drives are working at a time 8 lanes of 12Gb/s gives you ~300MByte/s per drive. That gets cut in half if you use a 6Gb/s backplane. These are just back of the napkin speeds. YMMV.

Anyway, I hope this helps. Tell us a bit more what you plan to use the system for and I'm sure folks can offer more specific suggestions.
 

rlorfing

Cadet
Joined
Jun 10, 2018
Messages
3
Thank you very much for all your recommendations. I am simply going to use this FreeNAS for is for storage for my home/VMWare environment.
I have a HP3000 blade enclosure with 8 G8 blades for my esx environment. Mostly all Ubiquiti gear for all my networking, 10gb switch US-16-XG.
 

rlorfing

Cadet
Joined
Jun 10, 2018
Messages
3
I have 2 backplanes.. both 24 drives on each side.. could I use a second controller, 1 backplace in one controller and 1 backplane in the other controller?


You'll want to be sure to flash this to IT mode. Hardware raid and software raid don't mix well. You could save money likely and pick up an HBA that doesn't have any RAID functionality to begin with. In order to make full use of the 12Gb/s speed of that card or a similar one you'll want to be sure you have a 12Gb/s backplane. I believe the case you listed does not. That matters because if all 40 drives are working at a time 8 lanes of 12Gb/s gives you ~300MByte/s per drive. That gets cut in half if you use a 6Gb/s backplane. These are just back of the napkin speeds. YMMV.

Anyway, I hope this helps. Tell us a bit more what you plan to use the system for and I'm sure folks can offer more specific suggestions.
[/QUOTE]
 
Top