24 bay chassis to grow into

[Zombie Twiglet]

After buying a 24 bay 4u chassis impulsively I though I should probably do some actual research into the rest of the parts I need to setup a decent FreeNAS box. After spending a few hours a night researching this over the last few nights by reading the fantastic guides/info throughout the forums I've come up with what I think will serve my needs.

Already have:

Chassis: X-Case RM424
Boot Drive(s): Corsair Force LS 60GB

Still to buy:

MOBO: Supermicro X10SRL-F
CPU: Intel Xeon E5-1620 V4
PSU: Corsair RM1000X
RAM: Crucial 64GB Kit (2 x 32GB) DDR4-2400 RDIMM
HBA(s): LSI SAS 9207-8i [x2]
HDD(s): 8TB WD Red WD80EFZX [x4]
SLOG: Intel SSD DC P3700 [400gb PCIx]

Future:

I'll probably be upgrading to 10gb networking gear in the next 12 months

Use Case:

NFS Target for ProxMox (moderate use, yes I've read the warning, I'm willing to put the time in tuning)
Media Storage
No VMs/Containers on the NAS

Setup/Expansion Plans:

I'm planning on running 2 disk vdevs then adding more as I need the space (upgrading the RAM as I go if/when I need to) as my usage is pretty linear over time. I have 5x 4tb (~12TB data) drives in a Synology NAS at present the plan is to get the 4x 8TB drives above, migrate the data, get a 6th 4TB drive and add them in as 3 more mirrored vdevs. NAS will be hooked up to a

Questions:

1. Is this a sensible plan with the 2disk mirrors? It seems like the easiest way to add capacity as I need it and from my understanding should be quicker to upgrade the disks in individual vdes in the future rather than shelling out for 20odd drives from the get go which will sit unused (space wise) for ages.
2. Is having different sized vdevs in a pool a bad idea? Can't seem to find much info on this anywhere other than "It's ideal to have similar sized ones".
3. Any suggestions on a redundant PSU? I have dual UPSes so would like to have redundant power but my I don't know much about PSUs and my googling hit a wall (I'm a liunx sysadmin but haven't touched hardware in about 7 years!)

 

[Chris Moore]

If you are looking for room to grow, try this out: http://www.ebay.com/itm/201614424117

The SAS backplane is SAS2 compliant and works with drives up to 8TB according to someone that has one of these cases. They have used 8TB drives in their case.

Only needs one SAS controller per backplane, so only 2 SAS controllers needed.

Already has redundant power supplies ...

 

[Chris Moore]

1. Is this a sensible plan with the 2disk mirrors? It seems like the easiest way to add capacity as I need it and from my understanding should be quicker to upgrade the disks in individual vdes in the future rather than shelling out for 20odd drives from the get go which will sit unused (space wise) for ages.
2. Is having different sized vdevs in a pool a bad idea? Can't seem to find much info on this anywhere other than "It's ideal to have similar sized ones".
3. Any suggestions on a redundant PSU? I have dual UPSes so would like to have redundant power but my I don't know much about PSUs and my googling hit a wall (I'm a liunx sysadmin but haven't touched hardware in about 7 years!)

No, I would say it is a bad plan. I would go with RAID-z2 as you can sustain 2 (TWO) disk failures and loose no data. Go ahead and make a bigger pool than you need and add another vdev to the pool when you need to have more space. I have around 12TB of share space now and am only using around 5.5 TB. When I get close to needing more space, my plan is to migrate onto larger disks. Migration to larger disks in your existing vdev is really simple and easy.
You can look at the way I have mine configured (in my signature below) as an example but it isn't the only way.

You would want to keep all your vdevs the same number of drives, but you could have one vdev of 2TB drives, another vdev of 4TB drives and another vdev of 8TB drives all in the same pool.

Also, that chassis I posted, it has everything you need to start with down to redundant power supplies. I am very tempted to buy a couple of those myself even if I never fill it with drives and you can always change the system board out for a more modern board. It will fit an EATX board.

 

[Stux]

Depending on the backplane on the case you only need two HBAs as there are ten Sata ports on the mobo.

The e5-2609 is a crappy Xeon. It's got a slow clock speed (bout half speed) so it'll bottleneck smb

Better to look at an e5-16xx CPU.

 

[Chris Moore]

After buying a 24 bay 4u chassis impulsively I though I should probably do some actual research into the rest of the parts I need to setup a decent FreeNAS box.

Sorry, I missed the part about having already bought the chassis.

Well, there is always next time.

As for the power supply, you don't want to play silly games with the power because everything in the chassis depends on that being quality and reliable. I would buy one of the ones from the manufacturer's site and have done with it. The only inexpensive way to get into a server chassis is to buy a used one and the only way to be sure the parts will fit is to go with the OEM parts. Sorry.

 

[Stux]

I have one of the corsair rm1000x PSUs in my Norco 24 bay system. Has so far proven good, although who can tell on a product with a 10 year warranty after just a couple of months.

 

[Zombie Twiglet]

Sorry, I missed the part about having already bought the chassis.

No worries, I did kinda jump the gun on the chassis unfortunately. Lesson learned.

No, I would say it is a bad plan. I would go with RAID-z2 as you can sustain 2 (TWO) disk failures and loose no data

Why is RaidZ2 better than mirrored vdevs? The way I understand it if I have say a raidz2 vdev with 6 disks then I can lose 2 disks, if I have 3x 2 disk vdevs I can lose 3 disks without data loss (assuming one from each vdev), rebuilds also don't take nearly as long with mirrors which should reduce the risk of data loss due to drive failure during a rebuild. I also understand mirrored vdevs give better performance than raidz vdevs and provide better performance if a vdev is degraded and I'd rather not shoot myself in the foot out of the gate if I'm using FreeNAS as a NFS target for VMs from ProxMox.

I'm fairly new to ZFS so till learning so if I am wrong I will happily listen, it's just everything I've read till new says mirrors suit my needs better than RAIDZ.

Edit: Clarity.

 

[Stux]

With mirrors you can lose the right three disks but also the wrong two disks can cause a failure.

With z2 you can lose any two disks.

Go with mirrors if you want it's far more flexible. Just buy pairs of the most affordable drives you can, and when you run out of bays or when drives fail just start upgrading drives.

 

[Robert Trevellyan]

if I have say a raidz2 vdev with 6 disks then I can lose 2 disks, if I have 3x 2 disk vdevs I can lose 3 disks without data loss

If you have 6 disks in RAIDZ2 and you lose one disk, the chance of the next disk failure destroying the pool is 0%.
If you have 6 disks in mirrored pairs and you lose one disk, the chance of the next disk failure destroying the pool is 20%.

I also understand mirrored vdevs give better performance than raidz vdevs

This depends on your workload. More vdevs deliver more IOPS, wider vdevs deliver faster streaming performance.

 

[SweetAndLow]

Do you need a dual socket cpu? You should check out my build in my signature it is better than what you are going to build. Do you need the x10sri over the srl? Also use mirrors whenever running a vm storage workflow. You might want to also check out a p3700 PCI ssd for a slog. This will increase write performance.

Sent from my Nexus 5X using Tapatalk

 

[Robert Trevellyan]

This will increase sync write performance.

FTFY.

 

[Zombie Twiglet]

Do you need a dual socket cpu? .. Do you need the x10sri over the srl?

As far as I can see the only difference between the SRi and SRL is the NIC controller and the PCI Express slots. The SRL is a tenner cheaper so I may as well swap the SRi for the SRL, thanks.

You might want to also check out a p3700 PCI ssd for a slog

I'll take a look, thanks.

Go with mirrors if you want it's far more flexible. Just buy pairs of the most affordable drives you can, and when you run out of bays or when drives fail just start upgrading drives.

That was the plan, my storage needs are fairly linear so with RAIDZ I would be buying a lot of space upfront that I wouldn't be using any time soon.


Thanks for all the help/advice guys, have updated the main post with the new specs!