BUILD Optimal number of drives for basic home build.

[Valcno]

I have finally decided to take the plunge into the FreeNAS world to simplify my home storage.

My main goal is to simply have bullet proof network storage for my various computers instead of storing everything on random computers. It would also be nice to be able to run plex with 1-2 streams going at a time.

I have read a bunch of other people's builds and wanted to post mine in case there was something I missed.

PCPartPicker part list: http://pcpartpicker.com/p/yRmm4D

CPU: Intel Core i3-4370
Motherboard: Supermicro X10SLL-F
Memory: Crucial 16GB (2 x 8GB) DDR3-1600 Memory
Storage: 5 x Western Digital Red 3TB in RaidZ2 or 6xWestern Digital Red 2TB in RaidZ2
Boot Media:2 x SanDisk Cruzer Fit 16GB
Case: Fractal Design Node 804
Power Supply: EVGA SuperNOVA G2 550W
UPS: CyberPower AVR CP825AVRG (Already purchased)

My current issue is the number of drives to put into the system. With my budget I can either go with 6x 2TB WD Reds or 5x 3TB WD Reds.

I know according to the ZFS best practices doc it doesn't recommend going to Z2 until 6 drives and I also found this old thread that specifically mentions 5 Disk Raidz2 as being "Bad". The original source that the post referenced isn't active anymore so I don't know the extent of how "bad" it is. I would prefer the 5 disk solution since it would give me more usable space for basically the same price as the option.

If anyone has any comments on the build in general or how many drives I should go with it would be much appreciated.

 

[Yatti420]

I use a 6x2TB.. Follow best practices.. O

 

[rs225]

If you care about the data, you have a backup.

If you don't care to have a backup, then just go RaidZ1 w/ 5 x 3TB and be reasonably safe.(With scheduled scrub and occasional long-test, of course.)

 

[Sir_Slappy]

I don't have personal experience with that case but I do currently own 2 Fractal ARC Mini R2's and have used a few of the Define cases for family member builds and one thing I can say is that the stock fans that come with that unit (Silent Series R2) are not really all that good. I would personally recommend getting some better fans for it. I use and recommend Noctua IndustrialPPC fans myself. If I was building that system I would go:

2x http://www.newegg.com/Product/Product.aspx?Item=N82E16835608052 for front intake
2x http://www.newegg.com/Product/Product.aspx?Item=N82E16835608038 for rear exhaust

Or if you wanted to stick with Fractal Design fans then you could do:

2x http://www.newegg.com/Product/Product.aspx?Item=N82E16835352022 for front intake
2x http://www.newegg.com/Product/Product.aspx?Item=N82E16835352021 for rear exhaust

Either setup should help give much better airflow through the case and the board you chose has 5 fan headers that would allow all four fans plus one on your heatsink. I do understand that the Noctuas that I pasted are 3000rpm but with PWM you can tune them down if need be. I myself prefer better airflow at the cost of higher noise but you may be different. Hope this helped!

 

[Mirfster]

Never been a fan of RaidZ1 (or Raid 5) for that matter. Especially with disk drives being so much lager nowadays. IMHO, best all around is RaidZ2 for most purposes.

 

[Valcno]

@Sir_Slappy thanks for the tip on the air-flow, I will keep that in mind when I do testing if I see drive temps going too high. The nas is going to be positioned right next to a white noise machine in my office so any additional fan noise wouldn't be noticeable.

Looks like 6 drives is the way to go. Newegg is running a deal on the 2TB Reds, $85 per, so that would bring the costs more in line with what I was looking for.

I ordered the case since it is going to take the longest out of all the components to get to me. I will probably order the rest of the parts in the next few days.

 

[nightshade00013]

If I was looking at drives it would be $ per TB vs cost per drive. 2TB drives @ 85 is 42.50 per TB, 3TB drives at 110 is 36.67 per TB. I would still do at least 6 drives in a raidZ2 though if I were planning to swap to even larger discs I would squeeze 7 drives somehow.

I guess that I am a little different and do a lot of things over an extended period of time and would rather wait a little longer to have something even better so I don't have to throw more money at it right away.

 

[Valcno]

Good point on trying to make it 7 drives. Unfortunately the budget is pretty fixed and I doubt I would be able to expand it. I checked the pricing history on all of the components and there didn't seem to be any huge variations over time so waiting wouldn't really allow me to squeeze an extra drive out of the savings if I tried to play the waiting game.

I was thinking about cost per TB more as cost per usable TB VS cost per raw TB.

With 5x3TB that is 550 for 9TB in raidz2, about $61 dollars per TB.

6x2TB is $515, 5@$85 and 1@$90 since Newegg has a 5 drive limit, for a total of 8TB of space in raidz2. This is about $64.50 per TB.

Going with the best practice compliant option for a 5ish% increase in cost per TB seems more valuable for this particular case than having an extra TB of raw space.

 

[Mirfster]

You could always consider buying used as well. I get a lot of my drives from eBay which are used/remanufacturered and have only had 1 fail in several years. Not that this is for everyone, but works for me. Just do the proper burn-in testing to be safe.

Example cost for a 3TB HGST with 1 year warranty (just make sure to keep a record with Serial #s and Order #) is ~$90.00. That comes out to ~$30.00/TB. 2TB HGSTs should run you about ~$60.00 used w/warranty...

/Yeah, I know I run 7200 RPM drives.. I don't mind the heat cuz I am so darn cool... :p

 

[nightshade00013]

I am running HGST drives as well. I got a steal on the 4TB drives at 35/TB brand new. It was cheaper than the WD Red drives at the time. They have since went back up but I was glad to get them when they dropped. By the time I add a second vDev I am hoping that the 6TB drives will be in the 20/TB range

 

[Mirfster]

By the time I add a second vDev I am hoping that the 6TB drives will be in the 20/TB range

I am currently waiting for SSDs to keep dropping (thinking ~ November) before I go that route. Gonna need like 50x 2TB. But if new spinny drives are in the 20/TB range, that would be awfully tempting...

 

[Robert Trevellyan]

I know according to the ZFS best practices doc it doesn't recommend going to Z2 until 6 drives and I also found this old thread that specifically mentions 5 Disk Raidz2 as being "Bad".

That advice is no longer relevant.

Choose the number of drives that gives you the right compromise based on your storage needs, budget and desired redundancy level.

 

[Yatti420]

That advice is no longer relevant.

Choose the number of drives that gives you the right compromise based on your storage needs, budget and desired redundancy level.

Just keep in mind vdevs with huge of drives and high densities with even raidz2 could be troublesome in the future.. You don't want to have so many drives in the dev or pool that it constantly has something wrong.. Keep it reasonable..

 

[Dice]

I might chip in, during my 'choosing amount of drives process', I found the bidulohms calculator hugely helpful in deciding between options that instinctively felt good/ reasonable.
https://jsfiddle.net/Biduleohm/hfqdpbLm/10/embedded/result/

 

[nightshade00013]

Well not to throw a major wrench into things but the 3TB Red's are down to 32.5/TB when you get a pair on NewEgg's shell shocker http://www.newegg.com/Special/ShellShocker.aspx?cm_sp=Homepage_SS-_-P5_2730220-_-03022016 Shall be tough to beat if you have the cashflow to get them now.

 

[Valcno]

The shell shocker actually simplifies things quite a bit. My budget for the disks was $550. I can get 6x3TB for only $30 over. I can probably make that difference up by driving to micro-center to get the processor instead of ordering it.

 

[rs225]

Never been a fan of RaidZ1 (or Raid 5) for that matter. Especially with disk drives being so much lager nowadays. IMHO, best all around is RaidZ2 for most purposes.

I see people using raidz2 as their justification for not having a backup, so I prefer those people be directed to raidz1+backup-drive(of critical data). I think it helps people grasp their true risk far more accurately. Raidz2 is only slightly better than raidz1, because it provides protection from only one possible type of loss: A drive dies, and then a second drive dies before resilver completes, but not a 3rd, 4th, 5th, or 6th drive, or the entire system. That is just flat out rare on a scrubbed pool.

The raidz2 voodoo comes from a misunderstanding of the BER math, which itself doesn't seem to accurately model to observed failures. A BER error means a block fails to read, not the entire drive. (Only RAID controllers abort the array on a single read failure, ZFS does not.) If the BER error rates were accurate, those of us with large, full pools should expect to see a read/checksum error on every scrub! I have never had a read/checksum error in five years of scrubs!

So, how exactly does a user benefit from filling a drive with an extra chunk of parity and no backup of anything?

That really is the unstated trade-off here.

Edit: Furthermore, once people have a true 100% backup, their interest in raidz2 suddenly vanishes, unless it is actually an availability situation.

 

[Dice]

^interesting!

 

[Mirfster]

Good points, however as you stated that is based on the System Owner/Admin making the fundamental mistake of not having any backups. Regardless of ones choice backups are a must. If someone is wholly depending on a single solution then they are already sorely mistaken. While not everyone can afford a complete On-Site, Near-Site and Off-Site solution at least getting two out of three is recommended.

All things being equal with appropriate backups being maintained/performed; then the actual questions should revolve around how much is the cost of downtime? To me it is a much better bargain to have paid for another drive for dual parity as opposed to trying to save a few dollars at the beginning.

 

[Robert Trevellyan]

I see people using raidz2 as their justification for not having a backup, so I prefer those people be directed to raidz1+backup-drive(of critical data).

Everyone needs backup of all data they don't want to lose, regardless of the level of redundancy they have in their pool.

Raidz2 is only slightly better than raidz1

This is nonsense.

The issue is how long it takes to resilver and the likelihood of encountering an error during that time. The twin goals are to improve data availability and reduce the chance of data loss from the pool. Restoring from backup is a last resort, and (I claim) sufficiently inconvenient for most FreeNAS users that extra redundancy beyond RAIDZ1 is well worth having, to minimize the chance of having to do so.