Theoretical question about 24 disk array, setup

[atob37]

Thinking about getting one of those NORCO RPC-4224 24 bay enclosures, and setting up a server for video storage. If I wanted a good level of redundancy, would creating 4 vdevs of 6 disks each in raidz2 configuration be a safe bet? I am going for a combination of redundancy and storage capacity here over speed.

Thanks!

 

[jgreco]

Six disks in RAIDZ2 gives four disks worth of usable space. Four sets of that gives sixteen disks worth of usable space out of twenty four disks.

Eleven disks in RAIDZ3 gives eight disks worth of usable space. Two sets of that gives sixteen disks worth of usable space out of twenty two disks.

 

[cyberjock]

I have that exact case in my basement. Currently I'm running 2 zpools. One is 18x2TB in RAIDZ3 and the other is 6xRAIDZ2.

Personally, I am completely comfortable with the RAIDZ3 as wide as it is because I have spares in a box ready to be dropped in at even the smallest sign of trouble and I don't hesitate to do replacements if I even think there is a problem. My server uses ECC memory and everything is overspeced and cooled enough that multiple fan failures will not cook my server. I get nightly emails of all SMART info on all of the drives and I read them every day and I run bi-weekly scrubs. I also do religious backups of the important data to a second FreeNAS server so if somehow I had 4 disks fail in my RAIDZ3 I'd lose the unimportant data, but I'm completely content with that choice.

Keep in mind that I am very committed to my server, I trust ZFS and I am very aggressive with finding and fixing any kind of small issues that I ever have with my servers. I doubt many people on this forum would recommend a RAIDZ3 of 18 drives for the average person. But since I am extremely aggressive I feel that I can comfortably handle the risk of possibly losing all of the data on the zpool(minus my backed up data) from 4 disks failing.

Yes, I am an engineer from a past life. :P

 

[atob37]

Six disks in RAIDZ2 gives four disks worth of usable space. Four sets of that gives sixteen disks worth of usable space out of twenty four disks.

Eleven disks in RAIDZ3 gives eight disks worth of usable space. Two sets of that gives sixteen disks worth of usable space out of twenty two disks.

So I could go to two 12-disk arrays in RAIDZ3, each using 11 disks with 1 hot spare?


cyberjock:

I have that exact case in my basement. Currently I'm running 2 zpools. One is 18x2TB in RAIDZ3 and the other is 6xRAIDZ2.

Did you swap out fans or anything to improve reliability of the stock Norco components? Any problems with the backplanes?

Thanks, both of you!

 

[cyberjock]

I did buy the backplane with the larger fans. I then bought 3 of these(I think this is the right size.. verify before you buy). I like those because they don't have bearings in the traditional sense so should last far longer than other fans. They're pretty quiet, they don't have the same CFM as other fans, but I bought them because they are sufficient and won't fail on me. Been using those fans for almost 3 years and had no failures out of 37!

I also replaced the case fans with 2 of those type.

Edit: Hot spares aren't "hot" on FreeNAS yet. I don't recommend them yet because they aren't hot. If I really feel the need to keep a spare I either go 1 disk wider and go with RAIDZ3, then you have a hot spare that is ALSO redundant, or I keep them on a shelf where nobody will touch them.

 

[jgreco]

So I could go to two 12-disk arrays in RAIDZ3, each using 11 disks with 1 hot spare?

Ultimately, yes. ZFS RAIDZ sizing rules make 11 a good size for a RAIDZ3. Manufacturing 12 and 24 drive chassis mean a drive bay is left over. Sadly, FreeNAS doesn't support fully automatic hot sparing at this time, but since you probably don't want to get out the sheet metal tools to add another bay when FreeNAS 9 finally appears, you can be prepared anyways. ;-)