12 disk best practice

[Oko]

After a 4.5 years of faithful service it is time for me to rebuild 9.2.1.9 FreeNAS boxen as I am running out of space on the RAIDZ2 pool of 10x3TB drives. The box has 12 slots for 3.5" data drives. I already bought 8TB WD Gold Enterprise drives for all available slots and one spare drive. The new box, just like the original, will act as a main file server accessed from Linux workstations via NFSv3 by few dozen people at any time. I am trying to maximize capacity and performance taking moderate risk of failure. I am not considering RAIDZ1 as an option.

A standard recommendation in the past was that one shell use power of 2 number of disks + parity i.e.

RAIDZ2 pool/VDEV should consist of 4, 6, 10
RAIDZ3 pool/VDEV should consist of 5, 7, 11

HDDs. In terms of the storage maximization I could create one RAIDZ2 VDEV consisting of 12 disk. IIRC I will lose 2 for parity leaving me with 10 disk of useful space. That is going contrary to the past recommendation. Two other options I am contemplating is having 2 RAIDZ2 pools of 6 disk each in which case I would probably maximize performance but lose 4 HDDs in total for parity. Other option I am thinking off is
11 disk RAIDZ3 with a hot spare. In that case I also lose 4 disks. I am not sure about the performance but I am three disks foul tolerant + I have a hot spare.

Any recommendations? Am I missing any other good options?

 

[Stux]

I would suggest 12-way Raidz2 is probably a bit too wide, and raidz3 is also slow.

I’d probably go the dual 6-way Raidz2

 

[danb35]

I’d probably go the dual 6-way Raidz2

Ditto.

 

[Jailer]

Yup, same here.

 

[Kevin Horton]

A standard recommendation in the past was that one shell use power of 2 number of disk + parity i.e.

That old recommendation is no longer valid, as data compression is now recommended, and selected by default. This means the number of bytes needed to store a given amount of data is variable, which invalidates the assumptions that drove that old recommendation.

 

[c32767a]

That old recommendation is no longer valid, as data compression is now recommended, and selected by default. This means the number of bytes needed to store a given amount of data is variable, which invalidates the assumptions that drove that old recommendation.

We use a lot of 15 drive Raid-z3s for large file storage and have good success. Only caveat is if you ever want to do a capacity upgrade, you're doing it groups of 15 drives.. :)