Looking at rebuilding my NAS

[Visseroth]

So due to running my ST2000DL004 drives configured in a 512byte configuration due to over site I'm looking to offload my data and rebuild my array though I'm not exactly sure as to the best configuration for the best redundancy and reliability.
The server I'm looking at supports 16 hot swap bays and I currently have 12 drives so I'm looking to add 4 more drives and using RaidZ3
I'm not sure if it is best to configure 4 pools, 4 drives each in a RAIDZ3 or to configure 2 pools of 8 in a RaidZ3.
Input?

 

[danb35]

A pool of four drives in a RAIDZ3 would be incredibly wasteful of capacity--you'd be giving up three drives' worth of capacity in each pool. On the up side, you could lose up to 12 drives without any data loss. If it were mine, I'd probably set up a single pool with two 8-disk RAIDZ2 vdevs. It's not an optimal configuration for performance, but I expect you'll still be able to saturate your network connection without issue.

 

[Ericloewe]

So due to running my ST2000DL004 drives configured in a 512byte configuration due to over site I'm looking to offload my data and rebuild my array though I'm not exactly sure as to the best configuration for the best redundancy and reliability.
The server I'm looking at supports 16 hot swap bays and I currently have 12 drives so I'm looking to add 4 more drives and using RaidZ3
I'm not sure if it is best to configure 4 pools, 4 drives each in a RAIDZ3 or to configure 2 pools of 8 in a RaidZ3.
Input?

A pool of four drives in a RAIDZ3 would be incredibly wasteful of capacity--you'd be giving up three drives' worth of capacity in each pool. On the up side, you could lose up to 12 drives without any data loss. If it were mine, I'd probably set up a single pool with two 8-disk RAIDZ2 vdevs. It's not an optimal configuration for performance, but I expect you'll still be able to saturate your network connection without issue.

A vdev of 4 disks in RAIDZ3 is impossible. You need at least 2 disks + parity disks (3 in the case of RAIDZ3)
RAIDZ3 vdevs with 8 disks are also not recommended (use 5, 7 or 11 disks with RAIDZ3).

 

[Visseroth]

So why the odd number of disks on a raidz3? So am I correct in saying that the best practice would be 2 pools of 8 in a raidz2 and at least 16GB of RAM?

 

[danb35]

For optimum performance, a RAIDZ vdev should consist of 2^n disks plus parity. IOW, 2, 4, or 8 disks, plus 1 (for RAIDZ1), 2 (RAIDZ2), or 3 (RAIDZ3) for parity. As with most other things, though, there are other factors to consider, specifically capacity and data security, not to mention other performance questions. As I mentioned above, if it were my system, I'd probably set it up with a single pool, consisting of two 8-disk RAIDZ2 vdevs. ZFS really likes for all the storage to be combined in one pool, and almost any reason you'd have for wanting separate pools could be addressed by using datasets instead. This configuration does not comply with the 2^n+p rule, but I suspect the performance hit will be minimal, if it's even noticeable.

 

[Ericloewe]

A vdev of 4 disks in RAIDZ3 is impossible. You need at least 2 disks + parity disks (3 in the case of RAIDZ3)
RAIDZ3 vdevs with 8 disks are also not recommended (use 5, 7 or 11 disks with RAIDZ3).

Recommended vdev sizes are 2^n+p, with n={1, 2, 3} (larger vdevs are possible but definitely not recommended) and p the number of parity drives (The RAIDZ level)

This means:
RAIDZ1 (which is a bad idea if you value your data): 3, 5 or 9 disks
RAIDZ2: 4, 6 or 10 disks
RAIDZ3: 5, 7 or 11 disks

Other configurations lead to misaligned ZFS sectors (kinda like the 4k drive issue a few years ago), which leads to performance issues. It may also lead to significant losses of capacity (even if it doesn't, fragmentation will be noticeable sooner than with optimum configurations).

 

[cyberjock]

Ericloewe,

To save yourself the time of explaining it to every user you might want to just point them to the noobie presentation I wrote. ;) I think I've seen you mention the philosophy for pool generation a few times and you could probably save yourself some time (as well as potentially answering other unasked questions they may have).

 

[Ericloewe]

Ericloewe,

To save yourself the time of explaining it to every user you might want to just point them to the noobie presentation I wrote. ;) I think I've seen you mention the philosophy for pool generation a few times and you could probably save yourself some time (as well as potentially answering other unasked questions they may have).

You're right, I think I'll start a list of stuff I can just copy-paste to answer common questions, with the link to the guide at the end.

 

[Visseroth]

Well thank you guys for breaking it down for me. I'm understand better now and thank you for being patient with me. I will hit up those newbie sticky threads and do some more reading but I have a much better grasp on what I should and shouldn't be doing with my drives now and it's all much much clearer!