Advice for very large volume build

[Kljajoni]

Hello,

I would like to ask you for some help regarding proper volume sizing for backup server. I have custom SuperMicro FreeNAS build with LSI 9211-8i card (IT mode), 64GB RAM, 1Gbit network (upgrade to 10Gbit planned later this year).

- Drives that are available for me to use are 18 x 4 TB WD Re Enterprise, i can extend it to max 24 drives if needed.
- Purpose of the server - storage for backups only (lz4 compression, no deduplication, couple of datasets on volume)
- Backups are done via rsync or Bvckup2 software (effectively backup period last at most 90-120 minutes, depends of the amount of data that was created that day)

So, usage of this server will be pretty low. Is it smart to create one huge RAIDZ2 volume to maximize drive usage (16 + 2 drives) or is it smarter to split it to 2 x RAIDZ2 volumes with 10 drives (8+2) ?

I do not know how will resilvering work with this large volume even if I am using enterprise drives. If anyone have suggestion for another RAIZ setup with 4TB enterprise hard drives, please advise

 

[ricky_198]

The freenas manual states that using anything more than 12 is not recommended. I would certainly advise splitting the volumes.
Zfs Primer:
"Using more than 12 disks per vdev is not recommended. The recommended number of disks per vdev is between 3 and 9. If you have more disks, use multiple vdevs."

 

[danb35]

If it were up to me, I'd probably suggest using three, six-disk vdevs. Performance will be better (more IOPS), resilver time will be lower when you need to replace a disk, you can easily expand the pool by adding a fourth identical vdev, and if you want to expand the pool by replacing drives, you'll only have to replace six at a time. As a fallback position, buying two more drives and doing two, ten-disk vdevs would be OK, but that's about as wide as we like to see any single vdev. A problem with that is that it only leaves four bays open in your server, so later expansion isn't as convenient.