RAIDZ Recommendation

[kris1351]

I am doing this as a definite overkill build, but it was hardware that was taken out of the racks recently that I decided would be best fit for our NFS backup server for a new set of servers. I actually have 2 of these I will implement just for this purpose

Supermicro 3u w/X7DBE MB
2x E5310 QC 8MB Cache
16GB ECC Ram
ARC 1260 16 port in JBOD w/battery and 1GB cache
10Gbe network card
16x1TB Seagate HDD
16GB USB Flash for OS drive

I am only doing NFS mounts for backups and restores and it will take care of about 20 servers with its capacity. What would be the best RAIDZ recommendation that I could achieve? I am looking for the best write performance, but I do not want to worry about this box. I was thinking multiple RAIDZ or RAIDZ2 volumes that were added to each other, but wanted to see what others thought.

 

[louisk]

If you actually are planning on getting 10G out of it, you will need 1 or both of the following:
1) Something like a ZeusRAM (http://www.stec-inc.com/product/zeusram.php) for the ZIL (best practice is to use 2 of these mirrored)
2) 32-256G SSD for the L2ARC (cache) - depending on how much you want to be able to cache, vary the size accordingly.

With out #1, you will not get 10G because the ZIL will get bottlenecked by your storage (even if it is a dedicated spindle).

The fastest I think would be mirrors, as that gives you the most vdevs in the pool, but you could use RAIDZ or RAIDZ2 as well. Just make sure that your data spindle count is a power of 2 (2, 4, 8) and I would keep the total number of spindles in the vdev at or below 10. So if you have 16 ports, you could have 3x 5 spindle RAIDZ (4 data spindles) and 1 spare.

 

[kris1351]

The 10Gbe is because bonding is fickle and quite buggy for the most part even with ether channels setup. This is purely a NFS backup server for multiple machines, huge speed is not necessary anything 100MB/s or better will suffice honestly which should be able to be done with very little tuning considering the number of devices. Capacity and redundancy followed by speed are the goals. When you say mirrors I assume you are speaking of Raid10?

 

[louisk]

We have a machine like that at work. 36 spindles, 2T SATA ea. 5x 7 spindle RAIDZ with 1 hot spare. We just got 10G interface for it, as we're bottlenecking on the gig-e when we run backups (800-900Mbit).

Yes, the equiv in ZFS of RAID10 is a pool of mirrors. ZFS stripes data over the vdevs in the pool, so it works out the same way when you use mirrors.

 

[kris1351]

So 3x5spindle RAIDZ with a hot spare will be my best ratio of capacity/redundancy/speed. Thanks for the help on this.