FreeNAS or a RAID card; And R10 or Z2

[Justin Aggus]

What I have now:
4U Supermicro 36 Bay X8DTL-3F
12x 8TB WD red
8x 6tb WD red
12x 3tb WD red
64GB ram
2x X5650 CPU's
Intel 10Gb/s lan card

This will be used mainly for backups (sequential write), with the occasional instant recovery (Random read). (OS will run from the NAS until recovery is finished)
I was about to order an expensive LSI Raid card when I remembered FreeNAS.

RAID10 FreeNAS should be able to max out a 10Gb/s Lan right? (Sequential write)

What about in RAIDZ2?
I know the rule of thumb for write speed is the slowest disk * the number of Vdevs
But I also know a real raid controller can do sequential writes in raid5/6 at close to (Number of Disks - 1)*(Speed of slowest disk)

I'm guessing this will come down to Sync vs Async writes?
This system is only used for backups, and the system will be in a proper datacenter (Meaning odds are this system will never see a power outage)
So I think Async writes should be fine.

I usually use SMB, but could switch to FTP or SFTP if it would be better.

 

[Ericloewe]

Just what format are these backups? Most importantly, are they file-based or big archived blobs?

 

[Justin Aggus]

Virtual Machine backups
MS SQL backups - Full, Diff and Logs
Physical Machines (image based)

So mostly big blobs.

 

[Ericloewe]

That's going to be sub-optimal with ZFS. If you still wish to take advantage of ZFS' features, your best bet is to follow the advice laid out for iSCSI users: Mirror vdevs, plenty of RAM, generous L2ARC.

For the amount of storage you seem to need, I'd evaluate bumping RAM to 128GB or more.

 

[Justin Aggus]

Based on this:
https://calomel.org/zfs_raid_speed_capacity.html

Z2 should work fine for me.

 

[Ericloewe]

Based on this:
https://calomel.org/zfs_raid_speed_capacity.html

Z2 should work fine for me.

The word "fragmentation" appears zero times. That is enough to dismiss those tests as not representative of a real workload.

 

[Justin Aggus]

The word "fragmentation" appears zero times. That is enough to dismiss those tests as not representative of a real workload.

Surely ZFS would not have issues with fragmentation with the workload I described?

Maybe with a database or VM workload.

 

[Ericloewe]

Unless you're going to transfer the whole blob every single time, that backup is going to fragment much like the original database would - only slower.