Backup Pool Question - RaidZ1 or RaidZ2

[Scharbag]

I have 12@3TB drives arranged in as a 2x6 drive vDev Raid Z1 pool. I know that RaidZ1 is a bad choice but my rational is that this backup pool stores data that is primarily stored on a RaidZ2 array. Also, any truly irreplaceable data is stored on a cloud service.

My question is this: is it better to use the 2x6 drive Z1 setup or a 1x12 drive Z2 setup. Given that this pool does nightly incremental backups, performance is not a factor. I know that 12 wide is not recommended. Fundamentally though, which option is more resilient? Also, given that I have the 12 drives, is there any other backup pool arrangement that I should consider?

And a note about my backup pool: I have suffered a few catastrophic drive failure thanks to the oh-so-wonderful Seagate 3TB drives without any data loss on my backup pool. So far, Z1 has been fine for my needs (knock on wood!!).

Cheers,

 

[Arwen]

Have you considered an 1 x 12 drive vDev on RAID-Z3?

That takes care of the larger 12 drive vDev, though you do loose one disk's worth of storage
compared to your other 2 options.

Basically it's all about risk management. I'd personally be fine with a backup pool using a
single vDev of 12 drives using RAID-Z2. Using RAID-Z2 would be my preference.

 

[Scharbag]

Have you considered an 1 x 12 drive vDev on RAID-Z3?

That takes care of the larger 12 drive vDev, though you do loose one disk's worth of storage
compared to your other 2 options.

Basically it's all about risk management. I'd personally be fine with a backup pool using a
single vDev of 12 drives using RAID-Z2. Using RAID-Z2 would be my preference.

Yeah, I guess my thoughts when I chose the 2x6 was that if I needed to add space in the future, it would only be 6 drives at a time.

That said, as I replace failed drives, I am using 4TB disks. so I will gain 8TB of space over time due to that process. As for Z3, I thought about it but I needed the storage space. Likely the only thing ZFS does not do well is allow for simple scaling of capacity within a vDev. Boy, would it be nice if you could grow a vDev by adding disks!!!

Some day I will have to add a JBOD chassis in order to allow for room to grow, or start using 8TB drives :) (or delete stuff I rarely use).

Thanks,

 

[Robert Trevellyan]

Fundamentally though, which option is more resilient?

If you lose one disk from your 2x 6-disk RAIDZ1 scenario, there is a 5 in 11 chance that a 2nd disk failure kills the pool.
If you lose one disk from your 1x 12-disk RAIDZ2 scenario, there is no chance that a 2nd disk failure kills the pool.

EDIT: corrected the odds.

 

[depasseg]

there is a 1 in 11 chance that a 2nd disk failure kills the pool.

Wouldn't it be a 5 in 11 chance? (full disclosure, it's been awhile since I've played the odds.)

 

[Robert Trevellyan]

Oh, good catch. Much worse odds.

 

[Stux]

Have you considered an 1 x 12 drive vDev on RAID-Z3?

That takes care of the larger 12 drive vDev, though you do loose one disk's worth of storage
compared to your other 2 options.

Basically it's all about risk management. I'd personally be fine with a backup pool using a
single vDev of 12 drives using RAID-Z2. Using RAID-Z2 would be my preference.

I'd go a single z2 in preference to two z1s

 

[Scharbag]

Wouldn't it be a 5 in 11 chance? (full disclosure, it's been awhile since I've played the odds.) :)

Yikes. Yes, there is a 45% chance that a second failure will murder the pool. Now that that is on paper, I have to rethink my strategy!!

Thanks guys.

 

[Mr_N]

Z2 or Z3 should be the only choices, next freenas update should ban using Z1 for any HDD's above 2TB :P

 

[Robert Trevellyan]

Yikes. Yes, there is a 45% chance that a second failure will murder the pool. Now that that is on paper, I have to rethink my strategy!!

Yes, those are the bare numbers, showing that RAIDZ2 is much more reliable than RAIDZ1, but as always, it's a matter of making a decision informed by both the numbers and your risk tolerance.

Unless you're determined to lose your data (or your SMART checks, SMART tests and email alerts aren't working), you wouldn't wait for a 2nd disk failure before replacing the first failed disk. If you have easy access to the machine, you can keep a burned-in cold spare on the shelf. If you don't have easy access to the machine, you might consider a hot spare shared between vdevs.

 

[Mr_N]

/me pats his 2x burned in cold spares

 

[Scharbag]

Z2 or Z3 should be the only choices, next FreeNAS update should ban using Z1 for any HDD's above 2TB :p

After seeing the above math, I would agree!!

I am rebuilding my shiny new RaidZ2 backup pool as I type. Will take a while to replicate/rsync the 16TiB of data :) (averaging about 130MBps on rsync, took about 2.5 days)

Thank you all again for pointing out what should have been more obvious. RaidZ1 is just not that awesome with large disks.

Cheers,

 

[Scharbag]

Yes, those are the bare numbers, showing that RAIDZ2 is much more reliable than RAIDZ1, but as always, it's a matter of making a decision informed by both the numbers and your risk tolerance.

Unless you're determined to lose your data (or your SMART checks, SMART tests and email alerts aren't working), you wouldn't wait for a 2nd disk failure before replacing the first failed disk. If you have easy access to the machine, you can keep a burned-in cold spare on the shelf. If you don't have easy access to the machine, you might consider a hot spare shared between vdevs.

My production pool has a hot spare. Love it cause I may not always be close to home/awake when a drive fails.

ZFS is neat.

Cheers,