RAID 5 Support?

[ThePowerTool]

What's going on with support for RAID 5?

I love RAID 5 because with both striping and parity if a drive in a RAID 5 array fails I replace it and still have degraded access to the array which automatically rebuilds. AFAIK a HD failure in any other scenario is just ugly.

Is there another alternative where a HD failure is as easily resolved?

It seems like every 3 drives I buy one fails in the 1st 18 months (the rest last forever). I don't understand why I'm getting such high failure rates but RAID 5 sure makes this less painful.

Thanks!

 

[Yatti420]

Raid 5 / Raid-Z1 is dead is it not? Raid z2 is what you want.. As far as I understand it with hdd failure rates going up due to density going up.. Raid-Z1 will save you when the first drive goes.. but any read/write error during scrub (to rebuild) results in the loss of entire pool..

 

[ThePowerTool]

So the "new" support means you can replace a HD and recover but any minor failure ("read/write error during scrub") and you're back to restoring from backup?

 

[Yatti420]

So the "new" support means you can replace a HD and recover but any minor failure ("read/write error during scrub") and you're back to restoring from backup?

As far as I understand it yes.. You will lose the entire pool.. With hard drives becoming cheaper etc the chances of errors appear to be increasing.. So without a backup (hopefully recent enough) you would lose everything..

I would assume this would apply to a mirror setups aswell with 2 drives in a vdev.. What im not sure is if you added another mirror vdev (2drives) and mirrored the original vdev is it now a raidz2 technically? Or only Z1 .. Planning is very important for ZFS use..

 

[jgreco]

Why are we talking about RAID5?

ZFS does not support RAID5. It does support something that is conceptually similar, called RAIDZ1. Neither RAID5 nor RAIDZ1 are considered wise choices, because of the substantial risk to your data should you run across a multiple drive failure event.

ZFS supports all the way up to RAIDZ3, which allows up to any three disks in a vdev to fail without data loss. The idea is that maybe two of them could fail at the same time and then a read error on a third during the resilvering, and there is still no loss of data as it can all be rebuilt from parity.

 

[HoneyBadger]

As far as I understand it yes.. You will lose the entire pool.. With hard drives becoming cheaper etc the chances of errors appear to be increasing.. So without a backup (hopefully recent enough) you would lose everything..

Scrub is not the word you want, that's the "ZFS background check" to look for correctable errors.

Rebuilding an array after a failed drive is a "resilver" - and yes, if you encounter a URE (Unrecoverable Read Error) during resilver of a RAID-Z1 the whole pool is toast.

I would assume this would apply to a mirror setups aswell with 2 drives in a vdev.. What im not sure is if you added another mirror vdev (2drives) and mirrored the original vdev is it now a raidz2 technically? Or only Z1 .. Planning is very important for ZFS use..

ZFS uses its own decision-making algorithms to decide which vdev gets the data in the pool, so there's no option to "mirror the original vdev" when adding a second. The classic RAID analog would be a RAID10 - you've made two two-drive RAID1s and then striped them in RAID0 fashion.

 

[ThePowerTool]

Scrub is not the word you want, that's the "ZFS background check" to look for correctable errors.

Rebuilding an array after a failed drive is a "resilver" - and yes, if you encounter a URE (Unrecoverable Read Error) during resilver of a RAID-Z1 the whole pool is toast.

So does this mean RAID-Z1 is similar to RAID3 or 5 in that a URE during rebuilding a RAID3 or 5 array would also result in a complete loss? I just want to know if I'm comparing apples to apples.

 

[jgreco]

No. But it means that the affected data will be lost. This is not particularly good for ZFS because the system is sufficiently complex that there aren't tools like "fsck" for when your filesystem becomes inconsistent through loss of data. You're expected to be sufficiently competent to make sure that isn't an issue, both through appropriate redundancies and through backups.

 

[HoneyBadger]

So does this mean RAID-Z1 is similar to RAID3 or 5 in that a URE during rebuilding a RAID3 or 5 array would also result in a complete loss? I just want to know if I'm comparing apples to apples.

Technically no, as jgreco mentions you would only lose the data on the dead drive. But because there aren't really any tools or professional services for ZFS data recovery ... effectively yes, the whole thing is toast.

 

[ThePowerTool]

So would it be correct to conclude that if the requirement is for a HA solution that allows a failed HD swap-out without requiring restoring from backup and while maintining the integrity of the stored data there is no longer a solution found in FreeNAS and I'll have to use something else?

 

[jgreco]

No, of course it wouldn't be correct to conclude that. Why would you conclude THAT?

 

[jgreco]

Oh. I see, you didn't understand what was said.

What I meant is that an admin is expected to have knowledge of their tolerance for potential pool loss and is expected to engineer sufficient resources to guarantee redundancy remains available.

See my previous post. Basically if you want to be able to survive a dual drive loss event, you should probably actually pick RAIDZ3 so that you can lose two drives and still have a few unrecoverable read errors on the remaining disks without any chance of data loss.

That amount of redundancy is unusual among filesystems, and you might not find it in anything BUT ZFS.

 

[ThePowerTool]

The way I read it was "there's no fsck so if something goes wrong you would normally fix with fsck you'd better have a backup because that's your only option".

Your clarification points me to Z3 and it's a step better than what I was asking for because with Z3 you can lose 2 drives (I see Z3 requires a minimum of 5) and still continue and recover without data loss. My goal is HA, speed, and minimize the need for recovering/restoring from backups (which I do weekly based upon risk and data requirements).

I'll be digging into the docs on Z3 for my next upgrade.

Thank you very much!

 

[HoneyBadger]

The way I read it was "there's no fsck so if something goes wrong you would normally fix with fsck you'd better have a backup because that's your only option".

That's true; but the advantage of ZFS is that most things that would require a fsck in a "normal" filesystem (eg: ext4) are just handled transparently and automatically by ZFS. Even if there was a "zfsck" tool, regular fsck wouldn't magically fix a RAID5 with two missing drives either.

I can see how that might have been misleading at first.

 

[jgreco]

Yes, there's no fsck. fsck dates from a time when you didn't build redundancy and there were lots of bugs in filesystem code, disk read errors, etc. so errors were a fact of life.

The ZFS design is reliant on its data protection and self-healing properties, not only for your own data, but for its metadata as well. Corruption of the metadata is basically not really fixable after the fact, but it is pretty paranoid about that. You are expected to have redundancy in the form of multiple drives. Etc.

ZFS does a lot to guard your data, but you have to be responsible when designing the server in order not to sabotage it.

 

[HoneyBadger]

ZFS does a lot to guard your data, but you have to be responsible when designing the server in order not to sabotage it.

*cough cough* BUY ECC RAM *cough cough*