How come ZFS doesn't support a RAID-1Z1 or RAID-15 design? Should it?

[LarchOye]

So, with all of the fear accompanying the possibility that losing a VDEV in a ZFS Pool means losing the entire Pool- I'm really surprised to see that no one has suggested implementing a sort of RAID-Z1 for VDEV's (as opposed to the RAID-0 that ZFS essentially creates across all of the VDEV's in a pool).

I currently run four mirrored vdev's with two drives a piece, and two hot-spares. 10x8tb HGST HUH78080AL4200 SAS3 disks on an LSI 9305-24i. All the disks were acquired second hand (off of ebay) at various times over the past 3 years, from various vendors. Some of the disks are closer to shot than others, and what I've been dealing with A LOT lately (not sure if it's due to Truenas Scale, my 9305-24i being shoddy, or some of my drives actually having issues): Is individual mirrors becoming degraded, the hot-spares taking over as they should, resilvering happens, and either another disk will want to throw errors, or the hot spare will- and then my pool sits degraded until I can figure out which disk it's actually talking about. (I love how the sdi, sdh, sdl, sdg, etc. labels change every time I reboot. I also love how I can't see the drive label, serial number, and other crucial information ON THE SAME SCREEN all at once. HINT HINT ;)

So anyways, I was thinking.

Why not append ZFS's striping functionality to include some type of parity calculation- and reserve a portion of each mirrored pair for rebuilding in the event of a VDEV failure? I know that people normally stripe their RAIDZ2 VDEV's for added performance, and I'm not looking to nest RAIDZ by any means... Just thinking that adding a level of VDEV failure tollerance might be handy for more than just myself someday.

Call it RAID-1Z, or RAID-15, or even RAID-1Z2, RAID-16 for double parity striped VDEV's!

Now please go ahead and tell me why this is a ridiculous idea, how it would never work, and why it isn't the NEXT NATURAL EVOLUTIONARY STEP for ZFS.

Thanks! :)

 

[jgreco]

Feel free to submit your patchset.

But seriously, it's easy to come up with "wouldn't it be nice if..." wishlists, but finding competent people with the motivation to do such work is not easy. ZFS is much more complex than your typical UNIX or Linux filesystem, and funding of development is sustained by a very small list of companies like iXsystems. Making major architectural changes is probably not in the cards.

 

[ChrisRJ]

In addition to what @jgreco so rightly said, we also need to look at this from a business continuity perspective (which also has very many non-technical aspects, BTW). If I have the need that the relatively low risk of loosing a vdev in an enterprise environment is not an option, there are better ways. Chances are that loosing a vdev in such an environment is caused not by too high a number of disks failing simultaneously. Instead we look at HBAs, disk enclosures, or entire servers that fail. While some of those scenarios could be mitigated with a vdev mirror, there are two challenges: 1) the overall complexity increases, which in itself is a considerable risk from a design and operations perspective; 2) some scenarios simply cannot be covered. So overall it is better in such a case to have redundancy at the system level and probably even a synchronously mirrored NAS in a remote data center.

I do not dispute that there would be benefit for some home or SMB scenarios. But ZFS was not designed for those. And if I am really worried about loosing an entire pool, then I would recommend to have one pool per vdev and organize things with symlinks or other means (if that is necessary at all).

Thanks, @LarchOye, for triggering this interesting thought! Please read my comment not as "brushing you off", but more in the spirit of the bigger picture that comes into play (at least for me).

 

[Ericloewe]

You can get halfway there today by using copies=2 or =3. ZFS does its best to place copies on different vdevs and pools should now be importable - read-only - if there's a missing vdev. With copies=2 or =3, you would be able to retrieve all data.

 

[Davvo]

Either go with Z2 or Z3 (right, you don't want to, why?) or add more hotspares.
Buying half new half used could help.

 

[sretalla]

Now please go ahead and tell me why this is a ridiculous idea, how it would never work, and why it isn't the NEXT NATURAL EVOLUTIONARY STEP for ZFS.

OK...

1. It's too costly compared to just doing your VDEVs right... you waste an entire VDEV to redundancy on top of it already having redundancy inside it.

2. It's complex to work out how you would do that without making additional VDEVs just a mirror of the present ones, hence more waste each time or only able to add pairs of VDEVs at a time as a new mirror... very ugly.

3. Write performance penalty... do I need to spell that out? Who wants their pool's write performance cut in half (because it needs to write twice as much data/twice as many IOPS for each transaction now) just because they can't manage their VDEVs properly?

It's a terrible idea with no vision to scale-out as the alternative... why wouldn't you just use SCALE and gluster to keep your "mirror" (additional VDEV) on another box? (hence providing the true system-level redundancy mentioned above for your trouble).

 

[Arwen]

You can get halfway there today by using copies=2 or =3. ZFS does its best to place copies on different vdevs and pools should now be importable - read-only - if there's a missing vdev. With copies=2 or =3, you would be able to retrieve all data.

I started to think the same thing, but you beat me to it..

 

[Etorix]

(I love how the sdi, sdh, sdl, sdg, etc. labels change every time I reboot.

Blame Linux for that… On CORE, ada labels are rather stable across reboot (but not guaranteed).

I also love how I can't see the drive label, serial number, and other crucial information ON THE SAME SCREEN all at once. HINT HINT ;)

What's missing from Storage>Disks?

Why not append ZFS's striping functionality to include some type of parity calculation- and reserve a portion of each mirrored pair for rebuilding in the event of a VDEV failure?

You don't really have enough drives to go there, but dRAID seems to offer much of what you want to achieve: Resiliance at pool level rather than at vdev level.

 

[Ericloewe]

On CORE, ada labels are rather stable across reboot (but not guaranteed)

Actually, it is supposed to be, as long as the hardware doesn't change.