RAIDZ expansion, it's happening ... someday!

[m0nkey_]

If you've not already seen it, RAIDZ expansion is happening. The OpenZFS project tweeted this earlier today.

RAIDZ expansion (most requested ZFS feature ever?) is coming, courtesy of @freebsdfndation. Sneak preview at OpenZFS DevSummit!

https://twitter.com/OpenZFS/status/921042446275944448

So what does this mean and why you should be excited? Well, lets say you have a RAIDZ pool and you want to add more drives, well this will let you do this. For example, take an existing 5 drive RAIDZ2 and add 3 more drives to the existing pool into the existing vdev, giving you a 8 drive RAIDZ2 pool. No more adding vdevs.

At this time, it's not available in FreeNAS yet. I will repeat, IT'S NOT AVAILABLE IN FREENAS! So don't go adding drives just yet. Please continue to create vdevs as normal until this heads downstream.

This is very exciting.

 

[dlavigne]

Looks like they let the cat out of the bag early as the official announcement will be next week during OpenZFS.

The exciting part for FreeNAS users is that FreeBSD will be the reference implementation and FreeNAS will be one of (if not the) first appliance to benefit from it.

 

[m0nkey_]

Looks like they let the cat out of the bag early as the official announcement will be next week during OpenZFS.

The exciting part for FreeNAS users is that FreeBSD will be the reference implementation and FreeNAS will be one of (if not the) first appliance to benefit from it.

This will be a huge boon to FreeNAS and its users.

 

[nightshade00013]

Yes very interesting. Will be able to take my 7 drive RaidZ3 to a 11 drive and double my storage space.

 

[Ericloewe]

Looks like they let the cat out of the bag early as the official announcement will be next week during OpenZFS.

The exciting part for FreeNAS users is that FreeBSD will be the reference implementation and FreeNAS will be one of (if not the) first appliance to benefit from it.

Well, it's been out of the bag for a month or so now. It's on the OpenZFS developer summit timetable.

 

[scwst]

The presentation is supposed to be on Oct. 25 (see http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit_2017) by Matt Ahrens, one of the original ZFS designers. Interestingly, the roadmap (http://open-zfs.org/wiki/Roadmap) had him also working on "Top-level vdev removal", which would also be big, but that doesn't seem to be a topic at the summit?

Anyway, great things are happening with OpenZFS. At-rest encryption is going to be a major, major point once those of us who are paranoid will feel secure enough to actually use it ...

 

[Ericloewe]

which would also be big

Big, but not huge due to its limited scope. It only works with single-disk vdevs and involves creating a virtual vdev on the rest of the pool (yes, the virtual virtual device part does not escape me, but that seems like an accurate description). It's really only useful if you accidentally added a disk and need to remove it.

 

[garm]

This might get me to give up mirror vdevs!! :D

 

[Stux]

Good point. Wonder if you can go from 3 disk z1 to 4 way z2

 

[Mlovelace]

Yes very interesting. Will be able to take my 7 drive RaidZ3 to a 11 drive and double my storage space.

"Yes, but this one goes to 11..." :D

 

[Arwen]

Good point. Wonder if you can go from 3 disk z1 to 4 way z2

My guess is no.

<Wishful thinking>But, that would make a nice companion project. Add disk for additional parity. Even going from striped pool to RAID-Z1 vDev, (and potentially later to -Z2, then even -Z3).</Wishful thinking>

Gee, I might as well ask for single disk vDev removal, or native OpenZFS at rest encryption...

 

[scwst]

Gee, I might as well ask for single disk vDev removal, or native OpenZFS at rest encryption... :)

I'm guessing it's been posted here before somewhere, but there is a great video by Tom Caputi where he explains step by step how native encryption works with OpenZFS: https://www.youtube.com/watch?v=frnLiXclAMo . Even I think I understand it. Primary development seems to be on Linux, but sooner or later this should make it to FreeBSD/FreeNAS.

 

[joeschmuck]

This will be so nice when adding a single drive will not put your pool at high risk. Well I guess anyone could screw it up and likely will.;)

 

[Stux]

On RaidZ1->RaidZ2 being usefull:

Matthew Ahrens‏ @mahrens1 Oct 19
I agree, but that's not part of this project. Unfortunately it doesn't *completely* absolve you of advance planning ;-)

later:
Matthew Ahrens‏ @mahrens1 23h23 hours ago
It will rebalance the data so that it's evenly across all disks in the RAIDZ group, and a big chunk of free space at the end of each disk.

Cool.

So, no it doesn't go from Z1->Z2, but it means you could start with 4 way RaidZ2, add a disk at a time, rebalancing each time (which is fantastic), grow to say 8 way... then add another 4 way Z2, and grow that to 8 way 1 disk at a time... Means no more having to buy 8 disks at a time!

And hopefully the next step will be to be able to increase RaidZ level over time too...

Assuming it works on RaidZ2.

 

[danb35]

It's really only useful if you accidentally added a disk and need to remove it.

...but I've lost count of how many folks here have needed this exact capability. Pity none of them think of reading TFM before trying to replace a failing disk...

 

[Ericloewe]

...but I've lost count of how many folks here have needed this exact capability. Pity none of them think of reading TFM before trying to replace a failing disk...

Yes, it solves 99% of the problem.

I really don't understand it. The GUI throws warnings at people and they just ignore them. Where have they been desensitized to warning messages?

 

[m0nkey_]

Yes, it solves 99% of the problem.

I really don't understand it. The GUI throws warnings at people and they just ignore them. Where have they been desensitized to warning messages?

The problem at the moment is that error is way to easily dismissed. It could use improvement.

 

[danb35]

I really don't understand it.

Nor do I, and thus far, despite my asking repeatedly, none of them has chosen to explain the thought process. The best I can figure is that they're just dead set that they have to be able to do the replacement through the Volume Manager. I guess providing a way to undo it is better than nothing, though (once FreeNAS integrates that into the GUI--how many years have we been waiting for them to let users add mirrored disks through the GUI?).

 

[danb35]

The problem at the moment is that error is way to easily dismissed.

It isn't easily dismissed. It requires you to switch to manual setup, and then use different steps to once again tell FreeNAS to do what you think you want to do (but don't actually want to do). I guess making them type, "I understand I'm doing something incredibly stupid" might slow them down further, but it's not as though they can simply click a "confirm" button to force the procedure.

 

[nightshade00013]

The GUI throws warnings at people and they just ignore them. Where have they been desensitized to warning messages?

Ever been to a theme park? Ever actually read the receipt that you got when you paid. You basically have no recourse for ANYTHING that may happen to you or anyone you are responsible for. And in a lot of states failing to read and follow the directions on the signs puts you personally at legal risk, some even verbal instructions by an operator have to be followed. So I could basically tell you to strip to ride a ride and LEGALLY you are required to do it. Not to mention that the rides can easily kill, most are just industrial machines with a pretty facade built around it. I ended up quitting because my supervisor was totally clueless about safety among other things and management didn't have the balls to fix the issues. I had no desire to be there when someone did get killed.

The reason why I mentioned this is because I worked at one once for a couple years and no matter how many times you post something or say something or play a message people will STILL do the opposite. I can post signs that say "once you enter this park you are transferring your home and all monies or other worldly possessions to the park owners," and most people will not blink at it until after it has happened.

Another more crude way to put it is, you can't fix stupid.