Expand Zpool -> Reallocate data?

[Sir.Robin]

I have a question :)

Lets say you have an ZPOOL with 5 drives in raidz (or whaterever) witch is barely full of stuff (data).
You need more space so you expand the pool with another vdev with another 5 drives in raidz. Doubling the capacity.

Now all is nice and dandy... but:

You now have 2 vdev's where one is nearly full and the other is empty(?)!

How do you reallocate data so that not only new data but also the existing data gets spread across both vdevs/all 10 drives?

 

[paleoN]

You now have 2 vdev's where one is nearly full and the other is empty(?)!

How do you reallocate data so that not only new data but also the existing data gets spread across both vdevs/all 10 drives?

1. Take half the data and COPY it to a new folder.
2. Delete the original half you copied.
3. MOVE the copied data back to the original folders.

Given that ZFS is a COW filesystem anything more may not be very useful. The best thing to do would be to delete the array, create a new array and restore the data from backup. If you know you will need more space then you are better off expanding sooner rather than later.

 

[jgreco]

That's approximately correct, but to be slightly more comprehensive, you should see if you can manage to get most of the first vdev filled (with null-filled files or whatever) first, because if I recall correctly, ZFS will continue to allocate space out of the first vdev even if it's "mostly full". There's some logic floating around in the allocation code to try to consider the case of an over-used vdev within a pool, but the obvious thing to do is to force the issue manually. It can't allocate blocks that are already allocated. ;-)

Some experimentation is probably required to determine how best to accomplish this.

 

[Sir.Robin]

Allright. Thank you both for the replies.

I was hoping there were some command like you have on NetApp (Data ONTAP is BSD based too) systems for this.

Won't be an issue for me for a looong time though... just qurious :)

 

[jgreco]

There's always a lot of exciting talk in the ZFS community about these sorts of things, about what could be done and what should be done and what will be done, but historically actual implementation has usually significantly lagged or been lacking. ZFS is at a bit of a crossroads in any case, due to what's happening at Oracle, and it'll be interesting to see what - if anything - happens in the near future regarding further feature development...

 

[Sir.Robin]

So Oracle owns ZFS? :-o
And what is "happening" at Oracle?

 

[jgreco]

Oracle bought Sun. Sun "owns" ZFS, at least as much as anyone can own something with an open source license.

I keep hearing different things. Among the more optimistic (and probably realistic):

http://forums.freebsd.org/showthread.php?t=27159

 

[bollar]

I may be able to report back with what happens when the first VDev fills soon. When I got to ~75% / 3T left I added another 18T Vdev and then added another 9T of data. About 7T wound up on the new Vdev, but over 2T is in the old one and I'm coming down to ~ 800G left.

There's another 4T to transfer, and I'll report back with what happens.

 

[Sir.Robin]

That would be great bollar! Wery interesting indeed :)

 

[bollar]

That would be great bollar! Wery interesting indeed :)

So, basically almost everything went to the new Vdev. The free space on the old Vdev fluctuated between 800-900G, but I don't think it ever got lower than 800G. Currently Vdev1 has 885G available and Vdev2 has 11.7T available.