Data distribution "normalization" when adding vdevs?

[gsrcrxsi]

maybe normalization isnt the right word, since that might refer to something else in the context of ZFS/truenas. But what I mean is how evenly the data is spread across each disk or vdev.

my current pool consists of 8x 12TB disks arranged as 4x mirror vdevs. this originally started as a 4-disk array with 2x mirror vdevs. and I noticed at the time that when I added the other 4 disks and vdevs, new data was written to all disks, but reading back older, I could see that it was only getting activity on the first 4 drives. I didnt really bother with it then since the amount of data on only the first 4 drives was quite small.

now, I'm about 66% pool filled, so I'm getting to the point of expanding the array. I could maintain the 8-disk 4x-vdev setup and swap out to larger disks, or I could throw in a 5th mirror vdev. but I would like to maintain the performance benefits of having all data striped across the whole array.

is there some command or function in ZFS to do this? I hope I've explained this well enough.

 

[HoneyBadger]

Hello,

ZFS will not relocate or redistribute existing data onto newly added vdevs unless the file is overwritten. That's why you see the "new data using new disks" and "old data" reading from the old ones. You can find some reports from other users here by searching for terms like "rebalancing" or "rewriting":

Rewrite pool in situ?

Hello, I'm planning to add two special vdevs to my main pool. To my understanding, in order to make use of them, I'd have to 'rewrite in situ' my pools data to populate the special vdevs. What would be the most efficient way to do this? A large chunk of my data is "too large to be doubled...

www.truenas.com

Moving data to a new pool, rebalancing?

My current pool has 3 12-wide vdevs, 2x8TB and 1x18TB. In hindsight I should have added the 18TB drives as a second pool. Ultimately what I'd like is a pool with 2 12-wide vdevs of 18TB disks, and then either repurpose or retire the 8TB drives. The original pool has around ~150TB of data on it...

www.truenas.com

Several users have made use of an "in-place rebalancing script" from Github that's been linked in some other places - this basically copies each file with a new extension, and then renames it to the old name - but you'd have to be cautious about the space overhead with an approach such as this, especially if you make use of snapshots for data protection, as ZFS will consider each of these files as a brand new copy, while retaining the old file in your snapshots.

 

[NugentS]

From memory it doesn't work under Scale - at least last time I tried. Might work as a container.

run at the command line - command not found - bc
Also I suspect that using NFS ACL's is also a problem