ZFS Fragmentation Performance issue mitigation?

2twisty

Contributor
Joined
Mar 18, 2020
Messages
145
I know this comes up all the time, and I am hoping that there is newer wisdom that what the older threads say about this.....

My pools are all used for block storage.

I have 1 NVMe-only pool that has 91% utilization, 50% free space frag
my disk pool is 67% utilized with 68% free space frag and i have SLOG on it

Code:
root@locutus[~]# zpool list
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
Tier1      2.77T   559G  2.22T        -         -    50%    19%  1.30x    ONLINE  /mnt
Tier2      21.8T  5.05T  16.7T        -         -    68%    23%  1.57x    ONLINE  /mnt



I suspect that some of the performance issues I am having are because the free space on my pools is fragmented, particularly the NVMe pool.

I know the sure-fire ways of decreasing utilization and fragmentation are to add more vdevs and/or destroy and rewrite the pool.

Since the free space on Tier1 is 50% fragmented, would simply deleting excess data from the pool improve that or will the space remain fragmented forever?

It seems that the way ZFS works will eventually cause data fragmentation that can (will?) lead to performance issues unless you periodically destroy/rebuild or keep adding space ad infinitum. This seems like a serious hole in ZFS's design that there is no way to mitigate fragmentation aside from offline and/or destructive processes.

Aside from doing the sure fire ways listed above, are there any other mitigation strategies?
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,110
Code:
root@locutus[~]# zpool list
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
Tier1      2.77T   559G  2.22T        -         -    50%    19%  1.30x    ONLINE  /mnt
Tier2      21.8T  5.05T  16.7T        -         -    68%    23%  1.57x    ONLINE  /mnt

Disable deduplication right now.
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,110
Even though I have 256GB ram?
Yes, see my post here:

 
Top