Pool fragmentation explanation!

[rio236]

I would appreciate an explanation for the appearance of fragmentation.
Build:
TrueNAS-12.0-U5
Plex server
Supermicro SC846BE1C-R1K23B
Supermicro X11SPH-nCTF
Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz
Samsung M393A2K40CB1-CRC 256GiB (ECC)
Supermicro SNK-P0068APS4
Seagate ST10000VN4 Ironwolf NAS 21 10TB HD - raidz3 3x7 vdev
Supermicro CSE-846BE1C-R1K03JBOD
Supermicro CBL-SAST-0690-1
Supermicro 16PORT Mini SAS HD INT-to-EXT Cable Adapter W/FH Brack
Pools:
boot-pool - Transcend 2 x SSD370S mirrored
tank0 (data) - Seagate Ironwolf NAS 21 10TB HD - raidz3 3x7 vdev
tank1 (apps) 2 x INTEL SSDPED1D280GA mirrored
tank2 (torrents) 4 x 3TB SEAGATE ST3000DM001-1ER1 - mirrored
tank3 (editting) 4 x 4TB ST4000VN000-1H41 - mirrored
Pools:

Files from tank3 are copied to tank0
When new files are added to tank0, the FRAG increases to 1%.
Please explain.
Thank you

 

[Heracles]

Hey @rio236,

Tank0 is already loaded up to 40% of its capacity Still usable but clearly significantly loaded already. Drives are spinning and ZFS will not defrag anything in any way. As such, when you wrote these 40%, they did not ended up on spaces 0 up to 39. Whenever a write is ready, disks are at a certain place. When next write is ready, drives will have move to a different place. When you do add more data, some will start using free spots here and there, creating fragmentation. Fragmentation will happen more and more, the more your pool will be loaded.

Also, know that often, it is better to use different datasets instead of different pool. Here, you have 2 types of vdev, so better not to mix and match these in a single pool. Should you wish to stay that way, a single pool of RaidZ3 vdev and another with your mirror would help you. Here, yourself are doing fragmentation, so clearly you do not help to avoid it by causing some...

 

[Patrick M. Hausen]

The fragmentation percentage shown really is the fragmentation of free space on the pool. So when you copy some new files to the pool, they may be written to a continuously allocated range of blocks but the directories that hold these new files - and similarly other metadata - needs to be updated, too. That implies, since ZFS is copy-on-write for everything that you now have a couple of blocks that were used for metadata marked as free again, lying in the middle of other data that was once written with the same transaction group.

This is nothing to worry about unless the pool gets 70-80% full as far as oral transmission of common wisdom claims. I have not seen some official documentation on it, yet.

 

[rio236]

Thanks for both of your replies.

1. Would expanding tank0 with another raidz3 3x7 vdev with larger disk size and more datasets be best?
   Or
2. Should I create a new pool tank? raidz3 3x7 vdev with larger disk size and more datasets in tank0?
   I'm looking at Seagate Exos X18 18TB Enterprise HDD ST18000NM000

 

[Patrick M. Hausen]

"best" to achieve what precisely?

 

[rio236]

Maintain fragmentation through expansion.

 

[Patrick M. Hausen]

What do you mean by maintain fragmentation? Fragmentation of free space is inevitable as data is rewritten and generally not a problem.

I would consider other things:

• will your application profit from more vdevs - more IOPS?
• are all these disks in the same enclosure?
• if not will adding them to the same pool increase the risk of losing an entire vdev, e.g. because an additional enclosure might go down
• lose one vdev - lose the entire pool, RAIDZ3 or not ...

HTH,
Patrick