Some files bigger after compression?

[CJRoss]

I've left the default lz4 compression on for all of my datasets except for a few that I've migrated to zstd. I was looking to get a bit more fine grained info on the amount of compression than what Compression Ratio can tell me. I found this post talking about the difference between du and du -A. https://www.truenas.com/community/t...n-of-a-single-file-has-answer-is-not-a.90145/

This led me to discover that some of my files are actually bigger on disk than they're reporting. I thought at first it might just be small files taking up a whole block but I found large files doing the same thing. I thought the compression was supposed to abort if it wasn't able to compress a file? So how did I end up with files bigger than the original? Am I missing something?

This is a dataset with 1M block size and lz4 compression.

truenas% du -Aha | sort -h | tail
3.6G ./S00E1075 - In_The_Next_Life_FOREVERVERSE_WEREWOLF [HHWVCyXJDUA].mp4
4.0G ./S00E1937 - Biomutant_-_Save_Point_w_Becca_Scott_Gameplay_and_Funny_Moments [uJJEkGhEhOc].mp4
4.2G ./S00E1926 - Minecraft_Part_3_-_Save_Point_with_Becca_Scott [eXTje8a8DsA].mp4
4.2G ./S00E1941 - Biomutant_Pt._2_-_Save_Point_w_Becca_Scott_Gameplay_and_Funny_Moments [QgUr08nAF34].mp4
4.4G ./S00E1916 - Jackbox_Games_Pt._2_-_Save_Point_with_Becca_Scott [L4haddIQBeo].mkv
4.5G ./S00E1923 - Fall_Guys_-_Ultimate_Knockout_-_Save_Point_with_Becca_Scott [JDEvGZjO14U].mp4
4.7G ./S00E1942 - Biomutant_Pt._3_-_Save_Point_w_Becca_Scott_Gameplay_and_Funny_Moments [0u0EwgmvAco].mp4
4.9G ./S00E1929 - Subnautica_pt._3_-_Save_Point_with_Becca_Scott [UxL8HYUrgig].mkv
5.7G ./S00E1919 - Subnautica_pt._2_-_Save_Point_with_Becca_Scott [I-ckh6CzuiQ].mkv
506G .
truenas% du -ha | sort -h | tail
3.6G ./S00E1075 - In_The_Next_Life_FOREVERVERSE_WEREWOLF [HHWVCyXJDUA].mp4
4.0G ./S00E1937 - Biomutant_-_Save_Point_w_Becca_Scott_Gameplay_and_Funny_Moments [uJJEkGhEhOc].mp4
4.2G ./S00E1941 - Biomutant_Pt._2_-_Save_Point_w_Becca_Scott_Gameplay_and_Funny_Moments [QgUr08nAF34].mp4
4.7G ./S00E1926 - Minecraft_Part_3_-_Save_Point_with_Becca_Scott [eXTje8a8DsA].mp4
4.7G ./S00E1942 - Biomutant_Pt._3_-_Save_Point_w_Becca_Scott_Gameplay_and_Funny_Moments [0u0EwgmvAco].mp4
4.9G ./S00E1916 - Jackbox_Games_Pt._2_-_Save_Point_with_Becca_Scott [L4haddIQBeo].mkv
5.0G ./S00E1923 - Fall_Guys_-_Ultimate_Knockout_-_Save_Point_with_Becca_Scott [JDEvGZjO14U].mp4
5.3G ./S00E1929 - Subnautica_pt._3_-_Save_Point_with_Becca_Scott [UxL8HYUrgig].mkv
6.2G ./S00E1919 - Subnautica_pt._2_-_Save_Point_with_Becca_Scott [I-ckh6CzuiQ].mkv
511G .

 

[Ericloewe]

I would not trust that output, as it should be impossible for files to take up extra space with compression on.

If (big if) du is not simply broken, it's possible that the on-disk file sizes are being inflated by snapshots.

 

[CJRoss]

I would not trust that output, as it should be impossible for files to take up extra space with compression on.

If (big if) du is not simply broken, it's possible that the on-disk file sizes are being inflated by snapshots.

That's why I was confused. I went back and verified multiple times to make sure I was using the right options in the du command.

I can't speak to if du is broken or not, but I don't think it's snapshots. Those are downloaded video files. They're not being modified once created. I even took some different files and copied them to a zstd dataset with similar results and those definitely didn't have snapshots.