SOLVED Dedup work at all with zvols and iSCSI block shares, on which VHD are stored?

[ee21]

I am new to TrueNAS, and wanted to experiment with dedup on a small amount of storage, rather than the whole pool, as most of that data are not ideal candidates for dedup...

So I turned it on for a single zvol, which is configured as an iSCSI "Device" type extent, and connected to a Windows Hyper-V initiator. On the iSCSI disk, there lives then a virtual disk for my VM. This VM is a network DVR which stores H265 encoded video.

I was sort of figuring that since most of the background in the video is static, that data might be a good candidate for deduping. However, a couple days later, I checked using zpool list and zpool status, and am seeing the ratio at 1.00, and the LSIZE allocated and referenced both the same.

Does deduping really work in this scenario where I am using iSCSI block storage, and a virtual disk stored on that? Specifically I am asking if dedup can target the data inside a VHD - or is my issue more on the side of H265 video not really being dedupable, even if some of the video is static and shared between recordings?

 

[Heracles]

and wanted to experiment with dedup

Just don't.... Dedup will not do you any good.

Better to use compression, to clone datasets, to dedup at the application level, ...

Dedup in ZFS itself will just kill your box for no gain.

Below 128G of RAM, better to forget completely bout Dedup...

I was sort of figuring that since most of the background in the video is static, that data might be a good candidate for deduping.

Nope. The video encoding will have reduce that long before the data hits ZFS.

 

[ee21]

Compression doesn't work on already compressed data, e.g. H265 video. I tried just out of curiosity and saw a 1.00-1.01 compression ratio.

Have 64GB ECC for now, but am looking to double that up to 128GB. Sounds like DVR recordings aren't good targets for dedupe regardless though from what I'm hearing.

 

[HoneyBadger]

Video isn't a dedup candidate either as you've found out; it has to be a 100% match, bit-for-bit, for the entire record, in order for it to take effect. One pixel off (or shaded as 0xFFFFFF vs 0xFFFFFE) could be enough to invalidate that record.

As mentioned before, the H265 compression algorithm (or any other decent video codec) should be generating a bunch of B/P-frames rather than storing nothing but I-frames, so you'll already be compressed as well as possible there.