lz4 compression of vmdk files

[elangley]

Hello All,

Running 9.2 and replicating vmdk files from a remote site.

I am curious how lz4 compression works?
Does it compress as files are being saved or after they are saved? How about when the file is being accessed? In this case a vmdk for a powering on virtual machine.

Wasn't sure where to post this, so please let me know if it is in the wrong forum.

~eric

 

[eraser]

ZFS compression happens in real-time when the block is first written to disk. When a compressed block is accessed it is decompressed. A block that was written uncompressed remains uncompressed when accessed.

 

[elangley]

Okay, thanks eraser.

So if I copy a set of VMware files to a compressed volume they will be compressed.
Then, when they are accessed by VMware they will be uncompressed.

When VMware releases them they would be compressed again?

Is that how it would work?

 

[eraser]

Almost!

Compression happens at the block level (not file level) during writes to disk. Compression is transparent to applications accessing data stored in ZFS.

If you copy a set of files to a compressed dataset (or volume) they will be transparently compressed before the blocks are written to disk.

When a file is read, each block of the file is transparently uncompressed in real time as it is read from disk.

If VMware writes new data (blocks) to the file, the new blocks are transparently compressed before being written to disk.

If VMware only reads from the file, there are no writes happening so compression will not happen.

Hope this helps!

p.s. 'lz4' Compression should be enabled because it is effectively 'free' (modern CPUs are very very fast at lz4 compression/decompression). Stay away from enabling Deduplication -- it is (almost) never worth the additional memory requirements.

 

[elangley]

Ah, block level compression. Cool!

Now I need to move all my data off and back on a volume to get them compressed.

Just read about lz4 here as well:
http://freebsdnow.blogspot.com/2013/07/freebsd-92-feature-highlight-zfs-lz4.html

Thanks, this question is answered.