Deduplicating on zfs send.

Status
Not open for further replies.

jafrey

Cadet
Joined
Dec 24, 2014
Messages
3
I was reading through the zfs man pages and saw that you can use deduplication on send even if it isn't used all the time.

Code:
-D      Generate a deduplicated stream. Blocks which would have been
                 sent multiple times in the send stream will only be sent
                 once.  The receiving system must also support this feature to
                 receive a deduplicated stream.  This flag can be used
                 regardless of the dataset's dedup property, but performance
                 will be much better if the filesystem uses a dedup-capable
                 checksum (eg.  sha256).


For a test I copied a bunch of images to a scratch ZFS pool and then put copies in 4 subfolders.

Code:
/mnt/scratch_zfs/test/2015Oct03_111345.cr2
/mnt/scratch_zfs/test/2015Oct03_111353.cr2
/mnt/scratch_zfs/test/2015Oct03_111354.cr2
/mnt/scratch_zfs/test/2015Oct03_111355.cr2
/mnt/scratch_zfs/test/2015Oct03_113914.cr2
/mnt/scratch_zfs/test/folder1/2015Oct03_111345.cr2 ...
/mnt/scratch_zfs/test/folder2/2015Oct03_111345.cr2 ...
/mnt/scratch_zfs/test/folder3/2015Oct03_111345.cr2 ...
/mnt/scratch_zfs/test/folder4/2015Oct03_111345.cr2 ...


Then sent the filesystem to a file with and with out deduplication:

Code:
# zfs send test@0001 > test.zfs
zfs send -D test@0001 > test.zfs
-rw-r--r--  1 root  wheel  570M Nov  4 12:08 test.zfs
-rw-r--r--  1 root  wheel   115M Nov  4 12:08 test-D.zfs
 

solarisguy

Guru
Joined
Apr 4, 2014
Messages
1,125
I think that the same rules about RAM requirements apply here as with the deduplication enabled on a dataset level.

(This is a repost, since the forum lost some posts in the last 24 hours.)
 
Status
Not open for further replies.
Top