Understanding deduplication ratios with zdb

[viniciusferrao]

Hello guys,

I've run zdb -S on one of my zpools to see if dedup would help. At it really would help with 1.55x ratio.

But, there's the "copies" value. And I don't know WTH is this value. And and the end, enabling deduplication would be worse.

Here's the output:

Code:

storage# cat nohup.out 
traversing objset 0, 104 objects, 0 blocks so far
traversing objset 21, 9 objects, 0 blocks so far
traversing objset 41, 2 objects, 1 blocks so far
Simulated DDT histogram:
 
bucket              allocated                       referenced          
______   ______________________________   ______________________________
refcnt   blocks   LSIZE   PSIZE   DSIZE   blocks   LSIZE   PSIZE   DSIZE
------   ------   -----   -----   -----   ------   -----   -----   -----
     1    72.6M    581G    581G   1.30T    72.6M    581G    581G   1.30T
     2    4.32M   34.6G   34.6G   79.0G    9.84M   78.7G   78.7G    180G
     4    2.15M   17.2G   17.2G   39.4G    11.0M   88.2G   88.2G    202G
     8     731K   5.71G   5.71G   13.1G    7.26M   58.1G   58.1G    133G
    16     178K   1.39G   1.39G   3.17G    3.61M   28.9G   28.9G   65.9G
    32    53.7K    429M    429M    981M    2.26M   18.1G   18.1G   41.4G
    64    22.0K    176M    176M    403M    1.86M   14.9G   14.9G   34.1G
   128    3.05K   24.4M   24.4M   55.8M     452K   3.53G   3.53G   8.07G
   256      329   2.57M   2.57M   5.87M     109K    870M    870M   1.94G
   512      329   2.57M   2.57M   5.87M     197K   1.54G   1.54G   3.52G
    1K       50    400K    400K    914K    66.3K    530M    530M   1.18G
    2K       46    368K    368K    841K     137K   1.07G   1.07G   2.45G
    4K       19    152K    152K    347K    98.6K    789M    789M   1.76G
    8K        8     64K     64K    146K    95.2K    762M    762M   1.70G
   16K        4     32K     32K   73.1K    86.4K    691M    691M   1.54G
   32K        2     16K     16K   36.6K    90.5K    724M    724M   1.62G
  128K        2    136K    136K    146K     298K   17.3G   17.3G   19.0G
  256K        3     24K     24K   54.8K     930K   7.26G   7.26G   16.6G
  512K        1      8K      8K   18.3K     946K   7.39G   7.39G   16.9G
    4M        2     16K     16K   36.6K    11.3M   90.5G   90.5G    207G
 Total    80.0M    640G    640G   1.43T     123M   1000G   1000G   2.21T
 
dedup = 1.55, compress = 1.00, copies = 2.26, dedup * compress / copies = 0.68

Please someone help-me understanding this? This makes no sense for me.

Thanks in advance,

 

[Dusan]

Did you change the zfs copies property? Run "zfs get copies" to find out. You can find the copies explained here: www.freebsd.org/cgi/man.cgi?query=zfs
The zdb output also shows that you would need about 24GB of RAM just to hold the dedup tables (80M allocated blocks * 320 bytes).

 

[viniciusferrao]

Did you change the zfs copies property? Run "zfs get copies" to find out. You can find the copies explained here: www.freebsd.org/cgi/man.cgi?query=zfs
The zdb output also shows that you would need about 24GB of RAM just to hold the dedup tables (80M allocated blocks * 320 bytes).

Here are my zfs get copies output:

Code:

storage# zfs get copies
NAME                              PROPERTY  VALUE   SOURCE
storagepool0                      copies    1       default
storagepool0/lapeui-storage       copies    1       default
storagepool0/lapeui-storage/data  copies    1       default
storagepool0/lapeui-storage/home  copies    1       default
storagepool0/lvm0                 copies    1       default
storagepool1                      copies    1       default
storagepool1/lvm1                 copies    1       default
vmpool0                           copies    1       default

How did you calculate the 24GB of RAM for dedup tables? I see you get 80M from the total of allocated blockes, but I don't know where you get the 320 bytes value. And what is the other values like refcnt, {l,p,d}size?

Thanks in advance,

 

[cyberjock]

Did you change the zfs copies property? Run "zfs get copies" to find out. You can find the copies explained here: www.freebsd.org/cgi/man.cgi?query=zfs
The zdb output also shows that you would need about 24GB of RAM just to hold the dedup tables (80M allocated blocks * 320 bytes).

AKA: Don't do it on a system with just 32GB of RAM like the OP has.

:)

The dedup table has to store the checksum of every block and its pointer. That's 320 bytes per block. So if you have 80M blocks the math is 80M * 320 bytes.

And now you see why dedup is just stupid unless you have insane amounts of RAM.

 

[Dusan]

Hmm, that's interesting. I would expect to see copies=2.26 in a case when you started with copies=3 and later switched to copies=2 (or 1) as the setting only applies to new data.
The size of one deduplication table entry is about 320 bytes: http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-113-size-zfs-dedup-1354231.html

 

[viniciusferrao]

AKA: Don't do it on a system with just 32GB of RAM like the OP has.

:)

Yeah, I will not! :)

I'm just sizing, we're already bought that hardware in the other topic. We are just waiting it to come :)

 

[viniciusferrao]

Hmm, that's interesting. I would expect to see copies=2.26 in a case when you started with copies=3 and later switched to copies=2 (or 1) as the setting only applies to new data.
The size of one deduplication table entry is about 320 bytes: http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-113-size-zfs-dedup-1354231.html

Ok, but what to do now? I never used the "copies" option.

I don't know if it's releated, but the zpool have a zvol with LVM inside from an iSCSI share.

 

[cyberjock]

This also demonstrates why I laugh SOOOO hard at this quote from the release notes for 8.3.0. Note the bold and tell me you didn't get a laugh at it!

ZFS v28 includes deduplication, which can be enabled at the dataset level. The more data you write to a deduplicated volume the more memory it requires, and there is no upper bound on this. When the system starts storing the dedup tables on disk because they no longer fit in RAM, performance craters. There is no way to undedup data once it is deduplicated, simply switching dedup off has NO AFFECT on the existing data. Furthermore, importing an unclean pool can require between 3-5GB of RAM per TB of deduped data, and if the system doesn't have the needed RAM it will panic, with the only solution being adding more RAM or recreating the pool. Think carefully before enabling dedup! Then after thinking about it use compression instead.

Whoever at iXsystems wrote that deserves an award. That's someone that made a freakin' amazing point about dedup while putting a sense of humor in it at the same time!

 

[cyberjock]

What command did you run to get that output? I'll run it on my pool just to see the results...

 

[viniciusferrao]

What command did you run to get that output? I'll run it on my pool just to see the results...

I've just run:

Code:

zdb -Sv nameOfMyPool

I've used nohup just to leave it running at night.