Storage Dashboard displays disk usage based on referenced data instead of allocated data.

M

matticustard

Cadet
Joined
Nov 2, 2023
Messages
4
Thank you for looking.

System​


Software: TrueNAS-SCALE-22.12.4.2
Motherboard make and model: Supermicro X9DR3-F
CPU make and model: 2x Intel Xeon E5-2697V2
RAM quantity: 128GB ECC
Network cards: 10Gtek with Intel 82599EN (Compare to Intel X520-DA2)
Hard disk controllers: 1x LSI 9207-8i, 2x LSI 9207-8e (all IT mode)

While this does not account for every disk in the system (47 total), the following list should be sufficient for the purpose of this post.

Pool: Archive
-- Hard drives: 6x HGST Ultrastar HUH728080ALE601 8TB 7200RPM SATA 6Gb/s 3.5" Hard Drive
---- Configuration: Striped mirror (3 zvol, 2 disks/zvol)
-- Hard drives: 3x Samsung MZILS1T9HCHP-000C3 1.92tb Solid State Drive (DEDUPLICATION VOLUME)
---- Configuration: Mirror (1 zvol)

Problem​

I have a pool that relies heavily on deduplication with a current ratio of ~1/10. Instead of displaying actual disk usage, the Storage Dashboard displays the total amount of referenced data (i.e. it adds up all deduplicated data) to determine the amount of used data and then adds this amount to the remaining space when calculating total usable capacity. This leads to a very inaccurate overview of disk usage.

1698949671092.png


What I see in the Storage Dashboard:​

--Values from image above--

Usable capacity: 50.47TiB
Used: 32.99 TiB
Available: 17.48 TiB
Percent: 65.4%

What I expect to see in the Storage Dashboard:​

--Approximate values given--

Usable capacity: 22TiB
Used: 4 TiB
Available: 17.48 TiB
Percent: 23%

Questions​


1. Is this the intended behavior?

2. If yes, why is this the intended behavior?

3. Is my volume going to begin showing over-usage errors when the Storage Dashboard exceeds 80% usage, despite being nowhere near that actual usage?

Deduplication stats​

Code:
bucket              allocated                       referenced
______   ______________________________   ______________________________
refcnt   blocks   LSIZE   PSIZE   DSIZE   blocks   LSIZE   PSIZE   DSIZE
------   ------   -----   -----   -----   ------   -----   -----   -----
     1    23.2M   2.88T   2.69T   2.69T    23.2M   2.88T   2.69T   2.69T
     2    7.13M    888G    872G    873G    16.5M   2.00T   1.97T   1.97T
     4    1.05M    119G    108G    108G    5.94M    658G    590G    591G
     8     603K   63.5G   54.8G   54.9G    6.75M    724G    627G    628G
    16     948K    115G   97.4G   97.5G    21.9M   2.65T   2.25T   2.25T
    32    1.44M    174G    149G    149G    58.0M   6.82T   5.83T   5.83T
    64     120K   14.2G   10.8G   10.8G    10.0M   1.18T    934G    934G
   128    18.4K   1.46G   1.11G   1.12G    2.93M    230G    177G    179G
   256     526K   63.4G   48.5G   48.5G     162M   19.6T   15.0T   15.0T
   512    25.6K   3.02G   2.68G   2.68G    17.0M   2.00T   1.78T   1.78T
    1K    2.19K    254M    223M    224M    3.22M    375G    330G    330G
    2K      579   63.8M   51.4M   51.5M    1.51M    170G    132G    132G
    4K    1.11K    139M    134M    135M    4.93M    615G    588G    588G
    8K       29   2.31M    542K    548K     317K   24.9G   5.81G   5.86G
   16K        6    286K     40K     40K     123K   5.96G    811M    811M
   32K        3    164K   12.5K     16K     136K   8.78G    561M    697M
   64K        2   85.5K     16K     16K     139K   5.79G   1.08G   1.08G
  128K        1   41.5K      8K      8K     129K   5.25G   1.01G   1.01G
 Total    35.1M   4.29T   4.00T   4.01T     335M   39.9T   32.8T   32.8T
 
M

matticustard

Cadet
Joined
Nov 2, 2023
Messages
4
MORE EDIT: Under disk configuration, I meant to use "vdev" not "zvol".

Jeez. Sorry for all the mistakes. I thought I would be able to edit the original post if needed.
 
HoneyBadger

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,112
You should be able to edit your posts shortly; it takes a few standard/approved ones to get out of the "new user" filter.

Nice deduplication stats! Can you show the line above where it says the number of records and their size on Disk/Core? That will give you an idea of how much it's "costing" in terms of RAM and SSD for that.

Regarding your questions - let me see if this was intended behavior with the UI team. With deduplication it's probably important to have both values represented, but the overall ratio might be better to reflect the physical disk allocation.
 
M

matticustard

Cadet
Joined
Nov 2, 2023
Messages
4
Thank you for your response!

Requested info from yesterday's data:​


Code:
dedup: DDT entries 36760599, size 581B on disk, 187B in core


Full detail from today:​


Code:
 dedup: DDT entries 36780146, size 581B on disk, 187B in core

bucket              allocated                       referenced
______   ______________________________   ______________________________
refcnt   blocks   LSIZE   PSIZE   DSIZE   blocks   LSIZE   PSIZE   DSIZE
------   ------   -----   -----   -----   ------   -----   -----   -----
     1    23.3M   2.88T   2.69T   2.69T    23.3M   2.88T   2.69T   2.69T
     2    7.13M    888G    872G    873G    16.5M   2.00T   1.97T   1.97T
     4    1.05M    119G    108G    108G    5.94M    658G    590G    591G
     8     603K   63.5G   54.8G   54.9G    6.75M    724G    627G    628G
    16     948K    115G   97.4G   97.5G    21.9M   2.65T   2.25T   2.25T
    32    1.44M    174G    149G    149G    58.0M   6.82T   5.83T   5.83T
    64     120K   14.2G   10.8G   10.8G    10.0M   1.18T    934G    935G
   128    18.4K   1.47G   1.11G   1.12G    2.93M    230G    177G    179G
   256     526K   63.4G   48.5G   48.5G     162M   19.6T   15.0T   15.0T
   512    25.6K   3.02G   2.68G   2.68G    17.0M   2.01T   1.78T   1.78T
    1K    2.19K    254M    223M    224M    3.22M    375G    330G    330G
    2K      579   63.8M   51.4M   51.5M    1.51M    170G    132G    132G
    4K    1.11K    139M    134M    135M    4.93M    615G    588G    588G
    8K       29   2.31M    542K    548K     317K   25.0G   5.82G   5.87G
   16K        6    286K     40K     40K     123K   5.96G    812M    812M
   32K        3    164K   12.5K     16K     136K   8.78G    561M    697M
   64K        2   85.5K     16K     16K     139K   5.79G   1.08G   1.08G
  128K        1   41.5K      8K      8K     129K   5.25G   1.01G   1.01G
 Total    35.1M   4.29T   4.01T   4.01T     335M   39.9T   32.8T   32.8T
 
You must log in or register to reply here.
Top