SOLVED Volume size appears incorrect

ADUR · Mar 4, 2019

Hi everyone,

I have seen a lot of posts of this kind on the forum, and even if I've been through some of them, I unfortunnately didn't solve my issue :/

Here is my issue :
I have a Freenas running 11.1U6 version
I have only one big storage supposed to be sized at 6Tb, and never have I done the "zfs irreversible" update (I still have this message : New feature flags are available for volume NAS-GDA2. Refer to the "Upgrading a ZFS Pool" section of the User Guide for instructions. )

I have issues with storage, that is not freeing space (I deleted about 1Tb and quite nothing happened on the space left).
It also look like I have locked snapshots, that are probably part of the problem here.
And last of all, I'm pretty sure I lost part of my storage here, doesn't it look weird ? Where are my 6Tb ..?

Here is my storage :

The "Sync" storage is a sync between this NAS and another, on the same network elsewhere. As I understand it, the synhronized data are stored on this NAS, it looks OK

My snapshots (in yellow the one locked):

zfs destroy of the two snapshot in yellow :

Code:

root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs destroy NAS-GDA2/nfs/Sync@auto-20180625.1437-1h
cannot destroy snapshot NAS-GDA2/nfs/Sync@auto-20180625.1437-1h: dataset is busy
root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs destroy NAS-GDA2/nfs/Sync@auto-20180625.1507-1h
cannot destroy snapshot NAS-GDA2/nfs/Sync@auto-20180625.1507-1h: dataset is busy

zfs list -t all -r NAS-GDA2

Code:

root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs list -t all -r NAS-GDA2
NAME                                                                             USED  AVAIL  REFER  MOUNTPOINT
NAS-GDA2                                                                        3.13T   275G   140K  /mnt/NAS-GDA2
NAS-GDA2@auto-20190304.0400-1d                                                    93K      -   140K  -
NAS-GDA2/.system                                                                 168M   275G   151K  legacy
NAS-GDA2/.system@manual-20150730                                                 105K      -  88.3M  -
NAS-GDA2/.system@manual-20150730-1420                                            105K      -  88.3M  -
NAS-GDA2/.system/configs-903a2a7d45924e86a448cccd86aa67c2                       42.4M   275G  42.3M  legacy
NAS-GDA2/.system/configs-903a2a7d45924e86a448cccd86aa67c2@manual-20150730           0      -   140K  -
NAS-GDA2/.system/configs-903a2a7d45924e86a448cccd86aa67c2@manual-20150730-1420      0      -   140K  -
NAS-GDA2/.system/cores                                                          18.3M   275G  16.4M  legacy
NAS-GDA2/.system/cores@manual-20150730                                              0      -  1.98M  -
NAS-GDA2/.system/cores@manual-20150730-1420                                         0      -  1.98M  -
NAS-GDA2/.system/rrd-903a2a7d45924e86a448cccd86aa67c2                            232K   275G   140K  legacy
NAS-GDA2/.system/rrd-903a2a7d45924e86a448cccd86aa67c2@manual-20150730               0      -   140K  -
NAS-GDA2/.system/rrd-903a2a7d45924e86a448cccd86aa67c2@manual-20150730-1420          0      -   140K  -
NAS-GDA2/.system/samba4                                                         5.81M   275G  5.35M  legacy
NAS-GDA2/.system/samba4@manual-20150730                                          122K      -   459K  -
NAS-GDA2/.system/samba4@manual-20150730-1420                                     140K      -   477K  -
NAS-GDA2/.system/syslog-903a2a7d45924e86a448cccd86aa67c2                        12.6M   275G  11.6M  legacy
NAS-GDA2/.system/syslog-903a2a7d45924e86a448cccd86aa67c2@manual-20150730         198K      -   878K  -
NAS-GDA2/.system/syslog-903a2a7d45924e86a448cccd86aa67c2@manual-20150730-1420    279K      -   959K  -
NAS-GDA2/jails                                                                  7.49G   275G   186K  /mnt/NAS-GDA2/jails
NAS-GDA2/jails/.warden-template-pluginjail--x64                                 3.16M   275G  3.16M  /mnt/NAS-GDA2/jails/.warden-template-pluginjail--x64
NAS-GDA2/jails/.warden-template-standard--x64                                    626M   275G   626M  /mnt/NAS-GDA2/jails/.warden-template-standard--x64
NAS-GDA2/jails/.warden-template-standard--x64-20140317184634                    2.06G   275G  2.06G  /mnt/NAS-GDA2/jails/.warden-template-standard--x64-20140317184634
NAS-GDA2/jails/.warden-template-standard--x64-20140323023236                    2.12G   275G  2.06G  /mnt/NAS-GDA2/jails/.warden-template-standard--x64-20140323023236
NAS-GDA2/jails/.warden-template-standard--x64-20140323023236@clean              71.4M      -  2.06G  -
NAS-GDA2/jails/.warden-template-standard-9.3-x64                                2.13G   275G  2.06G  /mnt/NAS-GDA2/jails/.warden-template-standard-9.3-x64
NAS-GDA2/jails/.warden-template-standard-9.3-x64@clean                          71.4M      -  2.06G  -
NAS-GDA2/jails/BackupPC3                                                         574M   275G  2.30G  /mnt/NAS-GDA2/jails/BackupPC3
NAS-GDA2/jails/BackupPC3@auto-20190303.1729-1d                                  77.7M      -  2.30G  -
NAS-GDA2/nfs                                                                    3.13T   275G  1.32T  /mnt/NAS-GDA2/nfs
NAS-GDA2/nfs/GDA2                                                                883G   275G   878G  /mnt/NAS-GDA2/nfs/GDA2
NAS-GDA2/nfs/GDA2@auto-20190304.1539-1d                                          105K      -   878G  -
NAS-GDA2/nfs/GDA2/HLP                                                           5.16G   280G    93K  -
NAS-GDA2/nfs/Sync                                                                966G   275G   689G  /mnt/NAS-GDA2/nfs/Sync
NAS-GDA2/nfs/Sync@auto-20180625.1437-1h                                          232K      -   317G  -
NAS-GDA2/nfs/Sync@auto-20180625.1507-1h                                          232K      -   317G  -
NAS-GDA2/nfs/Sync@auto-20190304.1518-1h                                             0      -   689G  -
NAS-GDA2/nfs/Sync@auto-20190304.1548-1h                                             0      -   689G  -

zfs list -t all -r NAS-GDA2/nfs (to lower to the storage I'd like to have 6Tb)

Code:

root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs list -t all -r NAS-GDA2/nfs
NAME                                      USED  AVAIL  REFER  MOUNTPOINT
NAS-GDA2/nfs                             3.13T   275G  1.32T  /mnt/NAS-GDA2/nfs
NAS-GDA2/nfs/GDA2                         883G   275G   878G  /mnt/NAS-GDA2/nfs/GDA2
NAS-GDA2/nfs/GDA2@auto-20190304.1539-1d   105K      -   878G  -
NAS-GDA2/nfs/GDA2/HLP                    5.16G   280G    93K  -
NAS-GDA2/nfs/Sync                         966G   275G   689G  /mnt/NAS-GDA2/nfs/Sync
NAS-GDA2/nfs/Sync@auto-20180625.1437-1h   232K      -   317G  -
NAS-GDA2/nfs/Sync@auto-20180625.1507-1h   232K      -   317G  -
NAS-GDA2/nfs/Sync@auto-20190304.1518-1h      0      -   689G  -
NAS-GDA2/nfs/Sync@auto-20190304.1548-1h      0      -   689G  -

I already deleted a bunch of snapshot, and it freed a few space, but really not so much.
Of course df -h don't work well (everytime I delete something, it only substract the space I deleted from the available space)

Later this week, I will try 2 things that I'm not able to try now :
- Reboot the server
- Upgrade to 11.1U7

I will be glad of every hint you can give me, or if you can explain me where my 3Tb are !
Many thanks for your time !

Best regards,
ADUR

garm · Mar 4, 2019

What is the configuration of your pool?
Post the printout of zpool status -v

ADUR · Mar 4, 2019

Hi garm =)

Here we are :

Code:

root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zpool status -v
  pool: NAS-GDA2
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0 days 08:36:34 with 0 errors on Sat Mar  2 02:45:12 2019
config:

        NAME                                            STATE     READ WRITE CKSUM
        NAS-GDA2                                        ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            gptid/884730ad-a893-11e3-aa4c-a0481cb833a0  ONLINE       0     0     0
            gptid/88a050dd-a893-11e3-aa4c-a0481cb833a0  ONLINE       0     0     0
            gptid/7cc35920-10dc-11e8-ba32-a0481cb833a0  ONLINE       0     0     0
            gptid/894bb460-a893-11e3-aa4c-a0481cb833a0  ONLINE       0     0     0

errors: No known data errors

  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:02:23 with 0 errors on Mon Feb 25 03:47:24 2019
config:

        NAME        STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2     ONLINE       0     0     0

errors: No known data errors

I'll give it a restart in a few hours FI

Thank you for your interest on this case =)

garm · Mar 4, 2019

NAME USED AVAIL REFER MOUNTPOINT
NAS-GDA2/nfs 3.13T 275G 1.32T /mnt/NAS-GDA2/nfs

Is the culprit, you have 1 TiB worth of snapshots in this dataset. Do you use ZFS Send/Receive?

With RaidZ2 you don’t get 6 TiB with 4x2 TB drives, you get ~ 3.6 TiB

ADUR · Mar 4, 2019

garm said:
Is the culprit, you have 1 TiB worth of snapshots in this dataset. Do you use ZFS Send/Receive?

With RaidZ2 you don’t get 6 TiB with 4x2 TB drives, you get ~ 3.6 TiB

Oh my bad about my understanding of the space supposed to be available !
Hum I was already sure it was under nfs, but I don't understand which snapshot is responsible ...

If I take the ones under NAS-GDA2/nfs :

ADUR said:
zfs list -t all -r NAS-GDA2/nfs (to lower to the storage I'd like to have 6Tb)

Code:
NAS-GDA2/nfs/Sync@auto-20180625.1437-1h 232K - 317G - NAS-GDA2/nfs/Sync@auto-20180625.1507-1h 232K - 317G - NAS-GDA2/nfs/Sync@auto-20190304.1518-1h 0 - 689G - NAS-GDA2/nfs/Sync@auto-20190304.1548-1h 0 - 689G -

I'm able to delete the ones of today, but they use 0 space.

And I don't know what ZFS send/receive is, so at first glance I would answer "no", but not sure about this one.

You think that if I was able to delete the locked snaps, I would free about 1Tb ?

In any case, kudos for your help of pointing the issue !

ADUR · Mar 4, 2019

I found help on this post :
https://www.ixsystems.com/community/threads/can't-delete-snapshots.24911/

There was a hold tag on my snapshot.

Here is what I've done :

Code:

root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs holds NAS-GDA2/nfs/Sync@auto-20180625.1437-1h
NAME                                     TAG           TIMESTAMP
NAS-GDA2/nfs/Sync@auto-20180625.1437-1h  freenas:repl  Mon Jun 25 14:37 2018

root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs release -r freenas:repl NAS-GDA2/nfs/Sync@auto-20180625.1437-1h
root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs release -r freenas:repl NAS-GDA2/nfs/Sync@auto-20180625.1507-1h

root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs list -t snapshot -r NAS-GDA2/nfs
NAME                                      USED  AVAIL  REFER  MOUNTPOINT
NAS-GDA2/nfs/GDA2@auto-20190304.1539-1d   174K      -   878G  -
NAS-GDA2/nfs/Sync@auto-20190304.1648-1h      0      -   689G  -
NAS-GDA2/nfs/Sync@auto-20190304.1718-1h      0      -   689G  -

root@nas-gda2:/mnt/NAS-GDA2/nfs/GDA2 # zfs destroy NAS-GDA2/nfs/Sync@auto-20180625.1507-1h
could not find any snapshots to destroy; check snapshot names.

I finally deleted also all of the snapshots under NAS-GDA2/nfs, to be sure none of them keeped space.

I now have 552Gb available on my NAS-GDA2/nfs volume.

I also assume that it will probably free some more space if I let the deletion make it's stuff (I've found on another post that freeing space is asynchroneous)

Many thanks @garm for your help, I hope it will be enough for my use !
I close this post, what a fast resolve !

pro lamer · Mar 9, 2019

ADUR said:
also assume that it will probably free some more space if I let the deletion make it's stuff

Has it actually happen? What is your free space now?

Sent from my phone

ADUR · Mar 25, 2019

pro lamer said:
Has it actually happen? What is your free space now?

Sent from my phone

Hi pro lamer, sorry for the long time I took to answer !

The free space didn't grow up after the actions I took, and since we still use the NAS and some data have been added, the remaining space today isn't relevant for your point unfortunately.

But a day after the actions, the free space was still 552Gb, and that's enough for what I wanted my NAS to have !

Feel free to ask anything else, I'll answer quicker this time !

Important Announcement for The TrueNAS Community.

SOLVED Volume size appears incorrect

ADUR

Cadet

Attachments

garm

Wizard

ADUR

Cadet

garm

Wizard

ADUR

Cadet

ADUR

Cadet

pro lamer

Guru

ADUR

Cadet

Similar threads