How to interpert available/used space?

[danwestness]

How is it that a ZVOL under a ZPOOL can have MORE available space than the pool itself. Is this some sort of perpetual motion device for block storage or what?

Also, If I made this ZVOL 2.00Tib in size, then how come it say's it is using 2.20Tib in space? (by the way I only store ~30GB of actual data in this volume at the moment)

 

[Samuel Tai]

These are quirks of ZFS being a copy-on-write filesystem, and including snapshots as part of the usage for a dataset or zvol. So your zvol (including snapshots) is using 2.2 TB on disk, since new blocks written don't overwrite old blocks (which allows for snapshot rollbacks). As for the apparent oversized available space, this is due to compression. You've stated you only have ~30GB active blocks in the zvol. The calculation extrapolates the compression ratio over those active blocks to the full 2.0 TB, and uses that to estimate the available space over the remaining free space in the pool. So if your active blocks are highly compressible, you'll end up with your amusing estimate.

 

[danwestness]

I'm not sure about the 2.2TB having anything to do with snapshots...
Mainly because I have no snapshots at all.
I created the ZVOL 2 days ago... Presented it as an iscsi target to VMware and added a single 30GB thick provisioned disk to it at this point.

So 2TiB ZVOL... No snapshots... Contains only a 30GB VMDK (which is also empty)

Still not seeing how this is consuming 2.2TiB

 

[Samuel Tai]

Because the VM has written .2 TiB. You can see this more clearly by running zfs get all <path/to/zvol> | grep usedby.

 

[danwestness]

Thanks, but this doesn't seem to answer much for me

root@truenas[/mnt/data]# zfs get all data/esxi-datastore-data1 | grep "usedby"
data/esxi-datastore-data1 usedbysnapshots 0B -
data/esxi-datastore-data1 usedbydataset 17.2G -
data/esxi-datastore-data1 usedbychildren 0B -
data/esxi-datastore-data1 usedbyrefreservation 2.18T -

 

[anodos]

Thanks, but this doesn't seem to answer much for me

It's a 2.18T refreservation on the dataset:

usedbyrefreservation The amount of space used by a refreservation set on
this dataset, which would be freed if the
refreservation was removed.

See also:

refreservation=size|none|auto
The minimum amount of space guaranteed to a dataset, not including its
descendents. When the amount of space used is below this value, the
dataset is treated as if it were taking up the amount of space
specified by refreservation. The refreservation reservation is
accounted for in the parent datasets' space used, and counts against
the parent datasets' quotas and reservations.

If refreservation is set, a snapshot is only allowed if there is enough
free pool space outside of this reservation to accommodate the current
number of "referenced" bytes in the dataset.

If refreservation is set to auto, a volume is thick provisioned (or
"not sparse"). refreservation=auto is only supported on volumes. See
volsize in the Native Properties section for more information about
sparse volumes.

This property can also be referred to by its shortened column name,
refreserv.

 

[anodos]

zfsprops.7 — OpenZFS documentation

openzfs.github.io

 

[danwestness]

It's a 2.18T refreservation on the dataset:


See also:

So if i wanted a 2TB volume... and reserve 2TB in the pool for it's contents... That's fine.. (Sounds like a "thick provisioning" situation to me)
BUT... How can i say it cannot consume MORE than 2TB... ever...

If i make a 2TB volume.... put 2TB of data in it in ESXi.. then it will consume 4TB of my pool? WTH?