Zvol used and available space

[jms123]

Having a bit of a crash course in TrueNAS at the moment :)

I understand that available space is the amount of free space remaining so total space minus used space. So I created a new pool with a mirrored vdev and it reported 929.51 GB raw space. Once created -

root@truenas[~]# zfs list TestVPS-Pool
NAME USED AVAIL REFER MOUNTPOINT
TestVPS-Pool 26.9M 899G 96K /mnt/TestVPS-Pool

I then created a zvol using 80% of 899GB as recommended so 719GB -

root@truenas[~]# zfs list TestVPS-Pool
NAME USED AVAIL REFER MOUNTPOINT
TestVPS-Pool 730G 169G 96K /mnt/TestVPS-Pool

this makes sense because if you add the two together you get back to the 899GB (I'm assuming it reports 730 GB as opposed to 719 GB because of overhead).

But the zvol output I can't make sense of -

root@truenas[~]# zfs list TestVPS-Pool/TestVPS-Zvol
NAME USED AVAIL REFER MOUNTPOINT
TestVPS-Pool/TestVPS-Zvol 730G 899G 56K -

how can the available amount be 899 GB - should it not be 730 GB as it is empty at the moment.

When I start to use the zvol it then shows -

root@truenas[~]# zfs list TestVPS-Pool/TestVPS-Zvol
NAME USED AVAIL REFER MOUNTPOINT
TestVPS-Pool/TestVPS-Zvol 730G 607G 292G -

so the available + the referenced (actual data) still equals 899 GB even though the size of the zvol is 719GB.

Thanks

 

[Arwen]

Interesting, I thought adding a Reservation of 719GB might solve the "AVAIL" problem but it appears that zfs create does that automatically. Unless perhaps you did it through the GUI and the GUI does not set a Reservation.

Get the output of this command and post here, (preferably in code tags);
zfs get reservation TestVPS-Pool/TestVPS-Zvol

 

[jms123]

Here you go -

Code:

root@truenas[~]# zfs get reservation TestVPS-Pool/TestVPS-Zvol
NAME                       PROPERTY     VALUE   SOURCE
TestVPS-Pool/TestVPS-Zvol  reservation  none    default

I did the pool and zvol creation from the GUI.

 

[Etorix]

For block storage, the recommended limit is 50%:

Resource - The path to success for block storage

ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle bits modified after creation. This is optimal for RAIDZ. It's what most...

www.truenas.com

It looks like you have a 730 GB (or GiB? beware of units…) zvol which is allowed to use the full 899 GB pool to store its data—and with CoW it will quickly need more "referenced" space than what is reported as "used" space in the zvol to the client.

 

[jms123]

Thanks for the bit about 50%, someone else pointed that out in another thread and will do but this is just a test.

So when you create the zvol using 80% of the pool as I did that is not a fixed size then ie. it can go above that into the extra 20% unused space ?

I thought the size you chose when you created the zvol was the maximum amount of space it could use and if that's not the case what is the point of allocating only 80% in the first place ?

 

[jgreco]

Forget 80%. That's a number often recommended for typical fileserver use of a ZFS pool. For block storage, it's more like 40%-50%. See the article linked above, which contains a graphic which helps explain it. There is no 80% anything for block storage.

 

[jms123]

I do understand that based on your response in the other thread and the post you linked to which I read but this question was about trying to understand how the space is reported for a zvol and as it is in a test environment I didn't think it really mattered what percentage I used as I am not going to be using it for any storage.

As a follow on to your point if you have created a zvol using 80% of the pool but make sure you only use 50% of the zvol is that achieving the same thing as creating the zvol using 50% of the pool ?

Also if you are using SSDs does this change the recommendation or not ?

 

[jgreco]

As a follow on to your point if you have created a zvol using 80% of the pool but make sure you only use 50% of the zvol is that achieving the same thing as creating the zvol using 50% of the pool ?

WTH is "50% of the zvol" about? Where did you come up with that from?

This is all about pool occupancy. Blocks are allocated out of the ZFS pool; there isn't a special space allocated for your zvol. Once the pool occupancy exceeds 40-50%, you are in a painful situation.

If you create a zvol that is allowed to use 80% of the pool, and then fill it to 50%, you have used 40% of the pool -- 0.8 * 0.5 = 0.4. The "size" of the zvol is more or less just a reservation. ZFS can compress blocks, so if you create a 1GB zvol with 64KB blocks, and then write zeroes to the entirety of the zvol, what happens is that ZFS gets the first 64KB block, compresses it down very efficiently, writes it out (probably in a single sector), and then repeats that for an additional 16,383 blocks. You end up using 16,384 sectors of raw disk, or 8MB of raw disk.

So the thing is, this means that there's a LARGE amount of raw space that hasn't actually been used inside this zvol. That works out well for pool occupancy rates, but it is also a little dangerous because if you then go and fill the zvol with data, your occupancy rate explodes.

 

[jms123]

Sorry, I should have been more precise with the figures but what I meant was what if I had already created a zvol using 80% instead of 50% as recommended.

Is the only way to fix this to recreate the zvol to use 50% of the pool or can I just make sure I never use more than what would amount to 50% of the pool so approx 60% of the zvol which would amount to just under 50% of the total pool space ?

 

[Arwen]

On the subject of AVAIL space, looks like the GUI does not normally set a reservation. You can manually set one;
zfs set reservation=730G TestVPS-Pool/TestVPS-Zvol
This will remove the reserved space from being available to any other use.