Sizing for ZVOL with compression

[vonion]

I created zvol for iscsi pupose, the size is 11.1TiB, which also present to my ESX host as 11TB. After I ported data into the datastore, now it used 8.4TB or 2.6TB available. However, on FreeNAS GUI volume page, it shows I have 7.7TiB available, which makes sense since I have a compression ratio of 1.64x. But it seems from ESX side I will never benefit from the compression since it calculates the actual data usage instead of compressed usage.

I think all SAN works this way, but on other SAN I'm able to oversubscribe to make sure of the dedup or compression, however in FreeNAS 11.1 is the limit and won't let me go over.

*I have zpool using 9 x SM863a 1.92TB in raid-z2

 

[bigphil]

Using the zfs sizing calculator here, that seems about right for your disk size and pool layout.

I think all SAN works this way, but on other SAN I'm able to oversubscribe to make sure of the dedup or compression

That's the difference between block storage vs file storage (iSCSI/FC vs NFS). If you were to configure a dataset on FreeNAS and connect it to ESXi via an NFS share, you'd see the compression (and dedupe if you used it...typically not recommended) benefits where with iSCSI you wont. Both have advantages and disadvantages and each array vendor is different. In FreeNAS, the benefits of iSCSI are full support of the VAAI primitives (when using zvol and device extents) and native multi-pathing support. NFS advantages in FreeNAS would be ease of configuration, able to see files without mounting a block device (for recovery, editing, etc), and array space savings techniques. Each has a use case...you just need to determine whats most important for you.

 

[Stux]

Wouldn’t the ZFS compression be transparent to the NFS client too?

Except for amount of Free space I guess.

 

[Ericloewe]

You can't have a magically changing size on an iSCSI share, that would break all sorts of things on the clients.

As for using the "extra" space, you probably don't want to. ZFS is very sensitive to free space and 50% is the usual rule of thumb. If your data turns somewhat less compressible, you end up deeper on the losing end of the performance curve.

 

[jgreco]

You can't have a magically changing size on an iSCSI share, that would break all sorts of things on the clients.

As for using the "extra" space, you probably don't want to. ZFS is very sensitive to free space and 50% is the usual rule of thumb. If your data turns somewhat less compressible, you end up deeper on the losing end of the performance curve.

That's probably too simplistic. Once you hit 50%, your system is already at a point where it is probably destined for suckiness as it progresses towards a fragmentation steady state.

https://extranet.www.sol.net/files/freenas/fragmentation/delphix-steady-state.png

What it works out to is that ZFS is able to allocate space very rapidly given lots of free space, so you can have a slowish pool at 50% occupancy or one that is around six times faster at 10% occupancy. Once you pass 50%, things get progressively worse.

The problem is that a ZFS pool will perform spectacularly when it is new and fresh, even up to 90% or more, so people sometimes make the mistake of misunderstanding what this is, fill their pool to 80%, then think "well this is performing great" and commit to that. The problem is that fragmentation is low in that initial data load, so writes are fast. What's being discussed here is fragmentation's effect on performance over time, and how the pool will perform at certain occupancy rates after lots of writes and activity have caused fragmentation to reach their ultimate plateau ("steady state").

I submit that just saying 50% is a rule of thumb without the full explanation is probably a disservice, but you are correct in that if the data is less compressible, you cross deeper into the losing end of the performance curve.

 

[Ericloewe]

I submit that just saying 50% is a rule of thumb without the full explanation is probably a disservice,

It's a good point, but it's a discussion too complex to constantly repeat. Do you have a resource covering this already, so that we can point people towards it?

 

[jgreco]

It's a good point, but it's a discussion too complex to constantly repeat. Do you have a resource covering this already, so that we can point people towards it?

I thought you had already taken all my important posts and made them resources. I really have no idea because I keep finding posts moved to stupid places, and searching doesn't seem to cover resources, and searching also doesn't seem to find other things that it ought to, so basically it's a little frustrating. If I search for "https://extranet.www.sol.net/files/freenas/fragmentation" which ought to reveal all the ones where I went to the most effort to explain, I get 36000 results, but most of them are crap. Whoever is maintaining the forum is doing a poor job at making sure the simple things work. Posts lag, posting silently in the background, returning without appearing to have posted. Searches don't work. Clicking on search results take you to the home page. Posts are mysteriously "edited". I'd probably get banned for life for saying what I really think about the current state of things.

 

[Ericloewe]

I thought you had already taken all my important posts and made them resources.

I tried, back when we finally got the Resources section to work (talk about wasted years of stickies and how-tos and everything else spread around...). I'll go through the Resources section and see if there's anything there...

And no, as far as I can see, none of your existing resources cover the topic of performance versus free space.

 

[jgreco]

I tried, back when we finally got the Resources section to work (talk about wasted years of stickies and how-tos and everything else spread around...). I'll go through the Resources section and see if there's anything there...

And no, as far as I can see, none of your existing resources cover the topic of performance versus free space.

So... I can blame you for that, right? (Because I've never written a resource.)

I make you a deal. If you promise to point people at my resource when questions like this come up, I guess I can see my way to writing out some more-detailed-than-usual explanation. One of the reasons I started with the stickies in the first place is because it was so tedious to repeat myself, which led to less-good answers, and this is by far the most tedious repeated topic I seem to discuss.

 

[Ericloewe]

. If you promise to point people at my resource when questions like this come up

You've got a deal.