Compression on Jails?

[winnielinnie]

Here's a question for TrueNAS Core users. (SCALE users, go away! You're too elitist for us! Hmph!)


Would you find it useful, when creating a new jail, if you could select the compression of its root dataset?


Here's why I ask: I notice that a freshly created jail's "root" is highly compressible, yet its root dataset defaults to LZ4 compression. You can manually set this dataset's compression to ZSTD after creation, which will apply to new files and package updates. Yet, it's a "bit too late" by this point. A jail's filesystem is read from much more often than it is written to.

If there was an option to set the jail's root dataset compression (upon creating a new jail), then you can take advantage of a highly compressible dataset by setting it to use ZSTD before it gets populated.


UPDATE: Jira Suggestion ticket created. Please "vote" for this ticket.

[NAS-122557] - iXsystems TrueNAS Jira

ixsystems.atlassian.net

 

[Patrick M. Hausen]

Is it actively set to LZ4 or is that inherited from the iocage/jails dataset?

 

[winnielinnie]

Oh, snap. Good catch!

Apparently, upon creating your first jail, the new "iocage" and "jails" datasets are set to LZ4. Any subsequently created jails will inherit this. Looks like manually setting "iocage" and "jails" to ZSTD will result in newly created jail roots inheriting ZSTD.

Still not as "granular" are setting it per new jail, but this works!

(Hard to catch these things without a full fledged vanilla "playground" server. I had been manually changing each jail's root dataset to ZSTD.)

 

[winnielinnie]

Fresh new 13.2-RELEASE Basejail with (inherited) LZ4 compression from "iocage/jails":

Code:

du -msA root 1223
du -ms  root  868

Fresh new 13.2-RELEASE Basejail with (inherited) ZSTD compression from "iocage/jails":

Code:

du -msA root 1223
du -ms  root  720

 

[HoneyBadger]

In my example below, I set up a new pool, selected it for jail use, and then manually changed compression level for the top-level iocagedataset to ZSTD. Have a look at the inheritance tree below - there's a lot of lz4/local

Code:

core13u51/iocage                                            compression  zstd            local
core13u51/iocage/download                                   compression  lz4             local
core13u51/iocage/download/13.1-RELEASE                      compression  lz4             local
core13u51/iocage/images                                     compression  lz4             local
core13u51/iocage/jails                                      compression  lz4             local
core13u51/iocage/jails/zstd-inherit                         compression  lz4             inherited from core13u51/iocage/jails
core13u51/iocage/jails/zstd-inherit/root                    compression  lz4             inherited from core13u51/iocage/jails
core13u51/iocage/log                                        compression  lz4             local
core13u51/iocage/releases                                   compression  lz4             local
core13u51/iocage/releases/13.1-RELEASE                      compression  lz4             inherited from core13u51/iocage/releases
core13u51/iocage/releases/13.1-RELEASE/root                 compression  lz4             local
core13u51/iocage/releases/13.1-RELEASE/root@zstd-inherit    compression  -               -
core13u51/iocage/templates                                  compression  lz4             local

@winnielinnie this would be a good candidate for an improvement or feature request if you'd like to submit one :)

 

[winnielinnie]

Have a look at the inheritance tree below - there's a lot of lz4/local

Which means that "iocage/jails" specifically must be set to ZSTD.

@winnielinnie this would be a good candidate for an improvement or feature request if you'd like to submit one :)

I'm not quite sure how to word the feature request.



Have all jails (and their roots) inherit compression from further up? (That is, inherit from "iocage" rather than "iocage/jails")

Or...

Have the entire tree below "iocage" inherit compression from "iocage"?

And... (Can be combined with the following (possible) feature requests):

Have "iocage" and "iocage/jails" default to ZSTD compression (and thus its children will inherit this by default)?

And / Or...

Provide an option to set a newly created jail's root dataset compression, as a "per-jail" option.



I think the best combination would be the following, since "iocage", "jails", and all the children below "iocage" populate with highly compressible data, and ZSTD is quite fast:

Have the entire tree below "iocage" inherit compression from "iocage". (This is because the jail roots aren't the only things that benefit from this. So do the "release" and "download" datasets.)

Combined with...

Have "iocage" and "iocage/jails" default to ZSTD compression (and thus its children will inherit this by default).

 

[Patrick M. Hausen]

If you just change iocage and leave iocage/jails at LZ4, of course new jails will continue to inherit that.

Also if you have an arbitrary tree of datasets and any attribute is inherited, the moment you change that at a higher level dataset the lower level ones will change from inherited to local. How would they not?

Also if you start to tinker with this I can picture wanting different algorithms for jails, releases, images, ... so I would definitely start at the iocage/jail level.

Mistake on my part. Inherited properties change with the higher level dataset.

 

[winnielinnie]

Also if you have an arbitrary tree of datasets and any attribute is inherited, the moment you change that at a higher level dataset the lower level ones will change from inherited to local. How would they not?

Changing "iocage/jails" from ZSTD to LZ4 likewise changed all my jails (and their "roots") to LZ4 (inherited).

Changing "iocage/jails" back to ZSTD likewise changed all my jails (and their "roots") to ZSTD (inherited).

So for the "compression" property, if it's set to inherited, it will not revert to "local" when you make a change to the parent (from which it inherits.)

 

[Patrick M. Hausen]

@winnielinnie See corrected post above.

 

[winnielinnie]

Now I'm mentally stuck with if and how I should write a feature request.

(See my conundrum above.)


EDIT: The "unseen" issue here is that the "System Dataset" and "iocage" dataset (and their subsequent children) are unlike all the other datasets you create on your pool.

"System Dataset" and "iocage" are created and configured by the middleware under-the-hood. (Unlike when you create a new dataset in the "Pools" GUI page.)

When creating your first jail, you're not given the option to configure the "iocage" dataset (and its children). The middleware just does a bunch of stuff under-the-hood. It might be that LZ4 (and where children inherit from) is just what the developers decided on a while ago, and it's been like that ever since.

 

[Patrick M. Hausen]

You could assign the jail pool without creating your first jail. If I am not mistaken (again ) that action creates the iocage dataset and releases, images, jails, ...
Change compression, then create your first jail.

 

[HoneyBadger]

You could assign the jail pool without creating your first jail. If I am not mistaken (again ) that action creates the iocage dataset and releases, images, jails, ...
Change compression, then create your first jail.

It does indeed - a brand new setup results in

Code:

core13u51/iocage                                            compression  lz4             local
core13u51/iocage/download                                   compression  lz4             local
core13u51/iocage/images                                     compression  lz4             local
core13u51/iocage/jails                                      compression  lz4             local
core13u51/iocage/log                                        compression  lz4             local
core13u51/iocage/releases                                   compression  lz4             local
core13u51/iocage/templates                                  compression  lz4             local

All sub-datasets are created and set to lz4/local - they can be overridden through the Storage->Pools UI, but this doesn't solve for a scenario where you might want to override the compression level on a per-jail basis, or where you want or expect the pool's compression level (eg: ZSTD) to be inherited down to all datasets including iocage.

Let me ask some questions internally.

 

[Patrick M. Hausen]

What I would prefer is if iocage would not explicitly set these attributes but jut inherit them from the pool. Less magic, more transparency. With a simple documentation entry the user can be advised how to change these according to their needs.

Why does iocage set these in the first place?

 

[winnielinnie]

Why does iocage set these in the first place?

That's why I think it's a relic from the earlier days.



Still would be nice to have more control. (I want to treat "iocage" and its children differently than the rest of my pool. My pool's root dataset is set to LZ4, as are my other (inherited) datasets, with a few exceptions.

If "iocage" simply inherits from the pool's root dataset, then it means it will inherit LZ4 in my case, and likewise the jails and their roots underneath.



What I think can work for most people, and still be clean, is something like this:

Upon first creating / setting an "iocage" dataset, offer a simple drop-down menu to select the compression for "iocage" (and this implies all its children will inherit it down the chain.)

1. Create / set an "iocage" dataset, because you want to start using jails on TrueNAS Core
2. One of the options that you can set during this step is "iocage" compression:
   • Choose from:
     • Inherit (inherits from the pool's root dataset) (DEFAULT)
     • ZSTD (explicit)
     • LZ4 (explicit)
     • Something else, etc, etc, etc
3. This will dictate not only "iocage" dataset's compression, but also its children ("jails", "roots", "releases", "download", etc) will inherit from "iocage"

The above implies that, by default, anything and everything below "iocage" will inherit from "iocage". Yet, you can always override this after-the-fact (in the Pools page). However, I think an added level of granularity won't hurt: an option to explicitly set the compression for a newly created jail. (You'd choose to leave it at the default of "inherit from iocage" or set it to something explicit, such as ZSTD, ZSTD-9, LZ4, etc)

 

[HoneyBadger]

That's why I think it's a relic from the earlier days.

That's the general consensus I've gotten - many moons ago, ZFS defaulted to LZJB compression, so iocage on TrueNAS (or even FreeNAS at the time) was probably set up to use LZ4 explicitly when that was integrated as it offered a significant improvement.

Changing compression level shouldn't impact anything from a POSIX/file level (it will obviously do things to CPU usage if you set it to gzip-9) so I would suggest that if you want to benefit from the improved compression, you set the iocage/jails dataset to ZSTD, so that new jails will inherit this.

 

[winnielinnie]

if you want to benefit from the improved compression, you set the iocage/jails dataset to ZSTD, so that new jails will inherit this.

What about a feature request, worded like this?

• Setting up iocage ("Jails") for the first time will include an option of what compression to use for the iocage dataset (implicitly inherited by its children, including jails and roots.)
• LZ4 is pre-selected as the default. (Can be changed to the user's preference, maybe with a helpful tooltip. Can even be set to "Inherit" from the pool's root dataset.)
• In addition to this feature (first-time setup of iocage / "Jails"), an extra option will be available when creating each new jail. It can be left to the default ("Inherit from parent"), or changed to something else.

Nothing changes with respect to manually setting the compression after-the-fact to specific existing jail datasets. (This is still the same.)

But now the user can set the compression for "all of iocage" when setting up "Jails" for the first time. And the user can also choose the compression for a new jail at the time of creation.

IN SUMMARY: Two new options in the GUI: (1) Compression, for a first-time setup of iocage, and (2) Compression, for each a new jail

The latter can be a small "drop-down" menu in the Jail creation page:

 

[Patrick M. Hausen]

Looks good to me. I really appreciate your effort and if you manage to get a new feature into the jails subsystem after years of stagnation, you will earn an extra round of applause.

 

[winnielinnie]

I'll make a feature request shortly!

 

[winnielinnie]

Ticket submitted!

[NAS-122557] - iXsystems TrueNAS Jira

ixsystems.atlassian.net

Make sure to smash that like button and subscribe if you wa-,

Please "vote" for this ticket so that it might gain more eyes and attention. (Feel free to share with others the link to the ticket, as well as this forum thread.)

Special thanks to @HoneyBadger and @Patrick M. Hausen

 

[HoneyBadger]

and don't forget to ring that notification bell