What is the future of TrueNAS CORE?

[victort]

I actually appreciate how iocage handles jails and ZFS datasets (filesystems & clones), and would love for BastilleBSD to do something similar (does it? Not too sure), but I agree that having ZFS should not be a requirement for a jail manager.

Is it possible to import an iocage jail with bastille?

 

[Juan Manuel Palacios]

Is it possible to import an iocage jail with bastille?

I honestly don't know. My tinkering with BastilleBSD has so far been rather limited, as currently I don't have a spare server handy to deploy that stack to and flesh out my infrastructure with it, and I'm trying to avoid standing up too many VMs on my TrueNAS rig.

Added to that, I've been working for a while already to make all my jails disposable, so I can quickly create them (via scripts that utilize private iocage plugin catalogs), mount data into them, dispose of them, and migrate to new ones, so I really haven't had much use for iocage's export functionality, either.

 

[victort]

XigmaNAS has a bastilleBSD extension. Playing around with it a bit…

XigmaNAS - Login

 

[winnielinnie]

We opted for now to use systemd-nspawn, which is much lower level like Jails are. It means you can run LXD/LXC, Docker, or $OTHER container tools within.

Three questions. Pretend I'm five years old.

1. Technically, it means we can arbitrarily install and run anything available from the Debian repositories? No need to rely on any type of "container tool", correct? (Essentially, we'd use "apt" and "systemctl" inside a chroot, rather than "pkg" and "sysrc" inside a jail).
2. systemd-nspawn in TrueNAS SCALE will have a first-class GUI? It won't simply be "available" via the terminal or TN's CLI tool?
3. Will SCALE users have an official (and clean) way to completely disable K3s?

 

[rvassar]

I'm a long time BSD fan... I grew up in the Kali east bay, back when you could hop on BART with a blank tape and (*cough*) a nickle bag, go find a grad student and trade for the latest BSD src. I remember going to my college library (I didn't make UCB's list... and no I don't partake of the stuff myself, I'm dumb enough naturally...) in the 80's and reading each installment of the Jolitz articles for 386BSD... And I get the bit about attachment... I got sucked into Sun until (credit B. Cantrill's allegorical labeling...) the Nazi's took it over, and got dumped after 16 years...

FWIW - My first Linux kernel was 0.95a... Downloaded on actual "floppy" floppy's... I would challenge @jgreco's Grinch for curmudgeonness, but he's so much better at it than I am...

Is it possible to import an iocage jail with bastille?

This is probably my problem... I have two down-rev jails. I don't want to port them to Linux/Docker/LXC/Etc... for the foreseeable future. Can SCALE give me a "BSD-JAIL" VM that can import them from TrueNAS CORE without me fiddling around with the networking, etc...? Even for a transitional horizon of year or three? I know I can pull this off myself after a day or two of work, but... I don't have time to poke at it and deal with the CORE -> SCALE uncertainties. I'm a power user, but not a paying power user. My understanding is this upgrade requires me to use a vacation day or two. Is there a solution that solves this problem in a day or two for all of us that don't have a day or two? Consider this may be one of the bigger logjams remaining in the CORE -> SCALE transition for actual paying iX customers.

 

[Whattteva]

I actually appreciate how iocage handles jails and ZFS datasets (filesystems & clones), and would love for BastilleBSD to do something similar (does it? Not too sure), but I agree that having ZFS should not be a requirement for a jail manager.

Honestly, I think using ZFS snapshot/clone isn't a good way to go about it unless your jails stay mostly passive, which I highly doubt is the case. It's nice in the beginning when everything is still mostly identical, but typically that won't stay very long. My jails diverged quite a bit because they all require different packages and some even still require 13.2 kernel cause the packages aren't updated to support 14.0 yet. Moreover, you have this dilemma that you can never delete that base snapshot that all the clones depend on until you've gotten' rid of all your child jails.

 

[Patrick M. Hausen]

Is it possible to import an iocage jail with bastille?

Definitely although I do not know how complicated it's going to be

An iocage jail is essentially a collection of ZFS datasets, an fstab file and a config.json file. That's it. All plain text and easily converted to some different format.

 

[NugentS]

Three questions. Pretend I'm five years old.

1. Technically, it means we can arbitrarily install and run anything available from the Debian repositories? No need to rely on any type of "container tool", correct? (Essentially, we'd use "apt" and "systemctl" inside a chroot, rather than "pkg" and "sysrc" inside a jail).
2. systemd-nspawn in TrueNAS SCALE will have a first-class GUI? It won't simply be "available" via the terminal or TN's CLI tool?
3. Will SCALE users have an official (and clean) way to completely disable K3s?

1. I have so far
2. No GUI
3. Not yet

Have a look in my sig (Jira Suggestions) for 1 & 8

 

[danb35]

2. No GUI

I think the better answer here is also "not yet"--iX seem to expect that a GUI for nspawn could happen, but not in the next fishy release.

 

[victort]

Honestly, I think using ZFS snapshot/clone isn't a good way to go about it unless your jails stay mostly passive, which I highly doubt is the case. It's nice in the beginning when everything is still mostly identical, but typically that won't stay very long. My jails diverged quite a bit because they all require different packages and some even still require 13.2 kernel cause the packages aren't updated to support 14.0 yet. Moreover, you have this dilemma that you can never delete that base snapshot that all the clones depend on until you've gotten' rid of all your child jails.

For me, jails have always been simply a way to run services, have their own IP, and be easily built and destroyed. Data stays outside the jail. Period. If I can’t do that, I won’t use it. Having to back up a database before deleting and reinstalling an app (as in Truecharts) just seems awkward, especially if there is an alternative way of doing it.

Iocage, BastillBSD and I’m sure others have a way of doing all this and much more that containers in Linux just don’t.

 

[NugentS]

All my containers (both K3S and Docker (under Portainer)) except where it really doesn't matter (like filebrowser & netdata) use data outside the jail. I don't use PVC's which are a bloody stupid idea (IMHO), designed to lose data, make restores / recovery difficult etc

 

[victort]

All my containers (both K3S and Docker (under Portainer)) except where it really doesn't matter (like filebrowser & netdata) use data outside the jail. I don't use PVC's which are a bloody stupid idea (IMHO), designed to lose data, make restores / recovery difficult etc

That’s one way of doing it, sure.

 

[Kris Moore]

Three questions. Pretend I'm five years old.

1. Technically, it means we can arbitrarily install and run anything available from the Debian repositories? No need to rely on any type of "container tool", correct? (Essentially, we'd use "apt" and "systemctl" inside a chroot, rather than "pkg" and "sysrc" inside a jail).

Correct, think of systemd-nspawn as a full "jail". It means you can run apt, systemctl, or even nest other container tech inside it. LXD, Docker, K3s, podman, pretty much whatever you want. Its a full sandbox, that persists across upgrades and gives you ultimate flexibility without worrying about messing up your TrueNAS base installed image.

EDIT: Forgot to mention, it doesn't have to be "Debian" either. Ubuntu, Arch and other "Distros" are possible, so you have some flexibility if Debian isn't your thing :)

1. systemd-nspawn in TrueNAS SCALE will have a first-class GUI? It won't simply be "available" via the terminal or TN's CLI tool?

Command-line only in Dragonfish. But on our road-map to see if we can make it all point-n-clicky to manage and deploy sandboxes later. Ultimately it is a full "jail-like" environment though, so you will need to be semi comfortable enough with the CLI to do things like using `apt` and friends.

GitHub - Jip-Hop/jailmaker: Persistent Linux 'jails' on TrueNAS SCALE to install software (docker-compose, portainer, podman, etc.) with full access to all files via bind mounts thanks to systemd-nspawn!

Persistent Linux 'jails' on TrueNAS SCALE to install software (docker-compose, portainer, podman, etc.) with full access to all files via bind mounts thanks to systemd-nspawn! - Jip-Hop/jai...

github.com

1. Will SCALE users have an official (and clean) way to completely disable K3s?

This can be done today, straight from the UI. Apps -> Settings -> Unset Pool will completely turn off K3s.

 

[Kris Moore]

For me, jails have always been simply a way to run the services, have its own IP, and be easily built and destroyed. Data stays outside the jail. Period. If I can’t do that, I won’t use it. Having to back up a database before deleting and reinstalling an app (as in Truecharts) just seems awkward, especially if there is an alternative way of doing it.

Iocage, BastillBSD and I’m sure other have a way of doing all this and much more that containers in Linux just don’t.

I think you are confusing "Containers" with "Jails". Containers are read-only images, and aren't designed to have that kind of crazy customization like a jail does. Its for ease of deployment, and sane upgrade paths. It's why for 2+ years I've been able to update plex, jellyfin and friends with just a single click on a UI button.

On Linux side, the Jail equivalent would be something like systemd-nspawn, LXD or similar technology. You get the same flexibility as jails, and I'd argue even more with native support for things like GPU passthrough and nested containers, etc. All the same options for network configuration, customization, etc etc.

 

[victort]

I think you are confusing "Containers" with "Jails". Containers are read-only images, and aren't designed to have that kind of crazy customization like a jail does. Its for ease of deployment, and sane upgrade paths. It's why for 2+ years I've been able to update plex, jellyfin and friends with just a single click on a UI button.

On Linux side, the Jail equivalent would be something like systemd-nspawn, LXD or similar technology. You get the same flexibility as jails, and I'd argue even more with native support for things like GPU passthrough and nested containers, etc. All the same options for network configuration, customization, etc etc.

Thanks for that.

 

[victort]

I guess I’ve also realized (like you) that we do have a life outside of all this. And again, life is too valuable to hold on to something that takes away from it.

If I have to sit endless hours to maintain a service on CORE that could easily be maintained using SCALE, that reduces the value of the life we have outside of all this.

Unless that is your life of course, then maintain away.

But some of us do this all as a hobby…

 

[Kris Moore]

I guess I’ve also realized (like you) that we do have a life outside of all this. And again, life is too valuable to hold on to something that takes away from it.

If I have to sit endless hours to maintain a service on CORE that could easily be maintained using SCALE, that reduces the value of the life we have outside of all this.

Unless that is your life of course, then maintain away.

But some of us do this all as a hobby…

That was the realization I came to a few years ago. I was like many BSD users on this thread. I had a sweet system setup for my jails, custom scripts, the whole nine yards. Everything dialed-in just so. But at some point real-life intruded and I realized I didn't like spending more time than I had to maintaining things just to stay on top of updates. I had more interesting things to do. This included my TrueNAS Jails, as well as some FreeBSD bare-metal and VM machines I hosted for other things. I spent the better part of a single Saturday converting everything to containers and haven't looked back since. I update my apps every few days (Even updated my kids new PalWorld server 3x alone this week). Each time I just click a button and walk away. I can't emphasize enough how much I appreciate the time-savings now.

I should mention, this also came with a bit of grudging on my part. I had to learn the ins-n-outs of this fancy new docker / linux container ecosystem, but once I got over those hurdles where there were gaps in my knowledge, the rest just fell into place and made sense.

 

[Davvo]

That's encouraging.

 

[Juan Manuel Palacios]

Honestly, I think using ZFS snapshot/clone isn't a good way to go about it unless your jails stay mostly passive, which I highly doubt is the case. It's nice in the beginning when everything is still mostly identical, but typically that won't stay very long. My jails diverged quite a bit because they all require different packages and some even still require 13.2 kernel cause the packages aren't updated to support 14.0 yet. Moreover, you have this dilemma that you can never delete that base snapshot that all the clones depend on until you've gotten' rid of all your child jails.

As @victort said above, my jails are pretty much disposable, container-like isolated execution environments where I deploy an app and expose it as a LAN service, nothing more. They're still read/write and persistent, of course, as opposed to real Linux containers, and they may even live a long time before I swap them out for replacement jails when the need comes. But the point is they're still disposable, there's nothing in them that I have to stop for a second to look at before deleting and replacing them with upgraded jails. Everything of value is simply mounted into them, or pushed to them via reproducible deployments (e.g. CI/CD pipelines).

This way, they will of course diverge from their origin, e.g. every time I run pkg update && pkg upgrade, but in a very manageable way.

As for not being able to delete the release dataset from which a specific jail was cloned, I don't see any issue with that, I think that makes perfect sense.

 

[winnielinnie]

Command-line only in Dragonfish. But on our road-map to see if we can make it all point-n-clicky to manage and deploy sandboxes later. Ultimately it is a full "jail-like" environment though, so you will need to be semi comfortable enough with the CLI to do things like using `apt` and friends.

It's not so much the discomfort of using the terminal, but rather that a GUI will give it a "seat at the table" of a fully-fledged and supported "NAS" feature. *(That, and it also saves time and is easier to manage, rather than doing every step manually.)

Would be nice (that's an understatement!) to have an "iocage equivalent" for systemd-nspawn "Linux jails".

* I honestly don't believe I would have gotten into using jails in TrueNAS Core if there was no iocage + GUI for iocage. (It streamlines a lot of things under-the-hood.)