What is the future of TrueNAS CORE?

[Patrick M. Hausen]

As Kris pointed out Linux containers provide better App lifecycle management where the App developer has more control and responsibility. App developers prefer this model. Most users enjoy this simplicity.

However, FreeBSD Jails are simple and powerful, if the (more technical) administrator wants to manage their own App lifecycle.

What I simply was not aware of is the fact that the containers are provided by the upstream projects. I always though, hey when they (iX) can create Linux containers that work why do they so neglect the jail based plugins? It's the same amount of work, right? Or would be if you were to create and maintain the containers yourself.

Now I see the appeal. I still hope there will be a FreeBSD 16, 17, ... and that in 20 years we can celebrate 50 years of FreeBSD like we did the 30th anniversary last year in Coimbra.

EDIT: still you will have to either move to FreeBSD 14 or 15 or make maintenance of CORE jails practically impossible some time in 2026, which is only two years away. FreeBSD 13 will be EOL by then.

 

[Whattteva]

What I simply was not aware of is the fact that the containers are provided by the upstream projects. I always though, hey when they (iX) can create Linux containers that work why do they so neglect the jail based plugins? It's the same amount of work, right? Or would be if you were to create and maintain the containers yourself.

Yeah, I would hope so. I myself maintain "my own version" of Jellyfin, Transmission, Caddy, etc. containers using a combination of BastilleBSD templates + ports. I admit that occasionally, some upgrades may cause breakage, but it's usually fixed by either contacting the port maintainer or just locking the port version or keeping the jail version behind (ie. 13.2 jail on 14.0 FreeBSD for OpenSSL upgrade breakage). It's what makes the jails flexible after all, the fact that you can run a different version independent of the currently running kernel.

Now I see the appeal. I still hope there will be a FreeBSD 16, 17, ... and that in 20 years we can celebrate 50 years of FreeBSD like we did the 30th anniversary last year in Coimbra.

EDIT: still you will have to either move to FreeBSD 14 or 15 or make maintenance of CORE jails practically impossible some time in 2026, which is only two years away. FreeBSD 13 will be EOL by then.

I hope so, too. Honestly, I think if projects like NetBSD can stay alive despite the incredibly niche userbase, I think FreeBSD (as the largest BSD) should be able to do so relatively easily.

 

[Kris Moore]

I'm not sure sidegrading to SCALE today would allow me and my system for a relatively simple moving of my jails with mounting points without data loss, but that might just be me not having looked deep enough in the process... as I don't even know whether mount points are even possible in a VM under SCALE (which as of today looks to be the cleanest way of moving to SCALE).

That being said, being able to hop from one to the other is indeed a great thing. None here undermines the great work you have done on SCALE.

The sidegrade to SCALE will preserve all your "data", but in the case of Jails you can't directly migrate those, since a FreeBSD jail/chroot isn't going to run on Linux. You can manually tar up the jail data and move it to a VM running FreeBSD, then setup NFS mounts, etc. Personally I'd recommend just setting up a Linux / Debian sandbox/jail though, migrate my Jail app configs over and just manage it via `apt` and friends. That would be far more efficient than a jail inside a VM, and you can access storage directly instead of over a protocol like NFS.

 

[Kris Moore]

Yeah, I would hope so. I myself maintain "my own version" of Jellyfin, Transmission, Caddy, etc. containers using a combination of BastilleBSD templates + ports. I admit that occasionally, some upgrades may cause breakage, but it's usually fixed by either contacting the port maintainer or just locking the port version or keeping the jail version behind (ie. 13.2 jail on 14.0 FreeBSD for OpenSSL upgrade breakage). It's what makes the jails flexible after all, the fact that you can run a different version independent of the currently running kernel.

That's where I came from also. All my apps were in a jail, and I just wrangled 'pkg' updates every few weeks, chasing fallout occasionally as upstream packages/ports moved things around. But when I went to SCALE I just set aside a Saturday morning and migrated all those configs over to Containers. The initial setup pain was far outweighed by the fact that I can update often, sometimes 2-3x a week just by clicking a single button in the UI and waiting 30 seconds or so. I'm almost tempted to have us make a feature next for SCALE to auto-update apps on a schedule that the user can set, it's that painless.

 

[winnielinnie]

What about this alternative option? (I'm just spitting freestyle, so pardon if this makes no sense.)

1. Keep your iocage/jails datasets. (I have them compressed as ZSTD, and they are lean, even for a full vanilla FreeBSD.)
2. Using whatever "App" (via official catalog or TrueCharts), point its configuration to the already existing path under <pool>/iocage/jails/<jailname>/root/path/to/config

Not sure how feasible this is. It also doesn't take into account other configurations, such as WireGuard and custom scripts within the jail. It also doesn't address the "Swiss Army knife" need.

 

[Kris Moore]

FYI - The docs page for the new "Sandboxes/Jails" on the upcoming Dragonfish release has gone live:

Sandboxes (Jail-like Containers)

Provides advanced users information on deploying custom FreeBSD jail-like containers in SCALE.

www.truenas.com

24.04-BETA.1 is tenatively slated for a week from now, 2/6. Appreciate any feedback on this before then.

 

[Kris Moore]

What about this alternative option? (I'm just spitting freestyle, so pardon if this makes no sense.)

1. Keep your iocage/jails datasets. (I have them compressed as ZSTD, and they are lean, even for a full vanilla FreeBSD.)
2. Using whatever "App" (via official catalog or TrueCharts), point its configuration to the already existing path under <pool>/iocage/jails/<jailname>/root/path/to/config

Not sure how feasible this is. It also doesn't take into account other configurations, such as WireGuard and custom scripts within the jail. It also doesn't address the "Swiss Army knife" need.

That may not work cleanly, since likely a lot of those "app" configs inside of the jail will need some adjustment to point to their new container equivalent location for settings, files, etc. To give you an example of how I did my migration of Plex back in the day, I more or less made a copy of the Plex configuration DB directories, and then followed their guide to migrate to another server.

Move an Install to Another System

It is possible to move a Plex Media Server installation from one computer/system to another in most cases. It won’t...

support.plex.tv

 

[victort]

FYI - The docs page for the new "Sandboxes/Jails" on the upcoming Dragonfish release has gone live:

Sandboxes (Jail-like Containers)

Provides advanced users information on deploying custom FreeBSD jail-like containers in SCALE.

www.truenas.com

24.04-BETA.1 is tenatively slated for a few week from now, 2/6. Appreciate any feedback on this before then.

Awesome. Was looking for this.

 

[Whattteva]

That's where I came from also. All my apps were in a jail, and I just wrangled 'pkg' updates every few weeks, chasing fallout occasionally as upstream packages/ports moved things around. But when I went to SCALE I just set aside a Saturday morning and migrated all those configs over to Containers. The initial setup pain was far outweighed by the fact that I can update often, sometimes 2-3x a week just by clicking a single button in the UI and waiting 30 seconds or so.

I would tend to agree with this sentiment especially for less tech-savvy users.

I'm almost tempted to have us make a feature next for SCALE to auto-update apps on a schedule that the user can set, it's that painless.

As far as painlessness though. I suppose it's not entirely painless right now because I've been seeing a lot of users complaining about apps stuck in "Deploying" and other issues especially after a major release announcement (ie. Cobia). I admit that a majority of this can be attributed to the unofficial TrueCharts apps, but like it or not, TrueCharts Apps is what drew a LOT of users to SCALE.

Here's one example of those pains that I mentioned.

Cobia shouldn't have made it to the update train

This is the most terrible user experience I've ever encountered in years. Everything is just so slow and rubbish. The interface is caching stuff, so for example in the apps list you'll have the wrong information shown until you press shift+F5. But refreshing on Cobia is a nightmare and will...

www.truenas.com

 

[Kris Moore]

I would tend to agree with this sentiment especially for less tech-savvy users.


As far as painlessness though. I suppose it's not entirely painless right now because I've been seeing a lot of users complaining about apps stuck in "Deploying" and other issues especially after a major release announcement (ie. Cobia). I admit that a majority of this can be attributed to the unofficial TrueCharts apps, but like it or not, TrueCharts Apps is what drew a LOT of users to SCALE.

Here's one example of those pains that I mentioned.

Cobia shouldn't have made it to the update train

This is the most terrible user experience I've ever encountered in years. Everything is just so slow and rubbish. The interface is caching stuff, so for example in the apps list you'll have the wrong information shown until you press shift+F5. But refreshing on Cobia is a nightmare and will...

www.truenas.com

That's fair and until some of those issues are smoothed out its whats preventing me from the auto-updater feature. In the end, we can't control all TrueCharts quality issues, but we'll get our official apps at least to the point where I feel better about offering this at some point.

 

[morganL]

What I simply was not aware of is the fact that the containers are provided by the upstream projects. I always though, hey when they (iX) can create Linux containers that work why do they so neglect the jail based plugins? It's the same amount of work, right? Or would be if you were to create and maintain the containers yourself.

EDIT: still you will have to either move to FreeBSD 14 or 15 or make maintenance of CORE jails practically impossible some time in 2026, which is only two years away. FreeBSD 13 will be EOL by then.

For 2024, the focus will be on getting 13.1 to the quality level of 13.0. CORE 13.0 is still the largest part of the TrueNAS community.

We understand that we have to maintain CORE and its primary features. We do not yet have a 2025 plan. It will depend on completing and prioritizing the list of features/vulnerabilities that are required. From that, we we will assess the technical options for moving forward. If we don't sustain a large community, we should expect to be forked.

 

[victort]

For 2024, the focus will be on getting 13.1 to the quality level of 13.0. CORE 13.0 is still the largest part of the TrueNAS community.

We understand that we have to maintain CORE and its primary features. We do not yet have a 2025 plan. It will depend on completing and prioritizing the list of features/vulnerabilities that are required. From that, we we will assess the technical options for moving forward. If we don't sustain a large community, we should expect to be forked.

You mean there’s still hope for us?

 

[Kris Moore]

You mean there’s still hope for us?

I'm not a fan of false hope, so don't rely on that too much

 

[victort]

I’m jesting. I fully understand your reasoning. But I would still like it if CORE wins out. Until that or other things happen, we need to take life as it comes, and make the wisest decisions we can. Indecision is wrong decision.

 

[Redcoat]

and make the wisest decisions we can.

And to do that with some chance of success for our individual situations we are very much reliant on straight and timely information from iX.

 

[Davvo]

@morganL @Kris Moore please just state things as they are when the time comes: we got ourselves burned already by the "SCALE will exist alongside CORE" message; no sugarcoating, no "megacorp" PR. As you did in this thread.

It would really make a difference to this community.

 

[ChrisRJ]

What I have always been wondering about in this discussion is how much it seems(!) to leave out the paying customers. There will of course always be edge-cases. But I fail to see how a large-enough portion of the existing as well as the future paying customer base would use the application side beyond what is possible with Core today.

My, perhaps wrong, assumption of the ICP is that they want serious storage for a fair price and without the risk of vendor lock-in. For edge-computing I can see a case for hyper-converged on the one hand. But on the other, the applications in question, will mostly be commercial and as such come with HCLs that will probably exclude them from running on TrueNAS.

Am I completely off with these thoughts?

 

[Johnny Fartpants]

I completely agree @ChrisRJ

 

[Ericloewe]

Am I completely off with these thoughts?

Yes and no. The practical reality is that there are some Storage-adjacent applications that fit neatly into the "run it on the NAS" model. Nextcloud's been mentioned, and it adds meaningful functionality - but it's not something a customer who ordered a monster HA system to provide storage for a bunch of VMs is likely to care about, they'd just run it elsewhere.

A big part of it is that server hardware is getting ever "larger" - if you'll pardon the imprecise wording - and thus practically demands consolidation. In the not-so-distant past, you might realistically buy a rackload of machines doing smallish tasks - case in point, I have worked with 2U rack workstations (Westmere-EP dual-Socket platform), configured with a single four-core CPU and 4 GB of RAM, and running Windows 7. Even when these were bought, this was absolutely ridiculous and puny, but it had to be because VMs were not mature enough (and the hardware to really make them viable didn't arrive until a generation later as core counts increased). It did help drive the cost of the solution up into the stratosphere, and that's not viable if the guy next door can keep his costs and thus prices down.
Today, you can buy a beefy server from a decade ago (e.g. a Dell R630), with dual Xeon E5s, and for next to nothing get 20 cores or more, in a server that is still viable for all sorts of not-cutting-edge workloads. And if it idles a lot, it's not going to be much worse power-wise than a brand-new system.

So, if you can run little bits and pieces of your infrastructure on your NAS, that's one less machine you need, which immediately saves cash and meaningfully reduces power consumption. That sort of thing changes the value proposition and can make a big difference - in some cases.

 

[ChrisRJ]

So, if you can run little bits and pieces of your infrastructure on your NAS, that's one less machine you need, which immediately saves cash and meaningfully reduces power consumption. That sort of thing changes the value proposition and can make a big difference - in some cases.

I completely agree, and actually this scenario was more or less what I had in mind. My point is: How many of those systems will make ixSystems any money?