TrueNAS Scale Release Plan

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
Blog is at: https://www.ixsystems.com/blog/truenas-scale-release-plan/
This is a copy and paste... for questions and comments. Please forgive the rough formatting.

While TrueNAS 12.0 (CORE and Enterprise editions) continues its release march, we’re also busy getting the first version of TrueNAS SCALE into the hands of many tech-savvy users. TrueNAS SCALE 20.10-ALPHA is planned for October and will be codenamed “Angelfish”.

As our initial community post on SCALE indicated, TrueNAS SCALE is defined by its acronym:

SCALE-Acronym.png


TrueNAS starts from the TrueNAS 12.0 base which includes OpenZFS, all the storage services, the middleware to coordinate these, and the web UI to present a user-oriented view of the system. This base has been tested by hundreds of thousands of users over the last few years.

The good news is that nearly all of this base has been preserved with relatively small software changes. For Enterprise users, it has also been possible to port over the High Availability (HA) software, enclosure management, and other Enterprise features. This means that SCALE will be able to run on TrueNAS M-Series and X-Series systems in the future and take advantage of the redundancy.

Being similar to TrueNAS 12.0 is awesome because it means it will be a similar UX, which minimizes the training necessary to get up-to-speed on TrueNAS SCALE, but it’s what you can do with TrueNAS SCALE that’s most exciting. The new capabilities being added define the new opportunities for SCALE:

  • KVM Virtualization: Mature Hypervisor with good reliability, Guest OS support, and enterprise features.
  • Kubernetes: Applications can be single (docker) containers or pods of containers.
  • Scale-out ZFS: SCALE will enable datasets to be defined as ZFS datasets or cluster datasets which span multiple nodes and ZFS pools. Cluster datasets will have a variety of redundancy properties and still support ZFS snapshots.
Unlike other Hyperconverged Infrastructure solutions, TrueNAS SCALE will have deployment benefits as a single node, an HA system, or as a cluster of multiple nodes. Start with a single node system and in the future, you will be able to scale-out.

Given the amount of existing software and new software, we have a release plan that lets the community confidently test and deploy SCALE as it becomes available. The high level plan follows this process.

SCALE-Table.png


“ANGELFISH”
Angelfish.png

Release numbering will be based on Year and Month. The first numbered release will come out in October and will be called TrueNAS SCALE 20.10 (Angelfish). The codenames will be alphabetically sequential and will be associated with aquatic animals that have scales or swim in schools (clusters).

The focus is on characterizing “feature groups” as either PREVIEW, ALPHA, BETA, RC, or RELEASE quality. Users should read the release notes to confirm support for their particular use case. Angelfish is almost feature complete in the NIGHTLY releases and includes the following feature groups:
TrueNAS-SCALE-AngelFish.png

It should be noted that KVM has little testing by this community but is widely used elsewhere. Kubernetes will also be based on stable, released code, but the WebUI and Middleware are expected to be PREVIEW quality.

Clustered datasets require some additional TrueCommand features (expected in November) to provide an easy-to-use WebUI. In the meantime, the CLI and APIs can be tested and this feature group is classified as PREVIEW status.

We appreciate the community feedback and bug reports and hope to get all those features to RELEASE quality faster.

Is TrueNAS SCALE for Users or Developers?
Right now, TrueNAS SCALE is for developers and bug hunters and can be downloaded here. For Linux developers, there are many opportunities to contribute to the Open Source TrueNAS SCALE project. We have made it a very well coordinated and managed environment to develop the best Open Hyperconverged Infrastructure. For more information, see this Community post.

The TrueNAS SCALE Angelfish releases in Q4 will be good for tech-savvy enthusiasts and testers. We’ll let you know when TrueNAS SCALE 20.10 is ready.

In 2021, TrueNAS SCALE is expected to get to full RELEASE quality for a clustered system.
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
Just to be clear, when we are talking "BETA" in Dec 2020, we are talking the oldschool definition of "BETA" being "A (mostly) feature complete release for testing and tweaking"?

Aka, can we expect it to be mostly feature complete by then?
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
Is TrueNAS CORE 12.0 feature complete? There's always additional features people need/want. However, for the set described, the basic features are planned to be there. We expect to see many "suggestions" and hope to see more developers join the effort.

The SCALE project doesn't have an installed "production" base, so the process being used is designed for more rapid development and release.

As indicated in the blog, we do think there will be varying maturity of the components (feature groups). Some feature groups, may be in ALPHA state, but these may not be required in some use-cases. So, read the Release Notes.

Tell us what you think is missing for "feature complete"....
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
@morganL "Feature complete" always meant "all features that we want in release" afaik.
I'm asking because a lot of projects anno 2020, are calling themselves "BETA" while still not having barely half of the features planned for release included.

Alpha or Beta never was a "quality" definition (that was a simplification and a logical consequence), both where meant for testers, Beta was just the one that had mostly all features for release incorporated, whereas ALPHA didnt.
(this did mean BETA was inherently more stable in most cases, but that wasn't the meaning of BETA)

There are always some edge cases of features being added in BETA (for example how zstd support got added in CORE during beta), so I'm carefull in picking my words here. "Feature complete" doesn't mean "any, every and all feature planned for release", more like "All core features planned for release"
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
It's worth documenting our definitions. Since TrueNAS is a storage product and data safety is a critical element, our process is important.

PREVIEW is definitely not feature complete. It is for demo, testing and experimentation. Current NIGHTLY is generally in PREVIEW status.

ALPHA may be feature complete but has had very little testing. Until it's tested, there may be new features added. Expect to find bugs, but OpenZFS provides data safety.

BETA implies a quality level and more importantly some testing has been completed. It is also associated with feature stability and therefore relatively feature complete. The Feature Groups labelled BETA are both feature complete and have had some testing. You may still find a bug, but these bugs are very unlikely to impact data safety.

RC implies feature complete and well tested, but not yet widely used in production. In some unusual situations, you will find a bug.

RELEASE implies, feature complete, well tested in the lab and in the field. Unlikely a normal user will find a serious bug. If it were a vaccine, you might take it.

For TrueNAS SCALE, there will be new features added more quickly. .. we expect to have more "edge cases", but will be very clear about what is being added and what has not been well tested.
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
@morganL You deserve an extra beer next staff activity for this reply: Clear, Precise and KISS. Thank you! :)

While we are at it:
Lets say I setup k8s on SCALE now via CLI, would it be corrupted/affected by the not-so-future API/GUI? (or visa versa)
(don't expect docker to be much of an issue, because it's quite simple... but as you guys are going to be implementating a management layer over k8s, I really don't know how that is going to work out)
 
Last edited:

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
K8s via CLI now would be pre-preview ... developers only.

If you really want to do that, I'd recommend running K8s within a VM, and then you will be isolated from all future Kubernetes development work.
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
K8s via CLI now would be pre-preview ... developers only.

If you really want to do that, I'd recommend running K8s within a VM, and then you will be isolated from all future Kubernetes development work.
Yeah so TLDR: our layer on top of K8S might or might not break custom K8S setups.
Very understandable though! :)

*edit*
@morganL
I looked into the current API PR and you're right: Setting up a cluster on the SCALE node outside of SCALE, will not work right with SCALE it seems. While docker would work fine, it's indeed ill adviced to run k8s yet :)
 
Last edited:

ikke

Contributor
Joined
Apr 22, 2012
Messages
124
Any considerations having podman available as an option to Docker? Benefits being no daemon, and can run containers as a user -> security. I switched there already a while back from docker.
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
Any considerations having podman available as an option to Docker? Benefits being no daemon, and can run containers as a user -> security. I switched there already a while back from docker.

The first versions are focussed on Kubernetes so that the system can run docker containers or Helm pods... either a s a single node or as a cluster.

Is there any specific functionality that would be missing with this approach?
 

ikke

Contributor
Joined
Apr 22, 2012
Messages
124
I am only a NAS user, so personally kubernetes is overkill for my single node NAS. I run it on clusters at work, but at home I have currently ansibled all my containers using podman, as they are static and need no scheduling. Kube would be great otherwise, but overkill for single home NAS nodes. For home systemd controlled podman pods are good choice.
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
Ikke, SCALE should be able to recreate that simpler user experience, even though Kubernetes is the underlying technology. More work to be done, but that is the intention.

Within a VM, you could run Podman or any other preferred package. That is available today.
 

ikke

Contributor
Joined
Apr 22, 2012
Messages
124
Ah, I thought the kube was running on host, not VM. I already do that using Fedora-IoT VM, kinda minimal only container purposed VM. It uses podman, and is nice to maintain using cockpit GUI. I was thinking of getting rid of virtualization and NFS, which would be ideal if host is Linux.

Anyhow, it was just a wish, thanks for replying!
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
Ah, I thought the kube was running on host, not VM. I already do that using Fedora-IoT VM, kinda minimal only container purposed VM. It uses podman, and is nice to maintain using cockpit GUI. I was thinking of getting rid of virtualization and NFS, which would be ideal if host is Linux.

Anyhow, it was just a wish, thanks for replying!
Kubernetes is running on the host. I was just pointing out that anyone that wanted their own flavor of kubernetes or docker could use a VM to do this.
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
Slight addition to clear up what Morgan wrote:
- For single node users you will have Five options:
1. Use docker without kubernetes on SCALE
2. Use kubernetes (which also works perfectly fine without multiple nodes, but has a learning curve)
3. Use a custom docker management system such as portainer. Although this might cause issues, at the current stage it should still be possible to do
4. Use the To-Be-Released SCALE plugins, which would be simplified kubernetes containers
5. Use a VM to use a customised way of running docker
 

brando56894

Wizard
Joined
Feb 15, 2014
Messages
1,537
I am only a NAS user, so personally kubernetes is overkill for my single node NAS. I run it on clusters at work, but at home I have currently ansibled all my containers using podman, as they are static and need no scheduling. Kube would be great otherwise, but overkill for single home NAS nodes. For home systemd controlled podman pods are good choice.

I agree, Kube is overkill even for homelabbers like us. I set it up, and then just went back to Docker with Portainer. It's easy enough to setup a VM, install Docker and go about it that way, though.
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
If you want a consolidated product, one has to make choices.
One of which is the type of orchestrator, which they picked the branch standard solution for: Kubernetes.

Their design and the goal of adding gui to it (and integrating it into the plugin ecosystem) simply doesn't allow for multiple orchestrators. If you want something that both TrueNAS CORE and TrueNAS SCALE doesn't offer... maybe it's time to look into either virtualising it on TrueNAS or go for a different solution al together.

The thing is: Orchestrators are mostly a preference thing, not a technical need. Not every product is going to ship your personal preference for an orchestrator.
 

ikke

Contributor
Joined
Apr 22, 2012
Messages
124
I didn't want to complain about design choices, I understand one needs to do selections to create and maintain a product. K8s and podman are not comparable. Docker vs. Podman are, as they both are lower level tools that run containers, they are not cluster orchestrators. And even there it was not about selecting one or an other. Just having podman as installed package alongside docker would have been my request. Not alternative, just addition for those who prefer it over docker. Both do run the same containers.
 

rssfed23

Cadet
Joined
Sep 9, 2020
Messages
9
One of which is the type of orchestrator, which they picked the branch standard solution for: Kubernetes.

To be fair, the "branch standard" solution for the container runtime behind Kubernetes these days (and definitely looking to the future) is not Docker. It's CRI-O (of which podman can work concurrently with) or Containerd. Most of the main k8s distributions have already migrated to a CRI native container runtime and as Docker-shim is going to be removed from k8s in a future release, going with the right implementation now may save dev work in the future.

Under the hood SCALE is using k3s, which uses Docker with Containerd under the hood. The k3s project will potentially migrate away from Docker one day in line with other distributions(*), and already supports any CRI implementation today.
Personally, I can't wait for people to longer think or associate Docker with Kubernetes in anything but a historical sense.
Doesn't help the homelabbers much of course as that community often feels k8s is too much for their needs (personally I disagree k3s is almost perfect for that group!), but I wonder how much $ Ix makes from the non-enterprise customer base anyway.

Not that this should all matter much, as if someone is using SCALE as intended then they'll be using the helm based plugins and shouldn't care too much about the underlying container runtime. If someone is wanting to do significant manual CLI work/manual deployments then one could assume this may eventually cause issues and they'd be better off running it in a VM or elsewhere.

Also, @ikke, you can always install Podman manually and give it a go (it used to conflict with Docker but I've just checked and that seems to be resolved). I can understand why they wouldn't bundle it in given if they include it they'd have to support it for their Enterprise customers so the less moving parts you ship the easier that support becomes.

(* As a senior employee of SUSE, who recently announced their acquisition of Rancher/k3s this should not be taken as indicative of any product roadmap announcement but a comment on a general trend in the k8s upstream community)
 
Last edited:
Top