Does Scale still make sense?

[Samuel Tai]

The rest of the storage ecosystem isn't standing still. Today, I learned OpenMediaVault supports ZFS on Linux via plugins and Docker containers. Scale-out storage has been available from cloud providers for several years now. Does Scale still make sense, business-wise? Or will it be defunct on release?

 

[shadofall]

As a omv zfs user I can tell you I'm not a fan. The user experience is kinda lousy. And since zfs is a plugin it's not really integrated in to the system as a whole. The primary maintainer doesn't build with zfs support in mind.

As for scale out storage in the cloud. That gets complicated since it it involves $$$ and weather or not the cloud will save them that $$$ or not

 

[Patrick M. Hausen]

I am looking forward to a platform on which I can deploy k8s or k3s workloads on my own infrastructure without much hassle. If that already exists in some other product, I am always curious

 

[Samuel Tai]

I am looking forward to a platform on which I can deploy k8s or k3s workloads on my own infrastructure without much hassle. If that already exists in some other product, I am always curious

How about PhotonOS from vmWare? It's a minimal Linux distro that's optimized for Docker, and can run Kubernetes too.

 

[morganL]

The questions is posed in a way where it almost answers itself. There are many products that do subsets of what SCALE does. But there are no single Open Source projects that cover the spectrum of Scale-out, Converged, Active-Active, Linux Containers. Unified Storage.

In the end, SCALE has to deliver more value through Functionality, Quality, Price (hard to beat Open Source), Ease-of-use and Technical support.

It's a journey to get to the quality we all want for production use, but the TrueNAS CORE base is providing a lot of advantages.

 

[derWalter]

The rest of the storage ecosystem isn't standing still. Today, I learned OpenMediaVault supports ZFS on Linux via plugins and Docker containers. Scale-out storage has been available from cloud providers for several years now. Does Scale still make sense, business-wise? Or will it be defunct on release?

you are quite right there, but "isn't standing still" does also mean the cycle of life, projects are dying. Look at the code frequency of OpenMediaVault and Rockstor on gitHub. Both are one-man-shows and while the initial autor of Rockstor quit the show, the OMV autor recently said, he is working on it when he has time and desire to do so. Compared to this Scale is developed by a real company which sets goals and if some developer drops out for what ever life events and reasons, they hire an other one and continue.

That alone makes it outstanding in the midst of competition and therefore it will be the an outstanding product nevertheless, at least IMHO

 

[ornias]

In the end, SCALE has to deliver more value through Functionality, Quality, Price (hard to beat Open Source), Ease-of-use and Technical support.

Precisely:

SCALE has quite the niche, It competes with Proxmox and OMV but:
- Supports just one (okey, insert sidenotes about ext4) file system and does so very well.
- Supports K8S out of the box, with minimal setup required
- Is expected to have a vastly more streamlined way of setting up clustering
- Has actuall enterprise support (instead of Proxmox's signature "my way or the highway" support)
- Doesn't use the community as a test-bed much, as versions are the same for everyone you are free to wait or be on the bleeding edge (instead of proxmox)
- Has an actual company behind it with a solid support track-record (which actually already gives insane levels of support for free users)

The thing with SCALE-out storage on cloud is very simple:
It's in the cloud and not everyone wants to (or can legally!) be in the cloud.

I think in comparison to OMV SCALE already offers a lot more.

The TRUE (pun intended) competition would be Proxmox in this example:
- Exposed VM features in the UI need expanding, Proxmox currently has much more things exposed
- LXC containers are really needed to have feature parity with TrueNAS CORE and Proxmox when it comes to persistant container technology


SCALE has the bones of surpassing even CORE in popularity, but just needs a lot of polish. However if it get said polish, it might even one day replace it completely or CORE gets turned into more of a niche.

 

[Yorick]

Not convinced about the need for lxc. k3s backed by containerd or cri-o does a lot, and if a more “full system” virtualization is desired, including the ability to SSH in, then there’s KVM.

Adding another runtime also means supporting it.

Do you have some specific use cases where lxc shines over k3s?

 

[ornias]

Do you have some specific use cases where lxc shines over k3s?

Every usecase that needs full persistance and doesn't warrant a complete VM.
(besides the fact that LXC is faster/easier to setup than a full VM for the average joe)

 

[Patrick M. Hausen]

How about PhotonOS from vmWare? It's a minimal Linux distro that's optimized for Docker, and can run Kubernetes too.

Doesn't come with a UI as far as I got from the docs. I need/want a solution for this Linux/container universe that is just as turnkey as TrueNAS Core is for jails and VMs.

 

[Patrick M. Hausen]

Every usecase that needs full persistance and doesn't warrant a complete VM.

In short: everything you would run in a jail if this was FreeBSD. We are actively investigating LXC as a complement to our jail based hosting to run e.g. SOGo, which is a pain to get to work on FreeBSD but still a great solution.

 

[Samuel Tai]

there are no single Open Source projects that cover the spectrum of Scale-out, Converged, Active-Active, Linux Containers. Unified Storage.

I'm concerned because many of the posters here are enthusiastically waiting for the container capability, but the scale-out storage capabilities seem to me the core functionality Scale will need to deliver. I don't know if sufficient progress is being made on the GlusterFS + ZFS integration, especially the active-active capability, and how to perform snapshots/rollbacks consistently across a cluster. There are many nasty race conditions that may end up unresolvable. This focus on containerization is starting to feel like requirements creep to me.

 

[ornias]

I'm concerned because many of the posters here are enthusiastically waiting for the container capability, but the scale-out storage capabilities seem to me the core functionality Scale will need to deliver. I don't know if sufficient progress is being made on the GlusterFS + ZFS integration, especially the active-active capability, and how to perform snapshots/rollbacks consistently across a cluster. There are many nasty race conditions that may end up unresolvable. This focus on containerization is starting to feel like requirements creep to me.

The container system is like 80-90% done already, which actually caused the delays in the scale-out storage capabilities (that would first be done in December). But I agree the silence on the clustering is worrysome

 

[morganL]

The container system is like 80-90% done already, which actually caused the delays in the scale-out storage capabilities (that would first be done in December). But I agree the silence on the clustering is worrysome

The building blocks of clustering are "emerging".. the APIs are in SCALE 21.02 and the 1st version (nightly) of TrueCommand 2.0 has the cluster management tools. That said, the clustering work is the major activity for next 2 quarters. However, clustering requires that each single node is stable and functional, so needed to be the last effort.

I'd disagree with @Samuel Tai on whether containers are a distraction. Linux containers and Kubernetes are the commonly agreed on method for scaleable applications. It's those applications that then drive demand for scaleable storage. We're just taking an alternative view which is that scaleable can be a single small node (e.g Mini) or a larger number of very big nodes. We would like to build a broad community that wants to help create turnkey infrastructure that can then be easily updated and maintained. Often that community has specific applications in mind.

 

[ornias]

The building blocks of clustering are "emerging".. the APIs are in SCALE 21.02 and the 1st version (nightly) of TrueCommand 2.0 has the cluster management tools. That said, the clustering work is the major activity for next 2 quarters. However, clustering requires that each single node is stable and functional, so needed to be the last effort.

Precisely, one of the primary challenges with scaling out, is to have it seemlessly work with the applications... Not just TrueNAS itself but also the clustered Apps and Helm Charts. Once those are solid (which they 90% are when it comes to the API), that gives a good solid test-bed for the actual storage implementation :)

 

[beagle]

- Has an actual company behind it with a solid support track-record (which actually already gives insane levels of support for free users)

That's a key differential that often gets overlooked.

- Exposed VM features in the UI need expanding, Proxmox currently has much more things exposed
- LXC containers are really needed to have feature parity with TrueNAS CORE and Proxmox when it comes to persistant container technology

^^^ I'm betting on that :)

 

[Arwen]

One thing that has not been mentioned is that SCALE, (& CORE), have alternate boot environments via ZFS snapshots & clones. This allows backing out an update. And making a backup of a configuration before modification, (which would just require a reboot to activate the old BE.)

Does OMV have that?
How about PhotonOS?

It does appear that Promox can have boot environments, though it's unclear if its as solid as TrueNAS Core.

 

[skittlebrau]

In my experience, it's really easy to 'break' OMV and get stuck in a 'apply changes' configuration loop if you work too quickly.

Being able to export a configuration and restore it like any decent 'appliance' is golden and is one of the reasons I use TrueNAS.

 

[headwhacker]

In my experience, it's really easy to 'break' OMV and get stuck in a 'apply changes' configuration loop if you work too quickly.

Being able to export a configuration and restore it like any decent 'appliance' is golden and is one of the reasons I use TrueNAS.

When I started out to build my own home NAS to replace my Asustor AS5004T, I looked at different options and at the time OMV appeared to be better suited in my use case (light-weight, native-linux which can run KVM and docker natively, etc). Then while trying it out, I experienced so many annoying things particularly with the updates, especially with OMV extras and ZFS plug-in installed. I mean I had to redo the install from scratch many times, and for some reason in my testing ZFS only works if I install the proxmox kernel.

When testing TN12, it was immediately obvious it is more stable. Some minor issues I encountered, but it was fully documented and fix was already in the pipeline. In short I went for TN12. Since U2, no issues and completely rock solid. Only thing I wish is that it runs on Linux so I can run native KVM/docker. Hence, I'm here watching the development of SCALE.

 

[ornias]

@headwhacker Be aware it doesn't run "native" docker, it runs k8s and the default docker stack is also modified in such ways that it won't work without docker-compose networking without changes. ;-)