Dragonfish, clustering, apps and my sense of disappointment

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
OK, so I feel the need to speak up a bit on a topic that I've had on my mind since the summer of 2000.

I had a bit of time on my hands and was looking to get involved in testing/helping during the first real development window for Scale (angelfish).

At that time, I was absolutely bursting with enthusiasm about what SCALE was promising... it was exactly what I wanted (and probably more) in order to meet my simple dream (I think I even posted something like it somewhere in a thread with iX product managers, but can't find it easily now):

Give me a way to store my app config and data on ZFS securely, but allow me to have more than one node to run those apps on, with seamless migration from one node to another.

It sounded simple enough, although was a big gap I had found in the way that Rancher 2.0 did things with Kubernetes... but I saw an opening there with "host local paths" (or something like that), meaning each node would look for config in the same (local) path, so all that was needed was a bunch of nodes that kept a set of files available in the same path synced up for the apps to use and some kind of "app coordinator" (like Rancher) which would decide which node was best to run an app on.

What showed up was not that.

Gluster looked like a good idea and I persisted with that a bit, but never really got what I needed. Now it's gone altogether, never having really got airborne.

SCALE prevented running a more capable front-end like rancher to do the kubernetes clustering and replaced that with what was a very low-tech alternative which as far as I can tell still isn't going in the right direction 4 years later and has really put a lot of folks off with the catalog wars and the unfortunate events that unfolded with TrueCharts and their initial releases of changes requiring app re-install (and then some more).

One glimmer of hope I saw was the iX publishing of syncthing, but I remain skeptical without any promise of a GUI to manage apps to consume it in a sensible way.

I'm now considering options for how I might be able to get closer to this utopian dream using the systemd-nspawn container system, but have hit walls with running Kubernetes in those already.

For something that once had its own use-case, I think SCALE is trying to be CORE too hard and forgetting the original goals of being SCALE-out.

I look forward to what the future and dragonfish may bring, but I suspect I'll be waiting a long time to have my dream setup since I see nothing in the notes so far to indicate it is moving in that direction.

I don't want this to sound "rant-y", so I'll add that I'm really happy to have a company like iX looking after the product in a way that allows access to a wide audience for a wide variety of use cases. (but I still want my dream setup @Kris Moore :P ).
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
OK, so I feel the need to speak up a bit on a topic that I've had on my mind since the summer of 2000.

I had a bit of time on my hands and was looking to get involved in testing/helping during the first real development window for Scale (angelfish).

At that time, I was absolutely bursting with enthusiasm about what SCALE was promising... it was exactly what I wanted (and probably more) in order to meet my simple dream (I think I even posted something like it somewhere in a thread with iX product managers, but can't find it easily now):



It sounded simple enough, although was a big gap I had found in the way that Rancher 2.0 did things with Kubernetes... but I saw an opening there with "host local paths" (or something like that), meaning each node would look for config in the same (local) path, so all that was needed was a bunch of nodes that kept a set of files available in the same path synced up for the apps to use and some kind of "app coordinator" (like Rancher) which would decide which node was best to run an app on.

What showed up was not that.

Gluster looked like a good idea and I persisted with that a bit, but never really got what I needed. Now it's gone altogether, never having really got airborne.

SCALE prevented running a more capable front-end like rancher to do the kubernetes clustering and replaced that with what was a very low-tech alternative which as far as I can tell still isn't going in the right direction 4 years later and has really put a lot of folks off with the catalog wars and the unfortunate events that unfolded with TrueCharts and their initial releases of changes requiring app re-install (and then some more).

One glimmer of hope I saw was the iX publishing of syncthing, but I remain skeptical without any promise of a GUI to manage apps to consume it in a sensible way.

I'm now considering options for how I might be able to get closer to this utopian dream using the systemd-nspawn container system, but have hit walls with running Kubernetes in those already.

For something that once had its own use-case, I think SCALE is trying to be CORE too hard and forgetting the original goals of being SCALE-out.

I look forward to what the future and dragonfish may bring, but I suspect I'll be waiting a long time to have my dream setup since I see nothing in the notes so far to indicate it is moving in that direction.

I don't want this to sound "rant-y", so I'll add that I'm really happy to have a company like iX looking after the product in a way that allows access to a wide audience for a wide variety of use cases. (but I still want my dream setup @Kris Moore :P ).

We were also disappointed that IBM/RedHat aborted Gluster without a hand-off. Unfortunately, it was economically or technically feasible to complete the project which would have integrated Gluster with ZFS snapshots. We have to switch investments into other solutions.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
We were also disappointed that IBM/RedHat aborted Gluster without a hand-off. Unfortunately, it was economically or technically feasible to complete the project which would have integrated Gluster with ZFS snapshots. We have to switch investments into other solutions.
Thanks for the response and I totally get that. Gluster may not have really been the "droid I was looking for" anyway... not sure if syncthing is it either.

In any case, I guess I was hoping this thread may raise a bit of awareness about the goal and maybe enough people will speak up to say, "that's exactly what I want too", making it more likely to happen if we're all searching for the method together.

Personally, I hope the method will be provided by SCALE one day... a little concerned that I don't see it coming though.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
Gluster looked like a good idea and I persisted with that a bit, but never really got what I needed. Now it's gone altogether, never having really got airborne.
Yeah, we've been using Gluster for almost a decade now and things were finally mostly usable. So much so that I was considering moving our stuff to TrueNAS Scale to reduce the manual management workload.
 

CountBuggula

Cadet
Joined
Feb 6, 2024
Messages
5
I too was suckered in by the early promises of Scale, looking at it as a way to ditch VMware while more fully embracing my conversion to containers. I was disappointed when first one then another roadmap date went by with no clustered kubernetes or VM HA ability. Years later and now there's zero mention of those things in their literature or roadmaps - it seems like they've completely abandoned the scale part of Scale.
I'm not willing to continue using a solution with zero failover capability, but VMware is no longer going to be a possibility after Broadcom.

Luckily I happened across a relatively new project called Harvester. It's backed by the SUSE guys so I have at least some faith in their ability to execute on their stated goals. It's already quite capable and I've been impressed with the little bit I've had a chance to play with it so far - I have a single node running and am working on building a cluster to test my proof of concept.

Clustered storage, live VM migration, and clustered kubernetes, all from the same management interface and designed to coexist nicely. Might be just what you're looking for.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
I'd come across the Longhorn thing, but haven't had a chance to give it a try.
 
Top