TrueCommand docker cluster creation issues

[buchwald]

Hello,

I am running truecommand system version 2.2.2, middleware version 2.2.2-20221018 and I am having continuing issues trying to configure a cluster. Below is a breakdown of what I am seeing when I try and get this setup.

- When I create a cluster via the "create a Cluster" option sometimes I get the selection for the Network interface option but not all the interfaces are present. Other times I get an Error:2 and can do nothing. Then other times during setup I get a NAS API Error. "[EFAULT] Error: Request timed out". When I try to delete a cluster that also fails with the same API error. I can delete a cluster if I go to each node and do "midclt call service.update 26 '{"enable": false}'" and reboot the nodes. After that I am able to delete the cluster and try to create a new one. But the network interfaces are again never correct, or I get the Error:2. So I end up back to where I was. I have done this a number of times and can never continue. The only way I can continue is if I choose "Skip this step" option.

- The systems that I am trying to cluster are five SuperMicro storage units that have two network interfaces. one in a public domain and the other private. All the systems have truenas scale-22.12.0 and both interfaces are connected and seen in the trucommand console view. If you like I can get you some screen shots if that helps your team with troubleshooting.

Thanks, Ted

 

[Samuel Tai]

TrueCommand doesn't create Docker clusters, but glusterfs clusters. Note, glusterfs clusters are still heavily under development, and aren't ready for production workloads yet.

/solutions/integrations/smbclustering/

 

[buchwald]

Thank you for the info. Is there any workaround to cluster creation using truenas scale?

 

[Samuel Tai]

Thank you for the info. Is there any workaround to cluster creation using truenas scale?

Scale is intended for Kubernetes clustering, not Docker clustering. I've seen mention on this board future builds of Scale will remove the Docker subsystem. If you need Docker clustering, your best bet is to create a VM on Scale, and run Docker swarm inside that.

 

[buchwald]

I was thinking more on the line of gluster storage clustering. Not so much docker, or Kubernetes. We are wanting to use gluster over CEPH as CEPH requires a lot of resources and is on the slow side.

 

[Samuel Tai]

You may want to look into Lustre instead of gluster for storage clustering. Lustre is very mature, and is used for exabyte-scale clustering of very high throughput data streams. It's the file system in use at the LHC to store the huge data streams in flight. Gluster is still small potatoes amateur hour, and hasn't been thoroughly tested for reliability.

 

[buchwald]

A while ago luster was one of the 1st I wanter to go with. My supervisor the time wanted to go with CEPH (sso I configured a few CEPH clusters) and or gluster. So gluster was his choice. Thanks for re-sparking my interest in luster clustering again...

 

[ckibet]

You may want to look into Lustre instead of gluster for storage clustering. Lustre is very mature, and is used for exabyte-scale clustering of very high throughput data streams. It's the file system in use at the LHC to store the huge data streams in flight. Gluster is still small potatoes amateur hour, and hasn't been thoroughly tested for reliability.

Great conversation. We are keen to use Lustre on TrueNAS enterprise for storage clustering for the HPC but haven't found any documentation on how to do that.Any pointers?