SOLVED TrueNAS-SCALE-21.02-ALPHA.1 - Apps take hour to deploy

[maximelu]

Hello,
I am new to TRUENAS Scale but have good experience with TrueNAS Core and K8s in my professional life.

However I am not sure I understand some of the mechanics of the Apps in TRUENAS Scale.
If I deploy an application for the first time, it takes a while but it works. However if I have to restart the machine or adjust parameters I have to stop and restart the applications.
In this case, the system then makes dozens or even hundreds of snapshots and clones of DATASET to redeploy and it takes more than 4-8 hours. Finally I have unset the pool in order to be able to destroy -r on the DATASET Docker.

Although I use K8s in my professional life I cannot make the parallel between this deployment mechanic and classical mechanics. I suppose the system makes a snapshot + clone for each docker layer.
Also, I would like to know how to make the operations simpler:
- Did I miss a parameter?
- Can we deactivate the snapshot / clone operations of the binaries which seem useless to me?

Thanks for everything in advance :)

 

[ornias]

I am new to TRUENAS Scale but have good experience with TrueNAS Core and K8s in my professional life.

Be aware the k8s support is still pretty much WIP, so don't expect it to be the same stability as both of the options from your professional life ;-)

If I deploy an application for the first time, it takes a while but it works. However if I have to restart the machine or adjust parameters I have to stop and restart the applications.

Having to stop and restart after editing the App settings is something thats definately a bug you should report...
I've had it myself and it shouldn't happen. Didn't have the time to make a good bugreport with debug log though.

Having to manually restart apps after a reboot is not something I experienced (though i don't use official Apps, just TrueCharts), but is also worth a bug report.

In this case, the system then makes dozens or even hundreds of snapshots and clones of DATASET to redeploy and it takes more than 4-8 hours. Finally I have unset the pool in order to be able to destroy -r on the DATASET Docker.

I don't think this has anything to do with apps, as I don't think it snapshots anything before/after reboot or just after starting/stoping the App.

Although I use K8s in my professional life I cannot make the parallel between this deployment mechanic and classical mechanics. I suppose the system makes a snapshot + clone for each docker layer.
Also, I would like to know how to make the operations simpler:
- Did I miss a parameter?
- Can we deactivate the snapshot / clone operations of the binaries which seem useless to me?

I think you should start with the actual problem: this actual behavior which it shouldn't do. Before going about wanting all kinds of things changed that have nothing to do with it.
I can with close to 100% certainty tell you: No, IX is not going to remove the default ZFS docker-store because you have a bug with it.

 

[maximelu]

Thank you because it tells me where to look to understand what is abnormal. I think I'll start by reinstalling TrueNAS to start again on a healthy basis

I'm aware about the WIP status of the distribution and I'm really enthousiast about TrueNAS Scale.

If the restart bug append a new time I will take the time for fill a report.

I will also take the time to understand TrueNAS Scale in depth

 

[ornias]

Thank you because it tells me where to look to understand what is abnormal. I think I'll start by reinstalling TrueNAS to start again on a healthy basis

I'm aware about the WIP status of the distribution and I'm really enthousiast about TrueNAS Scale.

If the restart bug append a new time I will take the time for fill a report.

I will also take the time to understand TrueNAS Scale in depth

Yeah it really shouldn't take long or hang on creating datasets... Keep an eye on it!

I've personally done about 5000 or more App deployments by now and I've had about 1000 of those hang, about 900 of which where my own mistake (long live development and CI) :P

Those other 100 where mostly due to Apps not terminating correctly in k8s.

I did however need to hit "start" after just about every edit of an App.

 

[maximelu]

As announced I reinstalled.

I updated the UEFI of the motherboard too.

Finally i've added SSD pool for the K3s part. The behavior is really better.

Kubernetes need a big bunch of IO and if we use only mechanical disk, the deployement is slow, especially if the cache is not build.

I think my thread is solved now :)