Container virtualization and the SCALE (RC-1) reality

[jeyare]

Tl;Dr : This is a long shot for those who are interested in learning more.
If you want to use TrueNAS SCALE for purposes such as Minecraft server, this article is not for you.
--------------------------------
I want to share my monthly experiences from TrueNAS SCALE on 22.02-RC.1-2 running in one of my VM nodes. For better segmentation of my thoughts:
My primary intent is to check the Scale platform to reuse its capabilities in SME/SMB segment in companies up to 100 employees. To understand me correctly, this includes small teams of 10 people for specific purposes (not a single holy grail) as a replacement for the existing Synology platform (my own and also for my customers).
My secondary intent: to check this platform as a replacement of my mixed home environment based on Synology and TrueNAS CORE (TNC). Each of them had its advantages because neither of them is perfect. But that's life. Nothing is perfect.


First – TrueNAS “honeypot” for people like me:

1. Pure Debian environment (5.10.70 kernel in RC-1) vs FreeBSD in both mentioned platforms (Syno or TNC).

2. Virtualization based on both Containers and VM in a single node or across multiple nodes (up to setup).

3. Enterprise Support. Yes, nothing is for free, and it is welcome for people like me to get in touch with another level of support than for SoHo. Of course, like many of you, I grew up in public communities, which I also actively use. Sometimes I feel that they know more than 2nd level NAS vendor support in those communities, especially if it is a downloaded package, which is part of the system but is not directly from the NAS vendor.

From any point of view, the SCALE product is more oriented to the SME / SMB segment than to SoHo. We will probably agree on that. This does not mean that people cannot use such a product at home. However, the primary goal is essential. And I hope that such a thing was also defined when creating the SCALE.

4. OpenZFS and Gluster technologies for hardening of the stable storage operation. Which seems to be a more suitable solution compared to LVM + BTRFS in Synology ecosystems.

5. Freedom to decide which HW to use for a particular environment based on the requirements of the environment itself and not the restrictions of the NAS system vendor. Over the years, it has been confirmed that freedom in this decision-making is more valuable than the comfort of one support centre for everything that still does not work well enough in an SME). And especially in today's world, where HW requirements change every year, and new available technologies are shifting at missile speed, tying up one HW vendor is a burden rather than an advantage. Once again - I do not describe a large enterprise environment like banks and the like.
This freedom has just been lost to the owners of Synology enterprise NASes, where due to the no longer supported Facebook Flaxcache (unchanged since 2014) and the already mentioned LVM + BTRFS connection, it has reached the point that Synology has severely limited the use of non-Synology branded (firmware tuned) HDD, SSD ( aka Toshiba), Synology RAM modules, Synology NIC and Synology PCIe card for NVMe Cache. One would say that it is suitable for the enterprise = this right choice should guarantee all. But, to be honest, it doesn't work. And right at all. Especially if you understand the details of disk behaviour and that not every PCIe card is the same.
This is exactly all I discovered in the SCALE. Well, that's about all the positive parts of this consideration. Only disappointment follows.

Second – the Containers “honeypot” was massively promoted by iX.

From the official SCALE web: TrueNAS SCALE provides simple access to the well-established Linux container ecosystem and makes application deployment easy. With support for KVM virtual machines, Kubernetes, and Docker containers, it’s easy to customize and add applications to suit a wide variety of needs.

Reality:

iX deployed Kubernetes as the primary containers orchestrator only. Docker (swarm, compose) does not officially support at all, and it is literally up to the user to adapt the system himself to use another orchestration (Docker-based). What will be valid until the next upgrade the SCALE (there is workarround already, described below).

The SCALE includes the ability to run Docker containers using Kubernetes (written in iX web, Last Modified 2021-04-02). Ok.
To be more consistent, iX used a simplified modified version of K8S in the form of K3S. And here comes the first problem. K3S itself is a platform initially defined as a Lightweight Kubernetes orchestrator designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances.
How it manifests itself in the SCALE practice:

1. It is not possible to use native commands created by Ranchers. E.g. try to use <kubectl>; you must search the Internet for why it doesn't recognize this command. Someone iX decided to use <k3s kubectl>. Yes, workaround is there (an allias) - but in RC-1??
2. It is not possible to use a network other than the "host network", which is a significant issue regarding the security or operation of segmented networks. Even for SoHo users who understand what this causes is a problem. Not to mention SME / SMB.
3. Kubernetes Administration / Monitoring. This is an entirely unmanaged part of the SCALE product. It does not support native elements from Ranchers such as Dashboard. So, for now, you can forget that you will orchestrate your containers through a complex environment such as Dashboard. You don't even have to try deployment for Dashboard. I tried it - it doesn't work. This was also confirmed by a TrueCharts representative here on the forum. This is a failure for SMEs / SMBs, even for SoHo power users.
4. SCALE APPs. As I learned here on the forum and not on the official iX website, this is a "glue" created by TrueChart (not part of iX) based on the simple deployment of freely available containers from the Docker hub environment, the purpose of which is "touch the button and play ”. In translation, this is a K3S deployment of existing Docker containers to be installed even for a novice in the world of virtualization which does not understand how to deploy containers and does not want to waste time with it. But that's still the definition of a SoHo user. Representative TrueCharts explained in this forum that it has added value for the user in case of update, rollback of the container. It is said that it is possible to request TrueChart support directly. What support? Binding container to K3S? In such setup of K3S within the SCALE? Are you serious? It has nothing to do with SME / SMB. To be sure, the SCALE is in RC-1 stage and not in Beta.
5. Heavy security issue with the SCALE APPs. For the reasons described in point 4, K3S deployments are set up so that only the user selects the appropriate APP from the catalogue, and all predefined settings have already been made for him in TrueChart. To make you understand - including setting up root / psw, usr / psw to a database which is a part of full-stack containers under single "APP name". This is an unacceptable and utterly wrong attitude. Even for the SoHo segment, it's across the line. This reality has also been confirmed by the trueCharts representative on this forum here.
6. There is no freedom to choose containers. You are strictly dependent on what SCALE APPS, TrueChart source, provides. An example - try to install a monitoring stack defined by Telegraf+InfluxDB+Grafana. First, you can't find the Telegraf or Grafana in the SCALE APP library. Some of you who know what relevant to chose InfluxDB v 1.8 vs 2.0 will understand this point. When you do not want to use the GUI - part of the SCALE APPs, you should turn it off. It's unworthy of the other quality features that the SCALE contains. Unfortunately, this is another example of how a good idea in the wrong hands ends tragically.

The SCALE APPs catalog is available here:
https://truecharts.org/apps/app-requests/
Compare that to what's available on the Docker Hub.

Unless you're tired of reading, we're reaching our goal of using the Docker regularly and more securely through the management and monitoring tools, as we do in 2021.
First, you need docker daemon to be active, which is part of SCALE( but only passively). The reasons have been described above. Then, the following procedure will help you:
https://gist.github.com/Jip-Hop/af3b7a770dd483b07ac093c3b205323f

Secondly, you need to create secure TLS docker node in SCALE for Portainer which is able to manage directly Docker containers (swarm, compose) and Kubernetes:
https://lemariva.com/blog/2019/12/portainer-managing-docker-engine-remotely
Of course, there are a few opponents of this method, so I'm happy to see their suggestions for comprehensive container lifecycle management in SME/SMB specially in this stage of the SCALE.

Done.
And this could have been done on the iX side a long time ago. Easy choice for the user:
- Unusable ecosystem in the form of K3S SCALE APPS from TrueCharts
- Or something that really works, including setting up your own root pswds, network setup / ENV options, Nginx Reverse proxy, ...

I still need to play with the docker socket to make everything work 100%, including access to the container console from Portainer. I have already run my first containers on TrueNAS SCALE. However, as I wrote in the introduction, I will watch the SCALE heading because the RC-1 is dedicated to a storage target only rather than the central host for container virtualization. It is still a long way from this goal, if at all. So what is the missed potential of this project. But I'm not the iX shareholder, so I can't comment otherwise.

Cheers.

 

[truecharts]

We are just a community catalog, with hard working volunteers. Trying to work with the tools iX-Systems gives us. It's totally uncalled for to attack us for "mistakes" (or choices) made by iX-Systems, while we are actually trying to compensate for the limitations on the platform, in the cases iX does not. We are just an Apps catalog. Aka: A collection of specially modified Helm Charts.

To thoroughly write a review like this, one needs to, at the very least, understand what Helm Charts are and how widely they are used. If so, it would quickly be clear why things like a comparison between TrueCharts and Dockerhub, are unwarranted and quite passive-agressive.

A good comparison for TrueCharts and other, industry-standard, solutions would be comparing TrueCharts to `k8s-at-home` or `Bitnami`. In that regard we are already one of the biggest central Helm-Chart repositories out there.

We will refreign to comment on the out-of-context reinterpreted quotes of ours in this article.
OP is still invited to drop on the discord to discuss his feedback about TrueCharts and get some explaination about what it is we actually do at TrueCharts.

---
TrueCharts Core Team
K.S.
N.S.

 

[jeyare]

Don’t take it personally.

Without exactly defined source of the containers you are unclear source for every healthy person in this world. But I understand that Pareto principle is anywhere. People don’t care about it and “touch the button” is a magic. Reason why we have increased amounts of ransomware and similar illnesses.

Just a recommendation- put there - in your app “details” a source linked. Or prepare a review of the containers for a vulnerability. It will be helpful for your time investment into such activities.

Containers are really interesting source of interest from dark net. Because people are crazy. Don’t help them to use your channel to a new spread.
I have long time research that just 3M of NASes from single vendor are freely exposed to internet with http or ssh ports. It just single vendor from 12 in SoHo. Just imagine how people use their data for mining or botnets spreading.

Last point. As was written no one from NAS newbies will install such “demanding” product as is the Scale.
They will purchase simple NAS from Asustore, Qnap, Buffallo or WD, … Touch the On and play. Then who is your target? Who will use your “simple” touch the button? Someone who can’t doit in TrueNas Core with Ubuntu VM and Docker daemon and Portainer with more freedom? C’mon.
I think it’s a time to evaluate your approach.

When you understand my point, then I’m glad.
I can help you to understand it from a better perspective. Ping me PM (when there is this option).

 

[Patrick M. Hausen]

I think it’s a time to evaluate your approach.

You should address that at iXsystems. The Truecharts project just tries to add value to the ecosystem as defined by iX. And they have no influence on the architecture whatsoever. You are beating not a dead but definitely the wrong horse.

Nobody on this forum has a say in the issues you raised. Which some I even find valid. I also would have preferred a simple UI for docker-compose, port forwarding, and be done with it.

If you want to reach out to iX, please open an issue on JIRA.

 

[truecharts]

Don’t take it personally.

Without exactly defined source of the containers you are unclear source for every healthy person in this world. But I understand that Pareto principle is anywhere. People don’t care about it and “touch the button” is a magic. Reason why we have increased amounts of ransomware and similar illnesses.

Just a recommendation- put there - in your app “details” a source linked. Or prepare a review of the containers for a vulnerability. It will be helpful for your time investment into such activities.

We do take it personally, because you are simply spreading completely false information about our project.
Which you are now doing yet again.

As stated earlier, we take security very serieusly and just gave a complete rundown of our security approach elsewhere on the forum as well. Said explaination is also on the to-do list to be added to the documentation as well.

Security Hardening is actually one of the features we at TrueCharts especially added, in comparison to the Official Apps by iX Systems. All containers are scanned and results are public, as is usual for a Helm Repository.

Your attacks are simply ridiculously misplaced.
To be frank: This looks like trolling.

 

[jeyare]

@truech
well,
maybe I’m alone who is looking for the source of installed “data” to my horses.
Yes, trolling is also answer for a described critique.

All containers are scanned and results are public, as is usual for a Helm Repository.

where exactly I can find the Security scan results:
- in your Scale APP list (Scale GUI)?
or
- in your Github?

One of the example from your Github:
/apps/charts/stable/airsonic/

there is just a link to Airsonic Github (this isn’t author of the container)
and link to the Docker hub and docker container from Linux Server (possible author of the container, but not defined)
and link to k8s, but the link is invalid (404). I will tell you why:
because this Airsonic is not maintained more than 2y (source: your iwn link to the Airsonic Github). Same is written also in Linuxserver Github.
What is a proof of your quality check.

Maybe i’m not a pleasant guy. But I’m not a sheep.
I don’t need continue in such discussion when I providing proof and you just empty wording. Done

 

[morganL]

1. SCALE APPs. As I learned here on the forum and not on the official iX website, this is a "glue" created by TrueChart (not part of iX) based on the simple deployment of freely available containers from the Docker hub environment, the purpose of which is "touch the button and play ”. In translation, this is a K3S deployment of existing Docker containers to be installed even for a novice in the world of virtualization which does not understand how to deploy containers and does not want to waste time with it. But that's still the definition of a SoHo user. Representative TrueCharts explained in this forum that it has added value for the user in case of update, rollback of the container. It is said that it is possible to request TrueChart support directly. What support? Binding container to K3S? In such setup of K3S within the SCALE? Are you serious? It has nothing to do with SME / SMB. To be sure, the SCALE is in RC-1 stage and not in Beta.
2. There is no freedom to choose containers. You are strictly dependent on what SCALE APPS, TrueChart source, provides. An example - try to install a monitoring stack defined by Telegraf+InfluxDB+Grafana. First, you can't find the Telegraf or Grafana in the SCALE APP library. Some of you who know what relevant to chose InfluxDB v 1.8 vs 2.0 will understand this point. When you do not want to use the GUI - part of the SCALE APPs, you should turn it off. It's unworthy of the other quality features that the SCALE contains. Unfortunately, this is another example of how a good idea in the wrong hands ends tragically.

The SCALE APPs catalog is available here:
https://truecharts.org/apps/app-requests/
Compare that to what's available on the Docker Hub.

Hi Jayere,

Thanks for taking the time for your extended review. I'll probably work with engineering and respond in a few different posts.

However, I think there's a fundamental misunderstanding of the Apps options. Apps are run on K3s... they are helm charts that run containerized applications. The infrastructure for these Apps is completely independent of TrueCharts, so iXsystems takes full responsibility for the Apps framework.

Apps provide four ways of getting an App set-up for a user:

1) There is a small set of prebuilt official Apps. These are Apps that iXsystems has taken the time to verify. It helps us test the system and ensures that initial install is easier and can be verified.

2) Apps provides a mechanism for deploying any Docker container .. eg from Docker Hub. This is done by mapping the Docker container into a helm chart that is set-up for TrueNAS.

3) The Apps allows community Catalogs to be included. TrueCharts is the largest and most active of these catalogs. This catalog also provides mechanisms for managing reverse proxies, load-balancing and wireguard VPNs. It's very cool and useful.

4) The Apps allows anyone to build their own helm chart for their specific apps. Any customization can be done this way. There is full freedom to choose containers. Its reasonable to say that its not extremely easy.... but there is full flexibility.

It should be noted that the goal of TrueNAS SCALE is the integration of Kubernetes, OpenZFS, Gluster, KVM, NFS, SMB, minio etc.... Integration makes installation and updates much easier, enables community/Enterprise support, and provides a single UI. However, it does also mean that there is a common middleware to manage the system configuration and we cannot easily support direct interfaces to each component. The system has to be supportable and relatively easy to use.

Thanks for reading... I hope this clarifies the current situation. Going forward, we plan to keep on increasing ease of use and are open to new ideas. Hope you can contribute.

 

[jeyare]

Nobody on this forum has a say in the issues you raised

maybe you need read more, in this thread you will find all the 6 findings:

https://www.truenas.com/community/threads/docker-within-scale-questions.93772/page-2#post-670347

or here:
https://www.truenas.com/community/threads/network-settings-for-apps.96888/

I can continue with others links, but it is a time waste. I’m looking for a solution.

 

[truecharts]

On community request, we do not deny depricated applications. That means not all applications inside the containers are secure (nor are all containers, obviously) , we never ever said they where and we always where super clear about the fact people shouldn't blindly trust us anyway.

The documentation is still lacking and that sad fact is confirmed by us daily, as we are working on it.
We are however on artifact hub, like any self respecting helm repository, and run multiple CI (security) tools to ensure good development practices.

While we tried to explain ourselves a bit here towards these outrageous accusations, we only owe explainations towards our own community.
So we will leave it at this.

TLDR:
Don't want it, Don't use it.

 

[jeyare]

@morganL
thx for the clarification.
Maybe it is another input for a better iX communication about the APPs in the official SCALE web, because this statement is missing there, which creates the following problems (also here).

And also change the main Scale picture, because it’s unclear re orchestration (supported) follow discovered situation in the RC-1:

 

[morganL]

Agreed the picture needs some revamp.... its a little old and is a marchitecture not a technical architecture.
It is intended to represent that Linux containers can run either as Docker containers or as Kubernetes pods which is true.
However, technical reality is that its Docker containers or Kubernetes pods run on a k3s base.
Clustering is part of the 2022 plan, but is based on the gluster (scale-out zfs) clustering that is now included.
TrueCharts and catalogs have simplified addition of more complex Apps.

 

[Kris Moore]

Just tossing my own 2C on this. We're happy to discuss our reasons for making specific decisions on how containers are implemented in SCALE today. It's not designed to be a "docker native" solution akin to just running docker directly on your own Linux box. Instead we're trying to build in features and functionality that make sense to us for the long-term support of containerized workloads running directly on storage. That is why K3's (K8's) was chosen for the bigger picture.

One thing we are aware of is that there are a myriad of configuration options potentially available to us to expose to the end-user. Features of docker sure, but also more K3's features, private networking, additional host passthrough options, etc. Not all of them we have had the cycles to build out on Angelfish, however we are already formalizing our roadmap for the 2nd release of SCALE (Bluefin) for later this next year. If you have specific knobs or features you think would be super beneficial to have exposed via our UI/API/CLI, please do open a ticket on https://jira.ixsystems.com and we're happy to consider it. SCALE is just at the beginning of its life, we're expecting to see a LOT of features continue to get added during the next few release cycles.

 

[jeyare]

@Kris Moore and @morganL
I can imagine what you had to do in iX to get here. As I wrote, from the point of view of the Storage segment, I am satisfied (the first large part of my review). I understand that the container orchestration is still in the neonatal stage of life. I know this from my own experience. I'll try to write you something there (Jira). There is also the security risk need to be mitigated (be clear with the sources of the charts and testing results).
I also understand that any feedback is a more important essence of progress. False patriotism has nothing to do with a project like the Scale. Of course, not everyone is ready to tolerate feedbacks. I appreciate your efforts to correct the debate. Nice weekend!

-------------------
This is the first Jira draft for your UX designers:

tested ScaleAPPs behaviour
Installed SQLitebrowser
Run the web
then Delete the APP

Experience:
- time consumed with the setup in the Scale APPs GUI is abnormal
- Portainer deployment with docker-compose is 10x quicker in the same host and much clearer for admin - from the same source of the container (DockerHub and linuxserver/sqlitebrowser)
- when I do not take into account the time of setting in the Scale APPs GUI, then the GUI isn't for container newbies (they will lose a way in the second click).

Who is the target group for the usage of the APPs GUI:
- experienced and patient k3s admins?
- experienced and patient docker swarm admins?
- experienced and patient docker admins?
- inexperienced container users?
The experienced group will be "depressed" from the GUI and the inexperienced will lose a way, how to do it.
This answer will help you to recreate the GUI.

Btw, you need better "clean service" for unused images. No one can recognize them in your Scale APP GUI.
Thanks to Portainer it is done clearly. Take it as an inspiration.

 

[Kris Moore]

Thanks for the feedback! I'll point some of my devs at this thread, but please do open those Jira tickets as well, since that's predominantly how we consume feedback :)

 

[HITMAN]

Btw, you need better "clean service" for unused images. No one can recognize them in your Scale APP GUI.
Thanks to Portainer it is done clearly. Take it as an inspiration.

At the moment you can type in search name and it will show you the images.

 

[NetCobra]

I agree to @jeyare

I started try TNS this week, I'm satisfied with other functions, but I have to say I'm confused by the App module in TNS.

I'm not a expert on docker/k8s, I just have some experience to use docker/docker compose/portainer to deploy applications on my home server (A PC installed with Debian Linux), and I learned something about k8s/helm as a beginner.

Now I found everything I learned is useless on TNS, I cannot use docker compose nor helm charts, I cannot connect to the k8s cluster on TNS, the only way to deploy an application is to follow the UI (So slow!), input something and click next and repeat, finally waiting and pray it can work (Some app failed to be deployed on my NAS, and I totally have no idea how to debug it).

And only application in catalog can be installed, if an application is not in the catalog, I have to ask and wait someone to add it into a catalog; and all my existing docker compose files and helm charts can not be used.

Maybe all these problem is because I'm stupid and not experienced enough on using TNS, so maybe I'm not a target user for it?

Could you give the user as me a chance to use docker/docker swarm/portainer?

Thanks!

 

[Patrick M. Hausen]

@NetCobra Run TrueNAS CORE if you want TrueNAS for other reasons (stability, ZFS, snapshots, replications ...), deploy a Debian VM, run docker/whatever inside that VM.

I am also experimenting and watching where this will lead, but for production - even if private/SOHO - use, SCALE is not yet ready in my opinion. OTOH iX never claimed it was. This is a public beta of an entirely new product.

 

[jeyare]

@Patrick M. Hausen
this thread is not about how to run containers on diff platform as the TrueNas Core. But understand your point.
This thread is about current stage of the containers operation/orchestration in RC-1 of the Scale and how to do it better to be useful for the target segment of the Scale product. And how to clearly communicate current stage to prospects who expected something based on reading from official iX web. It will be helpful for all the parties.


@NetCobra
my guide- how to run Docker swarm (mentioned above and works) in the Scale was targeted for just testing purpose of the Scale abilities. For a first touch and comparison with “comparable” solutions. You need setup Iptables chain and everything works as expected, include setup of bridge network, load balancing, include automated portainer agent services and ofc useful admin dashboard for all the operated containers (running, stopped, unhealthy) or for an excellent management of images or volumes. You can orchestrate both platform Docker swarm and k3s from single point = Portainer. Tested, works. From the Portainer you can setup Helm charts repository directly (Bitnami, …) for more flexible deployment = what is more clear source as existing solution in the Apps. For me it is enough for the tests. Need to wait for the next Scale stages. I think gents from iX should to try it for an inspiration. Ready to help them, to find more useful final stage.

 

[Janus0006]

This thread is very interesting.

I’m personally a homelab user of TNS, working in IT since a while using TNS since the alpha version. Also, I can make the difference between my homelab needs and the enterprise needs.

When I switched to TNS, the project seems to be interesting because I saw the potential of making a TNS server a big beast; allowing us to have Storage, docker/apps and mini-virtualisation environment (not a fan of KVM). With the possibility to run it in a kind of cluster (a potential best). I worked with Nutanix for this kind of features.

For the part of virtualisation and docker/apps, I was expecting something more customisable. I’m not an expert in dockers/Kubernetes, etc but I’m able to use dockers from docker hub, customise them and make them running. Using the huge library and potential from the hub. But, by being forced to use the k3s/GUI/pre-build apps from a specific library, I’m a quite disappointed. (and not being able to assign specific address to you apps… ) I completely understand you need to integrate a kind of k8s to let application works in a TNS cluster environnent. But if you cannot customize the apps, why uses Scale instead of Core with jail apps? If I must launch a Debian VM to run docker-compose, what is the advantage?

I know, my perspective is mostly oriented on a single node TNS, but it seems IX forget this kind of users. Even in enterprise, not all companies need the potential of a cluster. And if they need this potential, sorry, I think they will use other enterprises solutions than TNS.

I will continue to follow the thread to view many opinion on the subject, very interesting opinions here…

 

[jeyare]

so first Jira ticket is issued
Testing scenario with Sqlitebrowser at ScaleAPPs
Used pragmatic point of view include several recommendations, how to do it better.
Several serious and security issues found + UX anomalies.
Thanks to Portainer connected to the K3S operated in the TrueNAS I was able to do more than by current Scale APPs GUI.

Jus one pearl found in the test:
I was able to define a port number for the installed APP (for exposing purpose), which was already used by another service in the SCALE (I deliberately did it that way). Even worse, it was accepted by the APP (engine) and installed without any errors showing. OFC the installed APP was not available by browser, because of written problem. It can make a heavy mess inside.


@HITMAN
regarding the images, your point is not useful:

At the moment you can type in search name and it will show you the images.

for used scenario I wasn't able to find "named" SQLitebrowser image in the APPs image list, just hashed. What is not right behaviour of the images management. Yes the Search will show the image, w/o next details, what is inappropriate:

Just compare the same situation with Portainer, connected to the same K3S environment from another host:
- you can see exactly UNUSED image/s
- you can use filters or search
- you can click to the image and read all details before deleting - to be sure, that right image was choosen:

This is expected behaviour.

Btw: I was able to delete another image from the running pod in the Scale APPs GUI. Without any warnings. It is dangerous.

As was written, this is just a part of my heavy report sent to the Jira. Ready for a direct discussion with iX by Zoom or similar media. It can be a more efficient way to get more useful points. Ready to help them.

@Janus0006 - I created this thread because topic of the containers orchestration was insufficiently described and scattered all over the forum.
And it's really not enough for me if someone says that some repositories are safe and suitable and ... and orchestration is on k3s and docker swarm is not supported, Portainer is not supported, ...... and tests are starting to prove the real situation, different from that described. Unlike haters, I also offer recommendations on how to improve it. And I have no problem praising a well-done job.