Truecharts Nextcloud broke my system

[Robert Thomspon]

So, i tried installing truecharts version of nextcloud following their tutorial. i must have missed a setting or something as the app was created but got stuck at 0/3 deployed. when i navigated away from the applications page and came back, i was presented with a spinning circle. I ended up power cycling the TrueNAS machine (HUGE mistake). Now, it appears that none of my applications are launching, and all i have at the installed applications page is a spinning blue circle. Ive consulted Truecharts about this and they sent me to iX to look for help with you guys (all i REALLY wanted was a way to remove the nextcloud app that i just installed... but, there doesnt seem to be much CLI friendliness in the truecharts arena, making things really hard when you lose GUI)

Ive tried engaging with the system with docker commands (apparently a huge no no with truecharts. would be nice if they told you that.) Ive tried k3s commands as server (seems to crash), agent, kubectl, critctl (thislooked like it was going to work when i ran 'k3s critctl delete nextcloud' but it ended up timing out. also tried PID killing things, and force removing the folders that contain the pods. nothing has gotten my gui back up, but worse, the rest of the dockers (pods) arent loading either.

Truecharts is claiming this is an iXissue... so...

Anyway, attached are the info dumps Treucharts asked for. I am happy to include anything else that might help:


root@truenas[~]# service k3s status ● k3s.service - Lightweight Kubernetes Loaded: loaded (/lib/systemd/system/k3s.service; disabled; vendor preset: disabled) Active: active (running) since Wed 2022-02-09 16:56:42 EST; 1min 17s ago Docs: https://k3s.io Process: 2428415 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS) Process: 2428416 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS) Main PID: 2428417 (k3s-server) Tasks: 64 Memory: 561.9M CGroup: /system.slice/k3s.service └─2428417 /usr/local/bin/k3s server Feb 09 16:57:50 truenas.local k3s[2428417]: time="2022-02-09T16:57:50.384718589-05:00" level=info ms> Feb 09 16:57:51 truenas.local k3s[2428417]: time="2022-02-09T16:57:51.384846992-05:00" level=info ms> Feb 09 16:57:52 truenas.local k3s[2428417]: time="2022-02-09T16:57:52.384974985-05:00" level=info ms> Feb 09 16:57:53 truenas.local k3s[2428417]: time="2022-02-09T16:57:53.385104291-05:00" level=info ms> Feb 09 16:57:54 truenas.local k3s[2428417]: time="2022-02-09T16:57:54.385239122-05:00" level=info ms> Feb 09 16:57:55 truenas.local k3s[2428417]: time="2022-02-09T16:57:55.385372991-05:00" level=info ms> Feb 09 16:57:56 truenas.local k3s[2428417]: time="2022-02-09T16:57:56.385491352-05:00" level=info ms> Feb 09 16:57:57 truenas.local k3s[2428417]: time="2022-02-09T16:57:57.385616779-05:00" level=info ms> Feb 09 16:57:58 truenas.local k3s[2428417]: time="2022-02-09T16:57:58.385841424-05:00" level=info ms> Feb 09 16:57:59 truenas.local k3s[2428417]: time="2022-02-09T16:57:59.386007516-05:00" level=info ms> lines 1-22/22 (END)

 

[stavros-k]

Ive tried engaging with the system with docker commands (apparently a huge no no with truecharts. would be nice if they told you that.)

Sorry, but It's not a no no with TrueCharts. TrueCharts don't build the system :)
TrueNAS Scale does not official support docker, and interacting with docker commands might or might not break the kubernetes part.

 

[truecharts]

Ive consulted Truecharts about this and they sent me to iX to look for help with you guys (all i REALLY wanted was a way to remove the nextcloud app that i just installed... but, there doesnt seem to be much CLI friendliness in the truecharts arena, making things really hard when you lose GUI)

A short note on this:
Our support staff was not aware of any CLI command that would fix your issue, because the kubernetes CLI interface was also unresponsive.

It's not that we are CLI unfriendly, it's just outside the scope of our project to deal with the underlaying system (kubernetes) completely breaking down.

So in short we do want to help out, but we have no influence over the underlaying kubernetes installation. Installing Helm Charts (which is what Apps are "under the hood") should not nuke your Kubernetes installation. We understand how much it sucks, but we also cannot do much about it as we've only a few staffmembers who are versed enough in the iX Systems codebase to look into it.

 

[Robert Thomspon]

A short note on this:
Our support staff was not aware of any CLI command that would fix your issue, because the kubernetes CLI interface was also unresponsive.

It's not that we are CLI unfriendly, it's just outside the scope of our project to deal with the underlaying system (kubernetes) completely breaking down.

So in short we do want to help out, but we have no influence over the underlaying kubernetes installation. Installing Helm Charts (which is what Apps are "under the hood") should not nuke your Kubernetes installation. We understand how much it sucks, but we also cannot do much about it as we've only a few staffmembers who are versed enough in the iX Systems codebase to look into it.

Unsure if this was pure luck that it worked, but:

I reinstalled TrueNAS (assuming that that fixed the break in the nextcloud issue or whatever broke kubernetes), CLI removed the nextcloud application and reinstalled plex (not that plex was of any importance, its just the app i chose to install first), on the creation of Plex, all of my other applications came up and have since worked perfectly fine (seems the settings carried over as their directories were not destroyed. A Lot like docker will if youve ever used that).

I DO NOT RECOMMEND A SYSTEM REINSTALL TO REGAIN ACCESS TO YOUR APPS UNLESS ITS AN EMERGENCY!!!

As for the rest of the stuff said above (and not that this is really the place to complain that you didnt like the way someone talked about you in a ticket BUT): CLI is something that anyone out there doing this kind of thing REALLY needs to familiarize themselves with. rare are the instances in which small things happening can lead to the core and thus the CLI collapsing. And you NEED that CLI interface when your GUI fails, and a LOT of people using truecharts releases are going to have their GUIs fail at SOME point in time. I recognize that we are all dealing with "beta" builds, and as such, we expect things to break. What we DONT expect is that when using one teams applications and tutorials that end up breaking our systems is to be booted away and have the issue blamed on others. The others didnt write your app, the others didnt write your tutorial. So, when your app or your tutorial breaks someones system, a small part of that falls on you as the dev to, AT A MINIMUM, understand why. That was a shortcoming by truecharts on my failed app and subsequent request for support. Furthermore, your support staff (or, at least, 1 of them) seriously needs to learn some social skills. My ticket was closed without resolution, i was locked out of the ticket so i couldnt comment, and i couldnt reopen. I opened another ticket to state that the issue was resolved (and to bitch a little about some of the lack of social skills in their support staff), i then recieved a warning for doing so, commented on the discord app, got my comments deleted, another warning, and then when i called out the individual, i got a 24hr ban and another warning... as well as my comments deleted. I will say, there are some really good guys over at true charts... there are also a few that shouldnt be allowed to interact with other people.

Also note that the same people that are in your discord, probably come here and read material... so, while you can delete my comments there, its harder to do here.

Lastly, a tip to TrueCharts: Seriously, you shouldnt have some of your support staff engaging with the public. Im brash and crass and know some of my shit. Imaging what happens when this system isnt in a beta format and you have random people chiming in with all kinds of problems. whats the solution you are going to have for them? take it up with iX? too bad so sad? warn them? block them? scold them and close their tickets? You want this to be a robust and inclusive ecosphere, then you guys have a LOT more learning to do when it comes to the people side of all of this.

and on a final note: i get the ticket side of this for truecharts, but why not also have an open forum where the community members can help each other and be overseen by those infinitely more knowledge about the systems? Its the way ive seen MOST of these ecospheres exist (Unifi, Home Assistant, Tasmota, ESPHome, blissOS, readarr, docker, raspberry pi, homelab, even topics as wide as "linux"... why narrow yourselves to 1 or 2 people helping in a sea of (well, probably hundreds right now) thousands? it seems limiting and small, and honestly, a little power hungry/controlling.

 

[jgreco]

Moderator note: A number of increasingly inflammatory and unproductive posts have been trimmed, and further such discussion in this thread is ended.

This is not the place to air grievances about actions taken on Discord, IRC, Jira, or other support channels. Trimmed posts veered off into a clear violation of the Forum Rules, conveniently linked at the top of every page in red. To wit,

Be Respectful, Keep It Clean and Family-Friendly

Be respectful to others. Abuse, cursing, derogatory comments, insults, or personal attacks are not allowed. This is a dynamic community serving many different types of people from all over the world, and we all should be able to participate politely. Forum moderators will edit or remove inappropriate comments. In extreme cases, posters will be warned or banned.

In addition to these rules, everyone is asked to take a relaxed approach to helping others while posting at the TrueNAS Community. If you're having trouble talking in a calm and clear manner, please take a break and do not post. Come back later after the frustration has passed. Overly terse, derogatory, or insulting messages will be deleted and may lead to account suspension.

The TrueNAS community forums are here to provide user-to-user assistance and support. Please bear in mind that other community participants are trying to help, but this is all free software and you are not paying for support. Notwithstanding the advice in the Rules about frustration-posting, most people here have run into intractable problems and understand the desire for an expletive-laden outburst. However, this has gone too far and needs to be reeled in. I suggest redirecting your frustration in a constructive direction. Having to muddle through stuff trying to figure out answers is a large part of the job with free software. You don't always get a guide or how-to. You don't always get working software, even. That's opportunity to step up and step in, if you so desire.

Either way, please cease the inflammatory and hostile posts. Thank you!

</mod>