Kubernetes not working after upgrade to 22.12, doesn't work on 22.04 either

TempleHasFallen

Dabbler
Joined
Jan 27, 2022
Messages
34
Due to some issues (double boot device failure), I had to do a fresh install and restore the config from backup. (not to mention having to derive a missing pool encryption key from wrapping keys :/ )

Old system was 22.02.03, was restored to a fresh 22.02.04 installation.

After restore and unlocking of datasets, apps were visible and working without an issue in 22.02.04.
Shortly after, I upgraded to 22.12. Due to an issue with my X540-T2's (not recognized/no connection status), I switched back to 22.02.04.

In 22.04, the following errors are present and apps don't load at all, seems that kubernetes is not running:


Code:
Failed to configure kubernetes cluster for Applications: [EFAULT] Docker service is not running

Failed to start kubernetes cluster for Applications: (500) Reason: Internal Server Error HTTP response headers: HTTP response body: apiserver not ready

Failed to start kubernetes cluster for Applications: Cannot connect to host 127.0.0.1:6443 ssl:default [Connection reset by peer]

Failed to start kubernetes cluster for Applications: [EFAULT] Failed to configure PV/PVCs support: Response payload is not completed

Failed to start kubernetes cluster for Applications: [EFAULT] Failed to configure PV/PVCs support: Cannot connect to host 127.0.0.1:6443 ssl:default [Connection reset by peer]

Failed to start kubernetes cluster for Applications: [EFAULT] Failed to configure PV/PVCs support: Cannot connect to host 127.0.0.1:6443 ssl:default [Connect call failed ('127.0.0.1', 6443)]

Failed to sync TRUECHARTS catalog: [EFAULT] Invalid operation: ==


1681725863717.png

I've tried:

- Unsetting applications pool, rebooting, setting it again, waiting, rebooting, waiting.
- Restarting/Starting docker service
- Restarting middlewared
- Running 22.12 - apps not working there either
- Triple checked NTP settings (time is correct)
- Triple checked Timezone
- Starting k3s (works)
- Restarting docker (fails):
1681726615005.png

Apps are not located on the boot-pool but rather on a separate pool/dataset (system dataset, NVMe).

Is the upgrade 22.02.04 --> 22.12 a one way upgrade that upgrades apps or something in the cluster?
If so, could it be reverted by restoring the ix-applications dataset from snapshot to a version where it was running on 22.02?
I'm desparately trying to avoid having to do a fresh installation, due to lots of custom apps and a massive amount of configs.

Any help is greatly appreciated.

System specs are in signature.
 

highkick05

Cadet
Joined
Aug 7, 2022
Messages
3
having this problem also. everytime I reboot, I have to UNSET the POOL, then SET again for it to find all my docker apps. Looks as if kubernetes is not starting correctly
 

highkick05

Cadet
Joined
Aug 7, 2022
Messages
3
actually, this may be a different issue. Also having random issues with dockers failing to start and docker images dissapearring.. is it just better to detele the pool and restart all over again?
 
Top