SOLVED Kubernetes service is not running

[ddos127]

Gurus

I'll cut to the case, we're all busy

• Clean install of TrueNAS-SCALE-22.12.0
• Added TrueCharts
• The first app (Tailscale) install yields the below error

Error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/middlewared/job.py", line 426, in run await self.future File "/usr/lib/python3/dist-packages/middlewared/job.py", line 461, in __run_body rv = await self.method(*([self] + args)) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1152, in nf res = await f(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1284, in nf return await func(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/chart_release.py", line 397, in do_create await self.middleware.call('kubernetes.validate_k8s_setup') File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1306, in call return await self._call( File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1255, in _call return await methodobj(*prepared_call.args) File "/usr/lib/python3/dist-packages/middlewared/plugins/kubernetes_linux/update.py", line 496, in validate_k8s_setup raise CallError(error) middlewared.service_exception.CallError: [EFAULT] Kubernetes service is not running.

Any assistance would be appreciated

 

[morganL]

It can be a variety of issues...
Its important that Internet is working
One user found it was a bad DIMM.
Several reported that they had to unset and reset the Kubernetes pool.

Problem: Kubernetes service is not running - TrueNAS Scale

I recently updated my TrueNAS Scale system to version 22.02.1 and now my apps don't seem to be running and are not installable. Every time I try to install and start apps like netdata or plex, it says: Error [EFAULT] Kubernetes service not running. Then lists the following info: Error...

www.truenas.com

 

[ddos127]

It can be a variety of issues...
Its important that Internet is working
One user found it was a bad DIMM.
Several reported that they had to unset and reset the Kubernetes pool.

Problem: Kubernetes service is not running - TrueNAS Scale

I recently updated my TrueNAS Scale system to version 22.02.1 and now my apps don't seem to be running and are not installable. Every time I try to install and start apps like netdata or plex, it says: Error [EFAULT] Kubernetes service not running. Then lists the following info: Error...

www.truenas.com

Thanks for the reply

• Pool was unset and reset
• Network is working without issue

Output of systemctl status k3s
● k3s.service - Lightweight Kubernetes Loaded: loaded (/lib/systemd/system/k3s.service; disabled; vendor preset: disabled) Active: activating (auto-restart) (Result: exit-code) since Mon 2023-01-30 15:24:12 AEDT; 759ms ago Docs: https://k3s.io Process: 22586 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS) Process: 22587 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS) Process: 22588 ExecStart=/usr/local/bin/k3s server --flannel-backend=none --disable=traefik,metrics-server,local-storage --disable-kube-proxy --disable-network-policy --disable-cloud-controller --node-name=ix-truenas --docker (code=exited, status=2) Main PID: 22588 (code=exited, status=2) CPU: 17.798s admin@truenas[~]#

In the process of migrating data so a reboot will need to wait

 

[ddos127]

• Pool unset and reset
• Reboot

issue resolved

 

[morganL]

• Pool unset and reset
• Reboot

issue resolved

So.. reboot was the missing instruction. Good for others to know.

 

[a.dresner]

reboot worked for me

 

[ra4design]

I had the same problem.

• I am running TrueNAS-SCALE-23.10.1
• The NIC is configured with br0 (This is to prevent service to go out to network in order to communicate with NFS storage running on the same truenas).
• In network configuraiton br0 is configured with two IPs: first one to be used by web gui, second one by kubernetes
• In Apps -> Advanced settings:
  • the Node IP is configured to the second IP
  • Route v4 Interface to br0
  • Route v4 Gateway to second IP

After reboot the k3s service was not running and web interface showing "Applications not running". Starting the service on command line also did not work.

Checking the log:

sudo journalctl -u k3s -n 1000

it seems that the k3s agent tries to connect to the first ip address on which the Web GUI is running

Dec 22 10:10:39 tn k3s[9680]: time="2023-12-22T10:10:39-08:00" level=info msg="To join agent node to cluster: k3s agent -s https://<first IP address>:6443 -t ${AGENT_NODE_TOKE>

So I had to go back to

1. single IP address on the br0
2. Truenass web gui ports moved to 81 and 444
3. And Node IP on the for kubernetes to the same IP

After this everything worked. However, I would much rather prefer to be able to run kubernetes on a separate IP address.

 

[morganL]

So I had to go back to

1. single IP address on the br0
2. Truenass web gui ports moved to 81 and 444
3. And Node IP on the for kubernetes to the same IP

After this everything worked. However, I would much rather prefer to be able to run kubernetes on a separate IP address.

Suggest starting a new thread on the IP address topic... with the reasosn for your preference
if there are no solutions, then make a suggestion in the jira system.