Traefik not routing after reboot

dslewiston

Cadet
Joined
Jun 13, 2022
Messages
9
I had to reboot my TrueNAS Scale the other day after stopping the middlewared service remotely. I hadn't enabled SSH so I wasn't able to start it again and had to manually rebooted the server when I got home. Since then none of my routes are working for the apps through Traefik. For example I have nextcloud running from the TrueCHARTS catalog as https://cloud.MYDOMAIN ( my environment is linked with CloudFlare using the setup guides provided by the TrueCHARTS Youtube channel. ( Thank you guys for all your amazing work! )

Under Traefik I can see the http Routers and Services, and everything looks like it should be working until I look further and see that the IP's don't seem to match what Kubernetes says it should be. Since Traefik dashboard is largely just a dashboard and not an interface to edit anything I am limited on how I can troubleshoot here.
Untitled.png

I can load the Traefik dashboard shows everything is fine and when I explore into the HTTP services I get the server showing a green checkmark with an IP of 172.x.x.4 but when I run "k3s kubectl get svc -A -o wide" I get an IP for the Nextcloud stack that don't match what I am seeing in Traefik. 172.x.x.254
svc.png


I have rebooted a few times and edited the apps configuration for Networking and Services from ClusterIP back to Simple ( which works to load the app locally based on the NodePorts ) and disabled the Ingress options. I then re-enabled them hoping it would rebuild and map the paths but it still isn't working. I am new to the Kubernetes world and am struggling to know where to begin tracking down the problem. To me it is a simple "networking" issue where the proxy has the wrong internal IP's of the nodes but I don't know enough of how to start to trouble shoot and check.

Any advice is greatly appreciated.
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
I had to reboot my TrueNAS Scale the other day after stopping the middlewared service remotely. I hadn't enabled SSH so I wasn't able to start it again and had to manually rebooted the server when I got home. Since then none of my routes are working for the apps through Traefik. For example I have nextcloud running from the TrueCHARTS catalog as https://cloud.MYDOMAIN ( my environment is linked with CloudFlare using the setup guides provided by the TrueCHARTS Youtube channel. ( Thank you guys for all your amazing work! )

Under Traefik I can see the http Routers and Services, and everything looks like it should be working until I look further and see that the IP's don't seem to match what Kubernetes says it should be. Since Traefik dashboard is largely just a dashboard and not an interface to edit anything I am limited on how I can troubleshoot here.
View attachment 56215
I can load the Traefik dashboard shows everything is fine and when I explore into the HTTP services I get the server showing a green checkmark with an IP of 172.x.x.4 but when I run "k3s kubectl get svc -A -o wide" I get an IP for the Nextcloud stack that don't match what I am seeing in Traefik. 172.x.x.254
View attachment 56216

I have rebooted a few times and edited the apps configuration for Networking and Services from ClusterIP back to Simple ( which works to load the app locally based on the NodePorts ) and disabled the Ingress options. I then re-enabled them hoping it would rebuild and map the paths but it still isn't working. I am new to the Kubernetes world and am struggling to know where to begin tracking down the problem. To me it is a simple "networking" issue where the proxy has the wrong internal IP's of the nodes but I don't know enough of how to start to trouble shoot and check.

Any advice is greatly appreciated.

I'd suggest having the discussion on discord with TrueCharts team as well. If its a bug on TrueNAS side, the TrueCharts team will tell us.
 

dslewiston

Cadet
Joined
Jun 13, 2022
Messages
9
I'd suggest having the discussion on discord with TrueCharts team as well. If its a bug on TrueNAS side, the TrueCharts team will tell us.
Thank you for that suggestion @morganL!

I ended up getting some help via the amazing community and the suggestion to open a ticket on the discord.

For anyone in the future, this turned out to be an issue with the network card being used. My server has 2 network cards installed. After the reboot the server started serving all the applications and traffic via the second network card. Changing the forwards in the firewall resolved the issue thanks to the Support Team!!! Endless thankful!
 
Top