Installing Tailscale in TrueNAS SCALE - Apps stuck at "Deploying" - Everything else works

[soso]

Hi fellow TrueNAS volunteers!

There seems to be a problem after installing Tailscale on TrueNAS Scale. While it seems to be working at first (you can access it fine, services such as SMB work as expected), any docker apps are stuck at the "deploying" stage. If someone has more knowledge on how such tools would interfere with the kubernetes workflow, I would be thankful!

I tested on VMs and on actual hardware to make sure to reproduce the problem. There was also an older post confirming that it does interfere somehow with kubernetes.

VERSION:
TrueNAS-SCALE-22.02-RC.2

INSTALLATION:

Code:

curl -fsSL https://tailscale.com/install.sh | sh
systemctl enable --now tailscaled
tailscale up # Connect

 

[Kris Moore]

You don't want to be running that kind of install script on TrueNAS SCALE. It's going to modify the base system image and potentially break things as you've seen. However if you want to request we investigate some support for Tailscale, please open a suggestion ticket on https://jira.ixsystems.com

 

[Ixian]

The better way to do it is run the Tailscale docker; it will work with the host system just fine and you avoid messing with the base image.

You can pretty easily stand one up using the docker button in Apps. Or the Truecharts folks may have a version of it already in their catalog.

 

[soso]

Thank you for your responses! I will be looking for either a docker version or moving to OpenVPN. Thanks again!

 

[aadje93]

Thank you for your responses! I will be looking for either a docker version or moving to OpenVPN. Thanks again!

altough offtopic, can highly suggest wireguard with pivpn, literally a couple commands and you have a running wireguard VPN in a vm without any hard networking skills required.

 

[snakesandtrails]

ok i am extremely frustrated with truenas scale, truecharts, and tailscale. nothing is ever easy or user friendly. I previously used tailscale as a way to connect to my home network remotely to stream my jellyfin server but speeds were crap so I upgraded my modem/router/isp and needed to reinstall tailscale. its stuck on deploying. ive had a similar issue before and "settings, unset pool, restart, set pool, install tailscale" worked but I am at a complete loss about how to get tailscale back up and running and am desperate for some help. This is my first server build so I am not the most knowledgable but have learned a lot. what do i do? also I tried the discord for truecharts and it is a freaking process to figure out how to even use the service let alone get help with an issue. at least truecharts forums are more helpful that way just in my reading

 

[Kris Moore]

ok i am extremely frustrated with truenas scale, truecharts, and tailscale. nothing is ever easy or user friendly. I previously used tailscale as a way to connect to my home network remotely to stream my jellyfin server but speeds were crap so I upgraded my modem/router/isp and needed to reinstall tailscale. its stuck on deploying. ive had a similar issue before and "settings, unset pool, restart, set pool, install tailscale" worked but I am at a complete loss about how to get tailscale back up and running and am desperate for some help. This is my first server build so I am not the most knowledgable but have learned a lot. what do i do? also I tried the discord for truecharts and it is a freaking process to figure out how to even use the service let alone get help with an issue. at least truecharts forums are more helpful that way just in my reading

Can you perhaps provide some additional details on how you tried to install it and what kind of error or logs you got from the failure? Happy to help review.

 

[snakesandtrails]

here are the logs:

2023-02-14T19:12:39.162566545Z boot: 2023/02/14 19:12:39 Using authkey found in kube secret
2023-02-14T19:12:39.162694986Z boot: 2023/02/14 19:12:39 Starting tailscaled
2023-02-14T19:12:39.163012704Z boot: 2023/02/14 19:12:39 Waiting for tailscaled socket
2023-02-14T19:12:39.187402611Z 2023/02/14 19:12:39 logtail started
2023-02-14T19:12:39.187445321Z 2023/02/14 19:12:39 Program starting: v1.36.1-t576b08e5e, Go 1.19.4-tsdc0ce6324d: []string{"tailscaled", "--socket=/var/run/tailscale/tailscaled.sock", "--state=kube:tailscale-tailscale-secret", "--statedir=/tmp", "--tun=userspace-networking"}
2023-02-14T19:12:39.187641831Z 2023/02/14 19:12:39 LogID: f734b57b8b4915945b28f1ec4939d170d0a60c174f16104ed653220386072984
2023-02-14T19:12:39.187666217Z 2023/02/14 19:12:39 logpolicy: using system state directory "/var/lib/tailscale"
2023-02-14T19:12:39.187679993Z logpolicy.ConfigFromFile /var/lib/tailscale/tailscaled.log.conf: open /var/lib/tailscale/tailscaled.log.conf: no such file or directory
2023-02-14T19:12:39.187686204Z logpolicy.Config.Validate for /var/lib/tailscale/tailscaled.log.conf: config is nil
2023-02-14T19:12:39.188793797Z 2023/02/14 19:12:39 wgengine.NewUserspaceEngine(tun "userspace-networking") ...
2023-02-14T19:12:39.188940613Z 2023/02/14 19:12:39 dns: using dns.noopManager
2023-02-14T19:12:39.189015524Z 2023/02/14 19:12:39 link state: interfaces.State{defaultRoute=enp1s0 ifs={enp1s0:[192.168.1.11/24] kube-bridge:[172.16.0.1/16] kube-dummy-if:[172.17.0.1/32 172.17.0.10/32 172.17.9.214/32 172.17.25.192/32 172.17.156.214/32]} v4=true v6=false}
2023-02-14T19:12:39.189734346Z 2023/02/14 19:12:39 magicsock: disco key = d:93a107e575b20fc3
2023-02-14T19:12:39.189770013Z 2023/02/14 19:12:39 Creating WireGuard device...
2023-02-14T19:12:39.189912040Z 2023/02/14 19:12:39 Bringing WireGuard device up...
2023-02-14T19:12:39.190870533Z 2023/02/14 19:12:39 Bringing router up...
2023-02-14T19:12:39.190902443Z 2023/02/14 19:12:39 Clearing router settings...
2023-02-14T19:12:39.190945163Z 2023/02/14 19:12:39 Starting link monitor...
2023-02-14T19:12:39.190992582Z 2023/02/14 19:12:39 Engine created.
2023-02-14T19:12:39.206908916Z 2023/02/14 19:12:39 pm: migrating "_daemon" profile to new format
2023-02-14T19:12:39.211581232Z 2023/02/14 19:12:39 got LocalBackend in 24ms
2023-02-14T19:12:39.211630795Z 2023/02/14 19:12:39 Start
2023-02-14T19:12:39.212362301Z 2023/02/14 19:12:39 Backend: logs: be:f734b57b8b4915945b28f1ec4939d170d0a60c174f16104ed653220386072984 fe:
2023-02-14T19:12:39.212565533Z 2023/02/14 19:12:39 Switching ipn state NoState -> NeedsLogin (WantRunning=false, nm=false)
2023-02-14T19:12:39.212639693Z 2023/02/14 19:12:39 blockEngineUpdates(true)
2023-02-14T19:12:39.212829911Z 2023/02/14 19:12:39 health("overall"): error: state=NeedsLogin, wantRunning=false
2023-02-14T19:12:39.213013095Z 2023/02/14 19:12:39 wgengine: Reconfig: configuring userspace WireGuard config (with 0/0 peers)
2023-02-14T19:12:39.213056336Z 2023/02/14 19:12:39 wgengine: Reconfig: configuring router
2023-02-14T19:12:39.213204455Z 2023/02/14 19:12:39 wgengine: Reconfig: configuring DNS
2023-02-14T19:12:39.213233029Z 2023/02/14 19:12:39 dns: Set: {DefaultResolvers:[] Routes:{} SearchDomains:[] Hosts:0}
2023-02-14T19:12:39.213244650Z 2023/02/14 19:12:39 dns: Resolvercfg: {Routes:{} Hosts:0 LocalDomains:[]}
2023-02-14T19:12:39.213254589Z 2023/02/14 19:12:39 dns: OScfg: {Nameservers:[] SearchDomains:[] MatchDomains:[] Hosts:[]}
2023-02-14T19:12:39.415936167Z boot: 2023/02/14 19:12:39 Running 'tailscale up'
2023-02-14T19:12:39.420110146Z 192.168.1.11/24 has non-address bits set; expected 192.168.1.0/24
2023-02-14T19:12:39.420543301Z boot: 2023/02/14 19:12:39 failed to auth tailscale: failed to auth tailscale: tailscale up failed: exit status 1

 

[snakesandtrails]

i haven't gotten any error messages. I am using tailscale version 1.36.1_3.0.16 to scale 22.12.0 using a truecharts repository. I follows the truecharts install guide

 

[snakesandtrails]

also, I followed raid owls video about it https://www.youtube.com/watch?v=o2MX10C8Ez8&t=132s

 

[snakesandtrails]

i have already installed it and gotten it working before, Its just stuck in "deploying" on reinstall

 

[snakesandtrails]

I have even tried to clear out any kind of cache data. I deleted my ix-applications data set and that didn't work. I paused my SMB shares (works for jellyfin) but it doesn't work for tailscale.

 

[snakesandtrails]

i followed this guide on initial install and on secondary installs as well

How-To | TrueCharts

This is a quick how-to or setup-guide to use Tailscale using on your TrueNAS box.

truecharts.org

 

[Kris Moore]

Ok, so I'm no expert on Tailscale, never used it. But that last error message before it fails, shows that the container is indeed getting spun up properly, but at the end tailscale fails to auth and then exit.

Code:

2023-02-14T19:12:39.420543301Z boot: 2023/02/14 19:12:39 failed to auth tailscale: failed to auth tailscale: tailscale up failed: exit status 1

Are you having to paste any auth / credentials into the setup of TailScale to set it up?

 

[foxswj]

I know this is a bit old but google search brought me here. I tried to find other threads but no one else seems to be using the app. Anyway as it seems to still be a problem going to make a note for other people. In the first couple of lines in the log it shows the following. I get it in my own truescale and just gave up on it. Note this is from the original poster log file above but I get it after a restart

2023-02-14T19:12:39.187679993Z logpolicy.ConfigFromFile /var/lib/tailscale/tailscaled.log.conf: open /var/lib/tailscale/tailscaled.log.conf: no such file or directory

For whatever reason the storage for the container does not keep the file /var/lib/tailscale/tailscaled.log.conf holds the private and public key. When the container restarts it either has to use the auth key (which expires after 90 days) or fails and you just get deploying.

 

[gnator]

Thats what I saw too. For now the only way to fix it is to delete the application and re-install in order for it to bypass the missing file.