tailscale in TrueNAS Scale using Truecharts

[jengle]

Greetings,
First post here and learning lots.
I have SMB set up with two shares in two different pools set up and working; Nextcloud works and home-assistant is installed but I don't have devices added yet. No I want to set up a VPN to remotely access security camera video when I install them. I decided to use tailscale after reading how easy it was and got my key & set it up on two devices and my TrueNAS Scale installation. It worked - I was able to ping each device. However I was not able to access the CLI in tailscale getting:

WARN:[0000] Unable to read /etc/rancher/k3s/k3s.yaml, please start server with --write-kubeconfig-mode to modify kube config permissions
error: error loading file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied

With a lot of YouTube and web searches I tried lots of things with no success finally finding a good match using a curl command. That hosed up my system and after a few hours struggling with that I ended up removing my Apps and associated datasets, reinstalling Truenas with the reformat option and re-importing my pools. Reconfigured SMB and re-installed Nextcloud and home-assistant (then I did a backup!!!).

Following the instructions here: https://truecharts.org/charts/stable/tailscale/How-To-Guide/ I re-installed the app and I am still having problems. The primary problem is that I am getting invalid key: API key xxxxxxxx not valid in the log. I am using the same master key I used on the initial installation. I think this is the reason it is stuck at deploying.

My configuration;
tailscale setup Truenas Sysctl added variables per Truenas setup guide
net.ipv4.ip_forward - value 1 - enabled
net.ipv4.conf.all.src_valid_mark - value 1 - enabled
tailscale setup - Apps
Application name: tailscale
Desired replicas 1 (default)
Extra Args (not selected)
Timezone: America/Los_angeles'timezone
Extra Environment Variables (nothing)
App Configuration
- Auth Key (master key entered - worked first setup)
- Auth Once checked
- Userspace - checked
- Routes 192.168.1.0/24
- Dest IP blank
- Sock5 Server blank
- Outbound HTTP Proxy Listen blank
- Hostname blank
- Advertise as exit node checked
- Extra Args blank
- Tailscale Daemon Extra Args blank
Networking and Services
- Show expert mode checked
- Host-Networking not checked
- Host Interfaced - added 'enp4s0' Interfaced --> Added
- IP Address Management IPAM Type - Use DHCP selected --> selected
Add Manual Custom Services - none added
Storage and Persistence (all at default)
- Additional App Storage - nothing selected
Ingress
Nothing checked or selected as default
Security and Permissions (all at default)
- Change PUID/UMASK values - not checked
- Show Advanced Security Settings - not checked
- Pod Security Context
runAsUser - 0 (default)
runAsGroup - 0 (default)
faGroup - 568 (default)
Whenshould we take ownership - OnRootMismatch (default)
Supplemental groups - none added
Resources and Devices (all at default)
- CPU - 4000m (default)
- RAM - 8Gi (default)
- Mount USB devices - nothing added
GPU Configuration
GPU Resource (gpu.intel.com/i915) - Allocate 9 gpu.intel.com/i915 GPU (default)
Addons
Nothing checked or selected as default
Advanced
Nothing checked or selected as default

When I bring up the log I get a pop-up connection error: kubernetes.pod_log_follow:{"release_name":"tailscale", "pod_name":"tailscale-5688c7bfb7-j4ctz", "container_name":"tailscale", "tail_lines": 500} nosub

I am still unable to access the CLI with the same error message above,

I removed the machine from tailscale.com & restarted tailscale app, but same error message and stuck in deploying.

I can't find a recent guide that has a bit more information on the options and all the YouTube videos have slightly different options from the current version of tailscale. The tailscale forum site has no replies for any of the TrueNAS/Truecharts questions posted. It's so easy to set up on iOS and Windows devices. I can't set up a virtual machine on this system and would like to use a single package for my little setup.

Any help would be appreciated - I don't want to break my TrueNAS installation again

 

[jengle]

I deleted my tailscale account, re-added to generate a new key and the app runs now.

1). still unable to access the CLI for tailscale app with the error of unable to access k3s.yaml:

WARN:[0000] Unable to read /etc/rancher/k3s/k3s.yaml, please start server with --write-kubeconfig-mode to modify kube config permissions
error: error loading file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied

2) When I bring up the log I get a pop-up connection error: kubernetes.pod_log_follow:
{"release_name":"tailscale", "pod_name":"tailscale-5688c7bfb7-j4ctz", "container_name":"tailscale", "tail_lines": 500} nosub

all machines (TrueNAS server, Windows desktop and iPhone can be pinged, and are in the tailscale management screen.

how do I get access to the CLI of tailscale?

 

[mihies]

1). still unable to access the CLI for tailscale app with the error of unable to access k3s.yaml:

WARN:[0000] Unable to read /etc/rancher/k3s/k3s.yaml, please start server with --write-kubeconfig-mode to modify kube config permissions
error: error loading file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied

From my (limited) experience this error happens when the user account you are logged with doesn't have access to said file. Did you try logging in as root?

 

[jengle]

Thanks @mihies. For TrueNAS scale using root should not be required, and if so should be part of the instructions. Admin installed the app, and should be able to get into the CLI, so I don’t believe that logging in as root applies here. From the TrueNAS install instructions:

The root administrator login is deprecated for security hardening, and it might be disabled in future versions of SCALE.

Logging in with the root username and password generates an alert in the UI, instructing you to create an administrative user account.

As a security best practice and best compatibility with future SCALE version releases, immediately create the new administrative account after signing into SCALE.

I did figure out that I must have advertise as exit node for the app to run. I get the VPN working, and can ping the 4 configured peers using the tailscale assigned IP addresses but cannot get into the web UI for trueNAS, home assistant or nextcloud. Access is fine using the set-up VPN and using the domain name (duckdns).

Why can’t I use the tailscale assigned IP addresses?

Why can’t I access the CLI?

 

[mihies]

I agree on "it should not be required" and other points. However, the file in question is owned by root:root, at least on my machine. And logging in using root it allows you to use shell. Said that, I'm all for an alternative more secure option... Perhaps an option is to chown/chmod on said file.
Here they talk about it on k3s issues.

 

[jengle]

I agree on "it should not be required" and other points. However, the file in question is owned by root:root, at least on my machine. And logging in using root it allows you to use shell. Said that, I'm all for an alternative more secure option... Perhaps an option is to chown/chmod on said file.
Here they talk about it on k3s issues.

Good idea @mihies. I had tried the chmod (although it's not recommended for security reasons) but it was not persistent across system restarts. The chown did the trick. Thanks.

 

[mihies]

Yes, only using chmod is dangerous if you give everyone access. Hence chown and chmod on group and user, but not everyone. I guess. But I'd still like to see official TrueNAS recommendations. :)

 

[mihies]

Looks like it's a known issue and it will be fixed in future