Wireguard client o on TrueNAS scale

[krzysz00]

Since the one thread I found on this issue ( https://www.truenas.com/community/threads/connect-truenas-to-wireguard-mullvad.109716/ ) didn't seem to have a resolution, and since the OpenVPN client is being deprecated, I figured I should ask: how do I make my TrueNAS scale install a Wireguard client?

That is, I'll be running a wireguard server on another machine that's got a stable domain (let's call it foo.bar). foo.bar will, among other things, run a Wireguard VPN that will relay traffic between all its peers ... which will be my TrueNAS server (sitting behind a dynamic IP and a NAT, hence the extra host) and various devices like my phone.

The official wg-easy application appears (given things like the mandatory "Client DNS" setting) to be focused on running a Wireguard server. Unless I'm missing something fundamental about Wireguard, it seems like setting up the app isn't set up for allowing configuring something like

Code:

[Interface]
# Server's IP on my notional 192.168.23.0/24 subnet, where foo.bar is 192.168.23.1
Address = 192.168.23.2/24
PrivateKey = xxxxxx

[Peer]
AllowedIPs = 192.168.23.0/24
PublicKey = yyyyyyy
Endpoint = foo.bar:51820
PersistentKeepalive = 25

Is there a way to generate the equivalent of this configuration using the official wg-easy app?

If not, should I be using the @truecharts wireguard app to handle this usecase?

 

[krzysz00]

To answer my own question, I got something working - though this probably isn't anyone's Recommend Solution (tm) - by using TrueChart's wireguard app, pointing it at a wireguard config I'd generated, and (it's hidden under "Pod options") turning host networking on. (If I didn't do that, the wg-quick script would get permission issues)

Now my machine has an on-VPN IP address like I'd like it to, wg0 shows up as a regular host interface, and I didn't need to do anything that counts as hacking directly on the host system.

 

[nickcmaynard]

To answer my own question, I got something working - though this probably isn't anyone's Recommend Solution (tm) - by using TrueChart's wireguard app, pointing it at a wireguard config I'd generated, and (it's hidden under "Pod options") turning host networking on. (If I didn't do that, the wg-quick script would get permission issues)

Now my machine has an on-VPN IP address like I'd like it to, wg0 shows up as a regular host interface, and I didn't need to do anything that counts as hacking directly on the host system.

Just a quick thanks from me, @krzysz00 - exactly what I needed.

 

[ThibautK]

Got this working as well, but without host networking. I couldn't reach some of my application (immich) that way so i looked for another solution.
Figured in this case I had to reach the internal ip (172.17.0.0/16) of the application. So we have to be able to resolve the internal DNS names (*.svc.cluster.local) on the VPS, this is achieved this by using the k3s DNS resolver (172.17.0.10) of truenas on the VPS, which is all working over the wireguard client.

Network overview:
VPS Wireguard subnet:

• 172.30.0.0/24

Truenas application subnets:

• 172.16.0.0/16
• 172.17.0.0/16

Local subnets:

• 192.168.50.0/24
• 192.168.25.0/24

Application on truenas for this example:
- Metallb (optional)
- Traefik
- Blocky
- Plex
- Immich

1. VPS - Wireguard server config

Code:

[Interface]
Address = 172.30.0.1/24
ListenPort = 51820
PrivateKey = xxx
MTU = 1500
PostUp =
PostDown =

# Name:         truenas
[Peer]
PublicKey = xxx
PresharedKey = xxx
AllowedIPs = 172.30.0.2/24,172.16.0.0/16,172.17.0.0/16,192.168.25.0/24,192.168.50.0/24

2. Truenas - Wireguard client:

Code:

[Interface]
Address = 172.30.0.2/24
PrivateKey = xxx
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
MTU = 1500
FwMark = 0xca6c

[Peer]
PublicKey = xxx
PresharedKey = xxx
AllowedIPs = 172.30.0.0/24
Endpoint = xxx:51820
PersistentKeepalive = 15

Install and configure truecharts wireguard client:
- Config File Host Path --> Point it towards the file on truenas, no hostpaths (/mnt/pool/...)

At this point you should be able to ping applications on your truenas from your VPS.

3 - VPS - DNS resolving:

Code:

/etc/resolv.conf
nameserver 172.17.0.10
search svc.cluster.local cluster.local

Note: you might need to install resolvconf on your VPS in order to be able to edit resolv.conf..

When this is done, you should now be able to ping plex.ix-plex.svc.cluster.local from your VPS..

You can use this in your reverse proxy as well, if you have a domain. Make sure to use corects ports as well.

Example (caddyfile):

Code:

plex.yourdomain.com {
    reverse_proxy plex.ix-plex.svc.cluster.local:32400
    tls {
        ..
    }
immich.yourdomain.com{
    tls {
       ..
     }
     reverse_proxy immich.ix-immich.svc.cluster.local:10323
}
}

4. Traefik & blocky
Note the above was all for WAN traffic.. On LAN side i choose an easy to use approach by using Traefik and blocky (dns).

 

[wasalph]

Got this working as well, but without host networking. I couldn't reach some of my application (immich) that way so i looked for another solution.
Figured in this case I had to reach the internal ip (172.17.0.0/16) of the application. So we have to be able to resolve the internal DNS names (*.svc.cluster.local) on the VPS, this is achieved this by using the k3s DNS resolver (172.17.0.10) of truenas on the VPS, which is all working over the wireguard client.

Hi,

I was only able to make wireguard work partially and only when the host network was NOT check. When checked, it will stay as "DEPLOYING" and doesn't get into "ACTIVE".

With Host Network unchecked, it shows ACTIVE, however, I can not access the NAS from another location. Since I have existing wireguard setup from a VPS and several desktop clients in different locations and a few mobile phones wireguard clients as well, I just want to add the TrueNAS so that I can remotely access the SMB share folders.

PS: lets just assume the VPS wireguard server has the following setup:

server address: foo.bar:51800
server wireguard ip: 10.1.1.1

Remote Client1 (local IP: 192.168.1.5)
[Interface]
PrivateKey = **privatekeyclient1**
ListenPort = 51801
Address = 10.1.1.2/24
DNS = 1.1.1.1, 8.8.8.8

[Peer]
PublicKey = **publickeyserver**
AllowedIPs = 192.168.0.0/24, 10.1.1.0/24
Endpoint = foo.bar:51800
PersistentKeepalive = 15

Remote Client2 (local IP: 192.168.0.5)
[Interface]
PrivateKey = **privatekeyclient2**
ListenPort = 51802
Address = 10.1.1.3/24
DNS = 1.1.1.1, 8.8.8.8

[Peer]
PublicKey = **publickeyserver**
AllowedIPs = 192.168.1.0/24, 10.1.1.0/24
Endpoint = foo.bar:51800
PersistentKeepalive = 15

with the above...
... remote client 1 can ping 10.1.1.1, 10.1.1.3, 192.168.0.5. I can also view shared folder from 192.168.0.5
... remote client 2 can ping 10.1.1.1, 10.1.1.2, 192.168.1.5. I can also view shared folder from 192.168.1.5

Now in TrueNAS, I install Wireguard (via TrueChart) and provided the following wg.cfg stored in /mnt/pool/

TrueNAS (local IP: 192.168.1.6)
[Interface]
PrivateKey = **privatekeytruenas**
Address = 10.1.1.6/24
ListenPort = 51806
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
DNS = 1.1.1.1, 8.8.8.8

[Peer]
PublicKey = **publickeyserver**
AllowedIPs = 192.168.1.0/24, 10.1.1.0/24
Endpoint = foo.bar:51800
PersistentKeepalive = 15

with Host Network unchecked...
... remote client 1 can ping 10.1.1.6, but can not ping 192.168.1.6. Can not also view shared folder both 10.1.1.6 nor 192.168.1.6
... remote client 2 can ping 10.1.1.6, but can not ping 192.168.1.6. Can not also view shared folder both 10.1.1.6 nor 192.168.1.6
... truenas shell can not ping 10.1.1.1, 10.1.1.2, 10.1.1.3 or any of the 192.168.x.x
... truenas wireguard shell can ping 10.1.1.x, 192.168.x.x

using Truenas wireguard shell, i get this:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
3: eth0@if98: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 3e:13:f8:57:12:0e brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.16.0.92/16 brd 172.16.255.255 scope global eth0
valid_lft forever preferred_lft forever
4: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
link/none
inet 10.1.1.6/24 scope global wg0
valid_lft forever preferred_lft forever

How can I make my remote client access the TrueNAS shared folder or successfully ping 192.168.1.6?

 

[wasalph]

How can I make my remote client access the TrueNAS shared folder or successfully ping 192.168.1.6?

ok I found the problem.
when the TrueNAS - Wireguard - Host Network is check, I found the log file saying that wg0 already exist, no wonder it says in "DEPLOY" and doesn't go "ACTIVE". Using the TrueNAS shell then do an "ip addr", there indeed exist wg0. I search all throughout how to delete or uninstall wg0 but to no avail. What I did was install wg-easy, then delete wg-easy. The wg0 is gone and the TrueNAS wireguard now works with Host Network enabled... which solves all the problem of pinging and accessing SMB from remote client.

I think there is a bug on how Truenas scale report the status of the app being started. For something similar to my case, it should not stay in "DEPLOY" but maybe gives an error like failed to start or something.

Another bug is that how come the wg0 remains in the network list when wg-easy was first deleted.

Anyway, now it is working

 

[sfatula]

Glad you got it working. As an aside for others reading, you can also just use the wireguard client build into Scale. You just create a config file on some persistent storage and use an init script to start wireguard via wg-quick with the config file. I did this when trying to get inbound traffic to Truenas from an external VPS via wireguard (I have cgnat).

 

[Turnspit]

Glad you got it working. As an aside for others reading, you can also just use the wireguard client build into Scale. You just create a config file on some persistent storage and use an init script to start wireguard via wg-quick with the config file. I did this when trying to get inbound traffic to Truenas from an external VPS via wireguard (I have cgnat).

Could you elaborate?

I'm having the same issue with wg0 exisiting when trying to use Truecharts's Wireguard app in Host Network mode. Installing/Installing gets rid of the interface, leading to other errors whilst setting up the App though... It's a pain...

Just why did they remove OpenVPN Client from the GUI? It used to work so seamlessly...

 

[sfatula]

I have a script that runs after startup via the Scale UI named in my case /mnt/tank/Scripts/StartWireguard.sh, it looks like this:

#!/bin/bash sleep 120 /usr/bin/wg-quick up /mnt/tank/Scripts/Wireguard/wg10.conf

Assuming you know how to craft a wireguard config file on other hosts that works, then that's really all there is to it. It is unlikely my wg10.conf will work for you as it all depends exactly what you are trying to do in detail. There are many guides to getting Wireguard working (the conf file) and it's likely best to find one wanting to do the exact same thing as you if you are not familiar with wireguard.

I use mine for access to my lan from outside my lan via on demand wireguard on IOS and a travel router when on a trip as it serves other devices like Roku. In my case, behind CGNAT, Truenas wireguard actually connects to a VPS of mine as some sort of static ip without double NAT is needed. Most people likely wouldn't need that.

 

[Turnspit]

Thank you for the quick response!

So I took a conf-file which is working across different clients (Ubuntu, Windows, Android) without problems for a test run of your solution, placed it on a dataset on the TN machine and issued the following command:

wg quick up /mnt/tank/Wireguard/wg1.conf

Which is followed by:

[#] ip link add wg1 type wireguard [#] wg setconf wg1 /dev/fd/63 [#] ip -4 address add 10.8.0.3/24 dev wg1 [#] ip link set mtu 1420 up dev wg1 [#] resolvconf -a wg1 -m 0 -x [#] wg set wg1 fwmark 51820 [#] ip -6 route add ::/0 dev wg1 table 51820 [#] ip -6 rule add not fwmark 51820 table 51820 [#] ip -6 rule add table main suppress_prefixlength 0 [#] nft -f /dev/fd/63 [#] ip -4 route add 0.0.0.0/0 dev wg1 table 51820 [#] ip -4 rule add not fwmark 51820 table 51820 [#] ip -4 rule add table main suppress_prefixlength 0 [#] sysctl -q net.ipv4.conf.all.src_valid_mark=1 [#] nft -f /dev/fd/63

And creates the interface:

wg1: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000 link/none inet 10.8.0.3/24 scope global wg1 valid_lft forever preferred_lft forever

Yet it doesn't seem to connect to my Wireguard server. The exact same config-file works with the Truecharts Wireguard app (when not running in host network mode, which borks other stuff...), so I don't see a problem with the config itself.

Is there something I might be missing?

 

[sfatula]

That's a Wireguard question and that's not your wg1.conf config file. The wireguard subrediit in rediit is a good resource. I am not the best at Wireguard config files but it would be far more helpful to show your config file, with keys redacted of course. It's not unusual at all to have a config file that doesn't connect, wireguard is very exacting.

 

[Turnspit]

That's the problem... since my Wireguard config works virtually everywhere else (Ubuntu Server, Ubuntu Desktop, Windows 10/11, Android, iOS) the wg-folks will call it a day as a Truenas specific problem...

My conf file looks like this:

[Interface]
PrivateKey = <key>
Address = 10.8.0.3/24
DNS = 192.168.2.1

[Peer]
PublicKey = <key>
PresharedKey = <key>
Endpoint = mydomain.com:51820
AllowedIPs = 0.0.0.0/0, ::0/0
PersistentKeepalive = 25

As I said, works pretty much everywhere else, with routing between both networks (lan <-> wg) running seamlessly.
Also tried to change the DNS to something public and replacing the endpoint DNS-name with the static ip-address (just in case TN has somehow problems resolving names) - to no avail...

 

[sfatula]

Where are your iptables rules in the conf file? PostUp and PostDown? You are trying to route all traffic via mydomain.com?

 

[Turnspit]

Correct, my Wireguard setup consists of a publicly available bounce server that handles all the forwarding and routing, with all other peers connecting directly to it and virtually having no direct information of any other peer.

 

[sfatula]

Hmm, never ever seen a server wireguard config without PostUp and Down rules. They are not for security, they are for routing between interfaces on the server itself to allow Wireguard to work. I would expect the default iptables rules on Truenas to deny all traffic to the wireguard interface, mine does. That's why the rules were needed. Mine doesn't work without those rules.

 

[Turnspit]

Would you mind sharing your rules (default iptables blocking stuff on TN might indeed be the problem) for me to test things out, using them as a template, to modify them as needed?

 

[sfatula]

For wide open, this can be changed to limit if desired but start with this:

PostUp = iptables -I FORWARD 1 -i wg10 -j ACCEPT PostUp = iptables -A FORWARD -o wg10 -j ACCEPT PostDown = iptables -D INPUT -i wg10 -j ACCEPT PostDown = iptables -D FORWARD -o wg10 -j ACCEPT

Replace wg10 with your wireguard interface you are creating. It all goes in the conf file interface section.

 

[Turnspit]

Thanks again for your input!

Sadly, this template - modified accordingly - doesn't get stuff working for me as well. Tried a plethora of other possible iptables modification. to no avail sadly.

Just to be sure I fired up a fresh Ubuntu VM without enabling ip-forwarding and enabled ufw, thus blocking all traffic/ports. Installed wireguard, loaded up my wg1.conf file without any PostUp settings and it worked right out of the box. Was able to connect to my wireguard "server" and was able to reach the new VM from my LAN as well.

So it really seems that TrueNAS does block something here. Are you running on 23.10.x as well?

 

[sfatula]

I guess I have no more comments then, sorry. If I enable wireguard on a brand new ubuntu machine, enable firewall and block all traffic/ports, and, do not enable IP forwarding, I do not get a connection. IP forwarding won't matter unless trying to reach machines other than the wireguard one but it's enabled on Scale. The firewall will matter, that makes no sense so I guess I don't have an answer for you.

That a firewall won't block traffic for you is mystifying to me.

 

[Turnspit]

After (a lot) more tinkering I figured it out - the conf-file turned out to be the problematic part.

This was my initial one:

[Interface]
PrivateKey = <key>
Address = 10.8.0.5/24
DNS = 192.168.2.1

[Peer]
PublicKey = <key>
PresharedKey = <key>
Endpoint = mydomain.com:51820
AllowedIPs = 0.0.0.0/0, ::0/0
PersistentKeepalive = 25

What ended up being problematic (on a linux machine at least) were the AllowedIPs in the Peer section, somehow terminating the public interface/accessibility. An Ubuntu client on a VPS for example was reachable with the above mentioned config via it's wg-interface, but it's public IP wasn't reachable from the outside anymore.
Thus, I modified the conf-file like this:

[Interface]
PrivateKey = <key>
Address = 10.8.0.5/24
DNS = 9.9.9.9

[Peer]
PublicKey = <key>
PresharedKey = <key>
Endpoint = mydomain.com:51820
AllowedIPs = 10.8.0.1/32
PersistentKeepalive = 5

I only allowed my LAN as well as the Wireguard network on the server-Peer, and everything worked as expected!
For good measure I also changed the DNS-server to a public one, just to be sure.

Your startup script also works as expected.

EDIT:
Had to slightly change the conf after some more testing, only allowing my wg-'server' as a peer on my TN-conf, as well as enabling PersistentKeepalive on both side with a rather low value (though this might be increased again) for a constant and stable connection.