Route all Traffic through OpenVPN Service

HungrySkelly

Dabbler
Joined
Nov 20, 2020
Messages
11
Hi all,

I've set up a VPN server using the shiny new OpenVPN Service. I have successfully got the VPN up and running and have successfully connected via the OpenVPN iOS app on my phone.

My issue is that it doesn't seem to force all traffic through the VPN.
I'm able to communicate through the VPN by adding ```push "route 192.168.1.0 255.255.255.0"``` to the Additional Parameters, however I note that only local IP requests are routed this way.
I tried adding ```push "redirect-gateway def1"``` to the Additional Parameters, however this just locked up the VPN and stopped it from working.

Is there any particular tweaks I need to make to force all traffic through the VPN?
I'm wondering if perhaps I need to do something via my router (ASUS RT-AC68U running Merlin) instead of the VPN setup?

Thanks in advance for your help and guidance :)

Skelly
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
So you pushed "route 192.168.1.0 255.255.255.0" and that worked for local traffic.

The reason that worked for local traffic is because your local traffic is 192.168.1.0/24 (presumably).

Now, a few things.

1) Not everyone wants to force all traffic through the VPN connection. There are upsides and downsides. Personally, I don't see a good reason not to, except that many sites cannot handle the load. I would rather have my network connection go through a known quantity, without unexpected filtering, etc. So I do it all the time.

2) If you do force all the traffic through the VPN connection, you need to beware that the packets being exchanged with the VPN server will ALSO be pushed over the VPN link, and things will rapidly come to a stop. You need to include something like "route remote_host 255.255.255.255 net_gateway 1" to create a more-specific route that lets you reach the VPN server (which is known as remote_host).

3) My network is not your network, and my operating system is not your operating system, and I am not doing this on FreeNAS, though I've done the iOS bits at times. I have, however, found that there is a lot of crap related to default (0.0.0.0) network being routed and relearned by DHCP or other parts of an OS. As a result, I prefer to route 0.0.0.0/1 and 128.0.0.0/1 as two separate routes, rather than just the more obvious push "route 0.0.0.0 0.0.0.0". This works out to:

route 0.0.0.0 0.0.0.0 vpn_gateway 1
route 0.0.0.0 128.0.0.0 vpn_gateway 1
route 128.0.0.0 128.0.0.0 vpn_gateway 1

This is for a TAP-based OpenVPN setup. iOS uses TUN. I don't remember if there are changes to be made, I'm not trying to give you a complete answer, just some tips as to where the mines in the minefield might be.
 

HungrySkelly

Dabbler
Joined
Nov 20, 2020
Messages
11
Thanks for the reply, @jgreco - much appreciated!

So you pushed "route 192.168.1.0 255.255.255.0" and that worked for local traffic.

The reason that worked for local traffic is because your local traffic is 192.168.1.0/24 (presumably).

That is correct.

This is for a TAP-based OpenVPN setup. iOS uses TUN.

I see. I've got it set up as TUN for the VPN server as well, so no TAP used here.

I've tested what you've provided above to no avail.
I've also done a few hours more research based on your reply, but have not been able to make any tangible progress.
However it has lead me to believe that perhaps it's more of a routing issue (on the ASUS router) than a VPN server issue, since the VPN server is NOT on the same machine as the gateway to the network...
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
I've tested what you've provided above to no avail.
I've also done a few hours more research based on your reply, but have not been able to make any tangible progress.
However it has lead me to believe that perhaps it's more of a routing issue (on the ASUS router) than a VPN server issue, since the VPN server is NOT on the same machine as the gateway to the network...

Ah. Yes. I think I see your problem.

Because TAP interfaces create a layer 2 interface, the VPN endpoint ends up looking very similar to a natively connected ethernet host on the LAN. The LAN can take care of most routing policy, with certain things being overridden by the VPN.

For TUN designs, it is important that you have a coherently designed routing environment, because the TUN creates its own IPv4 networking (I believe 10.8.0.x by default).

Normally people "solve" this problem by placing the OpenVPN on the same host as the gateway, but this is only possible on relatively trite setups. The more correct setup, which solves the problem generally, is to make sure you have routes properly installed. Each device needs to have reachability to all other desired endpoints. This can get a little more complicated if you have cheap NAT gateways (ASUS, Netgear, etc., "routers") because these often don't have the ability to specify additional routes or to NAT from more than a single network.

I don't really have time to go into this in more detail right now, I'm sorry.
 

HungrySkelly

Dabbler
Joined
Nov 20, 2020
Messages
11
Thanks again @jgreco!

Normally people "solve" this problem by placing the OpenVPN on the same host as the gateway
I originally had (and still do have) a VPN on my router which functions correctly, however I'm attempting to migrate this over to the TrueNAS. Once I've got the VPN server setup, I'll also be able to connect my offsite NAS for replication purposes without having to bundle my whole home network through the VPN :)


This can get a little more complicated if you have cheap NAT gateways (ASUS, Netgear, etc., "routers") because these often don't have the ability to specify additional routes or to NAT from more than a single network.
In my case, my router (ASUS RT-AC68U running Merlin) does have routing tables. I've been playing around with them since my last reply here but haven't had much success. Here's a screenshot of the interface (along with my most recent routing attempt). 10.1.0.0/24 is the VPN subnet. 192.168.1.5 is the local address of my TrueNAS box. 192.168.1.1 is the local address of my router

1606079860233.png

1606080194056.png

1606080057269.png


I don't really have time to go into this in more detail right now, I'm sorry.
No need to apologise at all - your comments thus far have been valuable in helping me to isolate the source of my issue. Thank you!! :)
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
I meant to comment that the best strategies to debug this are to make extensive use of ping and traceroute, in my opinion. It may also help if you can have a laptop with OpenVPN handy, because it is easier to hack on ".ovpn" files on the laptop. Done correctly, there are huge similarities between the iOS config and PC config files, and I guarantee the hacking goes quicker on the PC.

Get a decent ping and traceroute utility. For connections to your LAN, which I believe you indicated already work, do a traceroute to nodes on the LA to get an idea of what that looks like (it'll probably go something like 10.1.0.1 on the first hop, then 192.168.0.X on the last hop). If it turns out that your outgoing traffic isn't going over the VPN, for example, that's a big clue as to what's gone awry.

So I do have one question here. By screenshots above, you are using your TrueNAS host (192.168.1.5) as the next hop. TrueNAS doesn't route, at least not by default. Now, bearing in mind that I do not know *ANYTHING* about the "Shiny New OpenVPN Service" which sounds like it might be a TrueNAS-ism, so that's not running in a jail or anything? Again, despite "moderator," and as an experienced Internet engineer who's designed incredibly complicated networks with FreeBSD, I haven't seen this OpenVPN service for TrueNAS, and I know nothing about it, so my question is as follows -

What happens if you type "sysctl -w net.inet.ip.forwarding=1" at the TrueNAS command prompt? Do things suddenly start working (or working "more")? Also interesting is if it is already enabled (it will say "1 -> 1" instead of "0 -> 1").

This is nonhazardous to your NAS, is not persistent in the configuration, and you can set it back to "0" at any time. It temporarily turns on packet forwarding, making your NAS into a router. The reason that this isn't needed for the OpenVPN jails is because their network stacks are separate from the TrueNAS host network stack, and I assume it is set to 1 within the jail. But if you are using the TrueNAS host itself in a role that should require routing, I would think you'd need that to be enabled.
 

HungrySkelly

Dabbler
Joined
Nov 20, 2020
Messages
11
Thanks @jgreco!

I meant to comment that the best strategies to debug this are to make extensive use of ping and traceroute, in my opinion. It may also help if you can have a laptop with OpenVPN handy, because it is easier to hack on ".ovpn" files on the laptop.
Great suggestion - I've got a laptop that I can use for this purpose so I'll give it a crack! I'll report back on this once I've had a chance to give it a go :)

I haven't seen this OpenVPN service for TrueNAS
It became available with Release 12.0 and can be found in the Services tab:
1606088916725.png


What happens if you type "sysctl -w net.inet.ip.forwarding=1" at the TrueNAS command prompt? Do things suddenly start working (or working "more")? Also interesting is if it is already enabled (it will say "1 -> 1" instead of "0 -> 1")
Response is "net.inet.ip.forwarding: 0 -> 1". I'll test to see if this changes things in the next day or so. Given that its default was 0, it would seem that the default intention of the the OpenVPN server service is to provide LAN access only and not a complete VPN tunnel.

It temporarily turns on packet forwarding, making your NAS into a router.
Focusing on the temporarily here, should this prove to be the solution (or at least in part), is it possible to set this permanently?

Thank you again - I'll be in touch after some more trial and error :)
 

HungrySkelly

Dabbler
Joined
Nov 20, 2020
Messages
11
well well well,
Super early report-back here, but it seems you were right on the money with the "sysctl -w net.inet.ip.forwarding=1" command!!!

For anyone else who is facing the same issue, the final solution involved the following three steps:
* executing "sysctl -w net.inet.ip.forwarding=1" via the TrueNAS CLI
* adding "push "redirect-gateway autolocal"" to the 'Additional Parameters' section of the Server Service
* the two routing rules per the screenshot in my earlier post

Thank you so much @jgreco - absolute legend :grin:

If there is a permanent way to perform the "sysctl -w net.inet.ip.forwarding=1", I'd love to hear how.

Thank you again!!
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
I think my concern here would be that I feel like we might be fighting against someone else's design intention here, in which case that's bad, but it could also be that whoever implemented the service isn't experienced with routing, figured all they needed to do with OpenVPN was to install it, did so, it seemed to work in their trite test environment, and then they called it a day. It is an easy mistake to make.

In the former case, I would rather understand what the design intent was and not paddle upstream, just in case there might be other changes that were intended to handle things like this. For example, there are capabilities within ipfw and ng that could potentially be used to implement or mimic forwarding, or there could be a tickbox somewhere in the GUI to enable forwarding that you did not notice.

In the latter case, a bug report and waiting for a fix might be a better idea, using the CLI tweak as an interim fix. I would suggest that you could use the "Report a Bug" link in the topbar to see if you can get any feedback from iX developers. If so, probably include a link to this thread in the bug report, and I suggest you post a link to the Jira issue here for the benefit of future readers.

Actually a bug report gets you the appropriate feedback in both cases. I'm a bit slow this morning. Sorry.

I can't think of a case where adding net.inet.ip.forwarding=1 as a system tunable would be damaging, but by itself it doesn't feel like the right fix for the long term, which is why I'm discouraging that course of action.
 

HungrySkelly

Dabbler
Joined
Nov 20, 2020
Messages
11
Thank you @jgreco

it could also be that whoever implemented the service isn't experienced with routing, figured all they needed to do with OpenVPN was to install it, did so, it seemed to work in their trite test environment, and then they called it a day.
I feel like this is the most likely case :wink:

include a link to this thread in the bug report, and I suggest you post a link to the Jira issue here for the benefit of future readers.
I have lodged a ticket here: https://jira.ixsystems.com/browse/NAS-108393

I can't think of a case where adding net.inet.ip.forwarding=1 as a system tunable would be damaging, but by itself it doesn't feel like the right fix for the long term, which is why I'm discouraging that course of action.
I acknowledge your thoughts on this (and tend to agree!). In the short term, I've just set up an automation in home assistant to execute this command for me daily. Once I receive further feedback from the iX team I'll adjust this hacky solution as necessary.
 
Joined
Nov 14, 2015
Messages
7
Hi all,

I've set up a VPN server using the shiny new OpenVPN Service. I have successfully got the VPN up and running and have successfully connected via the OpenVPN iOS app on my phone.

My issue is that it doesn't seem to force all traffic through the VPN.
I'm able to communicate through the VPN by adding ```push "route 192.168.1.0 255.255.255.0"``` to the Additional Parameters, however I note that only local IP requests are routed this way.
I tried adding ```push "redirect-gateway def1"``` to the Additional Parameters, however this just locked up the VPN and stopped it from working.

Is there any particular tweaks I need to make to force all traffic through the VPN?
I'm wondering if perhaps I need to do something via my router (ASUS RT-AC68U running Merlin) instead of the VPN setup?

Thanks in advance for your help and guidance :)

Skelly
Hi @HungrySkelly ,
I think I did pretty much what you did. I upgraded my FreeNAS to TrueNAS 2.0. I set up OpenVPN Server as a service and the client connects successfully on my first try. But I encountered two problems:
1) After adding 'push "route 10.1.10.0 255.255.255.0"' to additional rules (10.1.10.0 is my local network), the client still connects, but I cannot access any of my local hosts (i.e. any of the 10.1.10.* hosts except 10.1.10.41, which is the TrueNAS host)
2) I also added 'push "redirect-gateway autolocal" as per your post and manually ran 'sysctl -w net.inet.ip.forwarding=1' on the TrueNAS host. But I cannot get to the Internet from my OpenVPN client laptop. Did you have to do anything on the ipfw rules on the OpenVPN host?
Thanks in advance for your help!
 
Joined
Nov 14, 2015
Messages
7
Hi @HungrySkelly ,
I think I did pretty much what you did. I upgraded my FreeNAS to TrueNAS 2.0. I set up OpenVPN Server as a service and the client connects successfully on my first try. But I encountered two problems:
1) After adding 'push "route 10.1.10.0 255.255.255.0"' to additional rules (10.1.10.0 is my local network), the client still connects, but I cannot access any of my local hosts (i.e. any of the 10.1.10.* hosts except 10.1.10.41, which is the TrueNAS host)
2) I also added 'push "redirect-gateway autolocal" as per your post and manually ran 'sysctl -w net.inet.ip.forwarding=1' on the TrueNAS host. But I cannot get to the Internet from my OpenVPN client laptop. Did you have to do anything on the ipfw rules on the OpenVPN host?
Thanks in advance for your help!

Just a quick update, the local LAN access is working now after adding the static route 10.9.0.0/24 via 10.1.10.41 to the router.
However, I still have no luck with forcing the traffic through VPN for Internet access.
 

HungrySkelly

Dabbler
Joined
Nov 20, 2020
Messages
11
Hi @Chris Ekcennel,

I've re-checked my notes and can confirm that all the steps I took are listed in the posts above...
In my case, so long as I set the push autolocal parameter, issue the ip.forwarding command, and set up the appropriate routing rules in my router - it all worked fine...

Did you have to do anything on the ipfw rules on the OpenVPN host?
Nope - didn't touch any of those...

Sorry that's probably not much help :(
 

invar

Dabbler
Joined
Jan 23, 2021
Messages
36
I did things differently and it appears to be working:

With my TrueNAS server on its own box, and a separate router for my home, here's what I did:

Forward the appropriate port from your router to your TrueNAS server IP. 1194 is the default port so I just stuck with that. My TrueNAS server's IP on my LAN is 192.168.0.2 (yes, I know, I should change it to something else that is unlikely to be used in the outside world... I will in due time.

Once I confirmed the above was working and I could connect from my phone to my VPN, I disconnected and got to the real work, namely routing everything through the VPN, which is where I think most people get the hangup.

1) From TrueNAS WebGUI, go to System->Tunables and add the following 3 tunables:
Code:
Variable: firewall_enable
Value: yes
Type: rc.conf
Description: enable firewall

Code:
Variable: gateway_enable
Value: yes
Type: rc.conf
Description: enable gateway

Code:
Variable: firewall_script
Value: /mnt/NAME_OF_POOL/ipfwrules.sh
Type: rc.conf
Description: ipfw script to execute at boot time. I put it in the pool to ensure that it carries should I have to reinstall and restore from a config.


2) Open a console/shell. Personally, I SSH in with PuTTY as root. But the WebGUI Shell will suffice.

3) input the following line by line into the shell. Change 10.8.8.0/24 to whatever you configured OpenSSH to use for VPN devices and NAME_OF_POOL to the correct location of your ipfwrules.sh that you decided above in the Tunables. "re0" is the name of my network interface. Change to whatever yours is.

Code:
cd /mnt/NAME_OF_POOL
echo ipfw -q -f flush > ipfwrules.sh
echo ipfw -q nat 1 config if re0  >> ipfwrules.sh
echo ipfw -q add nat 1 all from 10.8.8.0/24 to any out via re0  >> ipfwrules.sh
echo ipfw -q add nat 1 all from any to any in via re0  >> ipfwrules.sh
chmod 755 ipfwrules.sh


4) Under the OpenVPN server settings, put the following in additional parameters:

Code:
push "redirect-gateway def1"
push "remote-gateway vpn_server_ip"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
duplicate-cn


The last line for "duplicate-cn" is only if you want to have the same user client/certificate be used simultaneously by multiple devices. (i.e. I use the same client configuration for my laptop and my smartphone, so this option must be included. Otherwise, OpenVPN will assign the same IP address to both devices and cause a conflict and cease working correctly.)

5) Save and download the new Client Configuration, choosing the correct Certificate.

6) Restart the TrueNAS server

7) EDIT the client configuration file with a text editor and change the line that starts with "remote" to reflect the correct address whether it is a static IP address or the Dynamic DNS you have setup. I use dynamic DNS so mine looks like so:

Code:
remote "mydomain.privatedns.org"


8) Test connect. Hopefully it works for you!

Note how I did NOT make any configuration changes to my router other than the port forwarding. I do NOT have any static routes setup anywhere, and I believe this should work fine as the TrueNAS server is performing NAT for anything originating to and going back to a VPN device.

Good luck!
 
Joined
Nov 14, 2015
Messages
7
@invar, I already gave up on it long ago. Then I saw your post today. Your solution works perfectly for my TrueNAS 12 setup with OpenVPN running as a service. Thanks you! You saved the day!
 

fre_

Cadet
Joined
May 10, 2021
Messages
4
I did things differently and it appears to be working:
...

Thank you very much for your contribution, Invar. It works like a charm.

From a Qnap-user that is in the process of probably migrating to Truenas.
 

kagstrom2100

Cadet
Joined
Jun 13, 2021
Messages
1
I did things differently and it appears to be working:

With my TrueNAS server on its own box, and a separate router for my home, here's what I did:

Forward the appropriate port from your router to your TrueNAS server IP. 1194 is the default port so I just stuck with that. My TrueNAS server's IP on my LAN is 192.168.0.2 (yes, I know, I should change it to something else that is unlikely to be used in the outside world... I will in due time.

Once I confirmed the above was working and I could connect from my phone to my VPN, I disconnected and got to the real work, namely routing everything through the VPN, which is where I think most people get the hangup.

1) From TrueNAS WebGUI, go to System->Tunables and add the following 3 tunables:
Code:
Variable: firewall_enable
Value: yes
Type: rc.conf
Description: enable firewall

Code:
Variable: gateway_enable
Value: yes
Type: rc.conf
Description: enable gateway

Code:
Variable: firewall_script
Value: /mnt/NAME_OF_POOL/ipfwrules.sh
Type: rc.conf
Description: ipfw script to execute at boot time. I put it in the pool to ensure that it carries should I have to reinstall and restore from a config.


2) Open a console/shell. Personally, I SSH in with PuTTY as root. But the WebGUI Shell will suffice.

3) input the following line by line into the shell. Change 10.8.8.0/24 to whatever you configured OpenSSH to use for VPN devices and NAME_OF_POOL to the correct location of your ipfwrules.sh that you decided above in the Tunables. "re0" is the name of my network interface. Change to whatever yours is.

Code:
cd /mnt/NAME_OF_POOL
echo ipfw -q -f flush > ipfwrules.sh
echo ipfw -q nat 1 config if re0  >> ipfwrules.sh
echo ipfw -q add nat 1 all from 10.8.8.0/24 to any out via re0  >> ipfwrules.sh
echo ipfw -q add nat 1 all from any to any in via re0  >> ipfwrules.sh
chmod 755 ipfwrules.sh


4) Under the OpenVPN server settings, put the following in additional parameters:

Code:
push "redirect-gateway def1"
push "remote-gateway vpn_server_ip"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
duplicate-cn


The last line for "duplicate-cn" is only if you want to have the same user client/certificate be used simultaneously by multiple devices. (i.e. I use the same client configuration for my laptop and my smartphone, so this option must be included. Otherwise, OpenVPN will assign the same IP address to both devices and cause a conflict and cease working correctly.)

5) Save and download the new Client Configuration, choosing the correct Certificate.

6) Restart the TrueNAS server

7) EDIT the client configuration file with a text editor and change the line that starts with "remote" to reflect the correct address whether it is a static IP address or the Dynamic DNS you have setup. I use dynamic DNS so mine looks like so:

Code:
remote "mydomain.privatedns.org"


8) Test connect. Hopefully it works for you!

Note how I did NOT make any configuration changes to my router other than the port forwarding. I do NOT have any static routes setup anywhere, and I believe this should work fine as the TrueNAS server is performing NAT for anything originating to and going back to a VPN device.

Good luck!

Just had to make an account to thank you for the excellent guide!
 

StanTheMa

Dabbler
Joined
Jun 30, 2020
Messages
13
Thanks Invar this is looking good.
Also some bullets for anyone struggling with OpenVPN - get the common name settings right eg server clienta clientb ..., and for some OpenVPN clients eg android there are limitations on which ciphers the TrueNAS OpenVPN service should be set to (I have AES-128-GCM working anything OFB didnt work (output feedback block cipher) ...
 
Top