SOLVED Openvpn tun interface issues in iocage

JohnnyGrey · Apr 23, 2019

WookieCookie said:
Ran into this same problem the other day and after some searching found this bug report https://ftp.freenas.org/issues/40872#note-44

-TLDR

You are a fricken life-saver! Pretty much every guide I've seen is now obsolete since the move to iocage. There are a couple newer ones, but they don't quite work well with PIA (Private Internet Access). I decided to try and tackle creating the jail completely manually, with some Google help, of course. I am a Linux command/script/CLI noob.

I too was running into the issue of 256 tun devices being created along with a fatal error. I removed the init task from the UI, issued your command, rebooted, and openvpn was already successfully running. I performed a

Code:

wget http://ipinfo.io/ip -qO -

, which I got from a previous guide, before and after stopping openvpn, and it works!

This may have been added to the UI, but I created the iocage jail via the "simple" interface rather than the advanced options.

Now all that's left is to convert my auto-reconnect script to iocage.

FreeNAS-11.2-U3

Derkades · Apr 23, 2019

JohnnyGrey said:
This may have been added to the UI, but I created the iocage jail via the "simple" interface rather than the advanced options.

There is an "Allow tun" option in the GUI if you choose advanced options (which I always do anyways to turn on "start on boot"). Just enable that and it should work

silverback · Apr 23, 2019

Derkades said:
There is an "Allow tun" option in the GUI if you choose advanced options (which I always do anyways to turn on "start on boot"). Just enable that and it should work

Running 11.2 u3 on a standard Supermicro e5 system I still need the devfs pre-init and command, along with the allow tunnel checkbox while building the vpn jail. Curious

JohnnyGrey · Apr 28, 2019

Derkades said:
There is an "Allow tun" option in the GUI if you choose advanced options (which I always do anyways to turn on "start on boot"). Just enable that and it should work

Appreciate it. I had a feeling there was a UI option for it. Most of what I saw in the advanced mode I didn't understand, so I opted against it. That auto start option would have been handy though :)

@silverback , I don't know man, I was able to remove the pre-init command. As soon as I issued the "allow_tun=1 transmission" everything started working.

catnas · Aug 16, 2019

Disregard. I'm stupid.

Minty Waffle · Jan 21, 2020

I know this is an old thread and it's been marked as solved, but I ran into this issue today.
After reading through everyone's lovely responses, I found a setting in the GUI marked "allow_tun" in jails/edit/custom properties. Maybe that's an option that was added recently and is of no use to the original poster, but it resolved the problem for me.
You know, just in case anyone else is still bumping into this in 2020.

Chwaee · Jan 26, 2020

I'm running 11.2-U7
I've followed mostly the same as Minty Waffle, but doing only that didn't work.

Jails -> Edit -> Custom Props -> "allow_tun" # Checkmark the box (at the bottom)
Jails -> Edit -> Jail Props -> "allow_raw_sockets" # Checkmark this box (3rd box down towards the bottom).... Not sure if this is required
REBOOT host

To check: open a shell in the jail, run ifconfig, look for a tun0 device, then tail /var/log/messages

PhiloEpisteme · Feb 4, 2020

Hi folks,

Hoping someone can help me figure out what I missed here. I have tried the steps outlined by @Chwaee in pose #47 and @WookieCookie's suggestion in post #34 to no avail.

I am running FreeNAS 11.2-U7 and trying to use OpenVPN as a client within a Jail for transmisison
The jail is running 11.2-RELEASE-p15.
From within the jail without trying to run OpenVPN I get the following output

Code:

root@transmission:~ # ifconfig
cxl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:08:43:33:48:e0
    hwaddr 00:08:43:33:48:e0
    media: Ethernet 10Gbase-SR <full-duplex,rxpause,txpause>
    status: active
cxl1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:08:43:33:48:e8
    hwaddr 00:08:43:33:48:e8
    media: Ethernet none
    status: no carrier
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether an:1f:7b:85:5b:2c
    hwaddr af:1f:7b:85:5b:2c
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
igb1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether an:1f:7b:85:5b:2d
    hwaddr ac:1f:7b:85:5b:2d
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 03:f6:5f:3b:a6:00
    inet 192.168.1.251 netmask 0xffffff00 broadcast 192.168.1.255
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 3 priority 128 path cost 55
    member: igb1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 4 priority 128 path cost 55

My config file for OpenVPN is as follows.

Code:

dev tun
fast-io
persist-key
persist-tun
nobind
remote <expressvpn server> <port>

remote-random
pull
comp-lzo no
tls-client
verify-x509-name Server name-prefix
ns-cert-type server
key-direction 1
route-method exe
route-delay 2
tun-mtu 1500
fragment 1300
mssfix 1200
verb 3
cipher AES-256-CBC
keysize 256
auth SHA512
sndbuf 524288
rcvbuf 524288
auth-user-pass /usr/local/etc/openvpn/credentials.txt

<cert>
-----BEGIN CERTIFICATE-----
. . .
-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN RSA PRIVATE KEY-----
. . .
-----END RSA PRIVATE KEY-----
</key>
<tls-auth>
#
# 2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----
. . .
-----END OpenVPN Static key V1-----
</tls-auth>
<ca>
-----BEGIN CERTIFICATE-----
. . .
-----END CERTIFICATE-----
</ca>

My best attempt at putting together the logs are as follows

Code:

Feb  5 04:19:11 transmission openvpn[28046]: WARNING: --keysize is DEPRECATED and will be removed in OpenVPN 2.6
Feb  5 04:19:11 transmission openvpn[28046]: OpenVPN 2.4.8 amd64-portbld-freebsd11.3 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Jan 31 2020
Feb  5 04:19:11 transmission openvpn[28046]: library versions: OpenSSL 1.0.2o-freebsd  27 Mar 2018, LZO 2.10
Feb  5 04:19:11 transmission openvpn[28047]: WARNING: --ns-cert-type is DEPRECATED.  Use --remote-cert-tls instead.
Feb  5 04:19:11 transmission openvpn[28047]: Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
Feb  5 04:19:11 transmission openvpn[28047]: Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
Feb  5 04:19:11 transmission openvpn[28047]: TCP/UDP: Preserving recently used remote address: [AF_INET]<IP>:<PORT>
Feb  5 04:19:11 transmission openvpn[28047]: Socket Buffers: R=[42080->524288] S=[9216->524288]
Feb  5 04:19:11 transmission openvpn[28047]: UDP link local: (not bound)
Feb  5 04:19:11 transmission openvpn[28047]: UDP link remote: [AF_INET]<IP>:<PORT>
Feb  5 04:19:11 transmission openvpn[28047]: TLS: Initial packet from [AF_INET]<IP>:<PORT>, sid=5d1566fe 48ac47c6
Feb  5 04:19:11 transmission openvpn[28047]: WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Feb  5 04:19:11 transmission openvpn[28047]: VERIFY OK: depth=1, C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=ExpressVPN CA, emailAddress=support@expressvpn.com
Feb  5 04:19:11 transmission openvpn[28047]: VERIFY OK: nsCertType=SERVER
Feb  5 04:19:11 transmission openvpn[28047]: VERIFY X509NAME OK: C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=Server-4114-2a, emailAddress=support@expressvpn.com
Feb  5 04:19:11 transmission openvpn[28047]: VERIFY OK: depth=0, C=VG, ST=BVI, O=ExpressVPN, OU=ExpressVPN, CN=Server-4114-2a, emailAddress=support@expressvpn.com
Feb  5 04:19:11 transmission openvpn[28047]: Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Feb  5 04:19:11 transmission openvpn[28047]: [Server-4114-2a] Peer Connection Initiated with [AF_INET]104.37.31.182:1195
Feb  5 04:19:12 transmission openvpn[28047]: SENT CONTROL [Server-4114-2a]: 'PUSH_REQUEST' (status=1)
Feb  5 04:19:12 transmission openvpn[28047]: PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 10.27.0.1,comp-lzo no,route 10.27.0.1,topology net30,ping 10,ping-restart 60,ifconfig 10.27.0.30 10.27.0.29,peer-id 5,cipher AES-256-GCM'
Feb  5 04:19:12 transmission openvpn[28047]: OPTIONS IMPORT: timers and/or timeouts modified
Feb  5 04:19:12 transmission openvpn[28047]: OPTIONS IMPORT: compression parms modified
Feb  5 04:19:12 transmission openvpn[28047]: OPTIONS IMPORT: --ifconfig/up options modified
Feb  5 04:19:12 transmission openvpn[28047]: OPTIONS IMPORT: route options modified
Feb  5 04:19:12 transmission openvpn[28047]: OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
Feb  5 04:19:12 transmission openvpn[28047]: OPTIONS IMPORT: peer-id set
Feb  5 04:19:12 transmission openvpn[28047]: OPTIONS IMPORT: adjusting link_mtu to 1629
Feb  5 04:19:12 transmission openvpn[28047]: OPTIONS IMPORT: data channel crypto options modified
Feb  5 04:19:12 transmission openvpn[28047]: Data Channel: using negotiated cipher 'AES-256-GCM'
Feb  5 04:19:12 transmission openvpn[28047]: NCP: overriding user-set keysize with default
Feb  5 04:19:12 transmission openvpn[28047]: Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Feb  5 04:19:12 transmission openvpn[28047]: Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Feb  5 04:19:12 transmission openvpn[28047]: GDG: problem writing to routing socket
Feb  5 04:19:12 transmission openvpn[28047]: ROUTE: default_gateway=UNDEF
Feb  5 04:19:12 transmission openvpn[28047]: Cannot allocate TUN/TAP dev dynamically
Feb  5 04:19:12 transmission openvpn[28047]: Exiting due to fatal error
Feb  5 04:19:44 transmission openvpn[28381]: WARNING: --keysize is DEPRECATED and will be removed in OpenVPN 2.6
Feb  5 04:19:44 transmission openvpn[28381]: OpenVPN 2.4.8 amd64-portbld-freebsd11.3 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Jan 31 2020
Feb  5 04:19:44 transmission openvpn[28381]: library versions: OpenSSL 1.0.2o-freebsd  27 Mar 2018, LZO 2.10
Feb  5 04:19:44 transmission openvpn[28382]: WARNING: --ns-cert-type is DEPRECATED.  Use --remote-cert-tls instead.

I think of particular note are the following messages

Code:

Feb  5 04:19:12 transmission openvpn[28047]: GDG: problem writing to routing socket
Feb  5 04:19:12 transmission openvpn[28047]: ROUTE: default_gateway=UNDEF
Feb  5 04:19:12 transmission openvpn[28047]: Cannot allocate TUN/TAP dev dynamically
Feb  5 04:19:12 transmission openvpn[28047]: Exiting due to fatal error

One further complication is that my connection to/through my server looks something like this. switch->freenas1.igb0->freenas1.igb1 via bridge0->freenas1.igb0. So, in the jail properties the network device being used is not igb0 but is bridge0. The jail has access to the web though so the only thing which is not working is OpenVPN.

Any clarification as to what I've skipped or what information I should provide would be very helpful. Thanks.

Edit: After doing even more work (many hours today so far) I discovered this great post which lead me to trying to configure a vnet for this jail. I made the following changes to my jail config.
Basic Properties->VNET->Checked
Basic Properties->IPv4 Interface->vnet0
Basic Properties->IPv4 Default Router-><ip of the nearest L3 router>
Network Properties->vnet_default_interface->bridge0
I restarted the jail, ran ifconfig tun create and then service openvpn start.

The above seems to work, for now. I am doing further testing and if I run into other issues will likely post the question in another thread and link.

TonyMEdKniven · May 5, 2020

FreeNAS-11.3-U2.1: update this thread helped me find this out.

In the gui config for the jail:
Jail Properties
devfs_ruleset check that it contains the unhide tun rule, if not change to ruleset that contains it. also the ? on this one say:
allow_mount - check this one
allow_mount_devfs - check this one

allow_tun - was checked but did not matter until I changed the above 3 settings

gui devfs_ruleset said 4, but when starting the iocage it was 6, changed the gui setting to 6 just to be sure since it contained unhide tun.
The ? told med to do the other allow_'s. And openvpn started working.

Someone should create a openvpn plugin. This is ridiculous :-D

WookieCookie · May 5, 2020

TonyMEdKniven said:
FreeNAS-11.3-U2.1: update this thread helped me find this out.

In the gui config for the jail:
Jail Properties
devfs_ruleset check that it contains the unhide tun rule, if not change to ruleset that contains it. also the ? on this one say:
allow_mount - check this one
allow_mount_devfs - check this one

allow_tun - was checked but did not matter until I changed the above 3 settings

gui devfs_ruleset said 4, but when starting the iocage it was 6, changed the gui setting to 6 just to be sure since it contained unhide tun.
The ? told med to do the other allow_'s. And openvpn started working.

Someone should create a openvpn plugin. This is ridiculous :-D

Myself and more than a few others have moved our vpn's to the edge of the network (the firewall itself). Running a custom built firewall with pfSense is about 1000x easier to manage than a jail sitting in FreeNAS.

TonyMEdKniven · May 7, 2020

restarting... I think it's wort of wrong to install vpn in the transmission plugin jail. I'd go as far at to say sickchill/couchpotato/transmission plugins are not to be used at all for a few simple reasons such at do not mess around much in a plugin for obvious update reasons, and the plugins run with diffrent gid/uid . ie. not made to work with each other.

Setting all of this up from base jails, increases understanding on whats happening and increases future update success, and most importantly predecided uid/gid.

I still think someone should make a "openvpn" plugin, with a nice config gui. Allowing routing of incomming traffic out via tun. no need for any killswitching if hosts simply use that plugin's ip as default route. And an "official" plugin sort of have to manage all this troublesome tun things in jails.

Personally, first sign of vpn/tun issues when I've re-done all from base jails, I'll setup a raspberrypi as vpn gateway.

@WookieCookie without new firware in my asus I can only vpn everything, no source routing. After using pfsense for 2 years I'm not happy using it again. Main reason being there is no log or any information on how much latency all features of pfsense adds. Example, turn on intense snort rules and geo-fw rules, run a vanilla minecraft server, remove snort rules until it allows you to play, experience the lag, remove snort/geoip rules and lag is gone. Nowhere in pfsense can you detect any issues, issues are caused by pfsense adding latency. I got annoyed since it was running on way to powerful hardware where cpu's didn't go above ~4% ie. not hardware related. So it's out the door along with all it's cool features :-D

WookieCookie · May 7, 2020

@TonyMEdKniven Adding IDS and IPS can cause latency. It's more of an art than a science when it comes to engineering for a good balance between safety and latency. To each their own when it comes to solutions, hundreds different flavors and millions of different ways to get the same outcome.

Anyone can create plugins ;-)

13. Plugins — FreeNAS®11.2-U3 User Guide Table of Contents

www.ixsystems.com

vinchi007 · Aug 10, 2021

Just upgraded from 11.2 to 12 and having issues with my transmission jail which includes openvpn client.
By default migrated jail (9.3) does not have allow_tun enabled so upon startup it spawned 255 tunnels. After enabling allow_tun in GUI openvpn wont start. Above mentioned settings applied. Also, in desperation upgraded jail to release 12 (same as host) and now jail reports .so lib errors (pkg, openvpn etc).
At this point I have no other choice (wasted so many hours) on these jails (plex, openvpn, transmission etc) - now will have to recreate them all from scratch.

WookieCookie · Aug 10, 2021

@vinchi007 - With all of the issues I've had in the past I've moved to storing the config on the main storage blob and mapping it to the jail so I can burn it down and build it back up quickly. Below is an example of my script to build a new Plex server and update it nightly using a cron job without worrying about the watch status and users.

Code:

# Install plexmediaserver-plexpass
# iocage console plex
pkg update && pkg upgrade -y
pkg install nano
mkdir -p /usr/local/etc/pkg/repos/
nano /usr/local/etc/pkg/repos/FreeBSD.conf
FreeBSD: {
    url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest"
}
pkg update && pkg upgrade -y
pkg install plexmediaserver-plexpass


# Exit plex jail
iocage exec plex mkdir -p /mnt/plexMD/
iocage exec plex mkdir -p /mnt/media/movies
iocage exec plex mkdir -p /mnt/media/music
iocage exec plex mkdir -p /mnt/media/tv
iocage exec plex mkdir -p /mnt/media/tvrecorded
iocage fstab -a plex /mnt/cargoBay/apps/plexMD /mnt/plexMD nullfs rw 0 0
iocage fstab -a plex /mnt/cargoBay/media/movies /mnt/media/movies nullfs ro 0 0
iocage fstab -a plex /mnt/cargoBay/media/music /mnt/media/music nullfs ro 0 0
iocage fstab -a plex /mnt/cargoBay/media/tv /mnt/media/tv nullfs ro 0 0
iocage fstab -a plex /mnt/cargoBay/media/tvrecorded /mnt/media/tvrecorded nullfs ro 0 0
iocage exec plex chown -R plex:plex /mnt/plexMD
iocage exec plex sysrc plexmediaserver_plexpass_support_path="/mnt/plexMD"
iocage exec plex sysrc plexmediaserver_plexpass_enable="YES"
iocage exec plex service plexmediaserver_plexpass start


# Automate plex updates
# iocage console plex
pkg install wget
pkg install ca_root_nss
pkg install perl5
mkdir /usr/local/PMS_Updater
cd /usr/local/PMS_Updater
wget https://raw.githubusercontent.com/mstinaff/PMS_Updater/master/PMS_Updater.sh
sh PMS_Updater.sh -v -a

# In Tasks > Cron Jobs on Truenas
/usr/local/bin/iocage exec plex /bin/sh /usr/local/PMS_Updater/PMS_Updater.sh -v -a


# Update plexmediaserver-plexpass manually
service plexmediaserver_plexpass stop
pkg update && pkg upgrade -y
service plexmediaserver_plexpass start

Important Announcement for the TrueNAS Community.

SOLVED Openvpn tun interface issues in iocage

JohnnyGrey

Dabbler

Derkades

Dabbler

silverback

Contributor

JohnnyGrey

Dabbler

catnas

Explorer

Minty Waffle

Cadet

Chwaee

Cadet

PhiloEpisteme

Guru

TonyMEdKniven

Cadet

WookieCookie

Dabbler

TonyMEdKniven

Cadet

WookieCookie

Dabbler

13. Plugins — FreeNAS®11.2-U3 User Guide Table of Contents

vinchi007

Dabbler

WookieCookie

Dabbler

Similar threads

Important Announcement for the TrueNAS Community.

SOLVED Openvpn tun interface issues in iocage

Dabbler

Dabbler

Contributor

Dabbler

Explorer

Cadet

Cadet

Guru

Cadet

Dabbler

Cadet

Dabbler

Dabbler

Dabbler

Important Announcement for the TrueNAS Community.

Related topics on forums.truenas.com for thread: "Openvpn tun interface issues in iocage"

Similar threads