Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.
Resource icon

How-To: Setup a Wireguard VPN Server in a Jail

SamPool

Neophyte
Joined
Jan 15, 2021
Messages
7
I may have spoken too soon.. So I restarted the Client and it looks like the Handshake times out..

Log from the client applicaton:
Code:
2021-01-15 17:39:01.370899: [APP] App version: 1.0.12 (22)
2021-01-15 17:39:06.057708: [APP] startActivation: Entering (tunnel: test3)
2021-01-15 17:39:06.061245: [APP] startActivation: Tunnel is disabled. Re-enabling and saving
2021-01-15 17:39:06.093018: [APP] startActivation: Tunnel saved after re-enabling, invoking startActivation
2021-01-15 17:39:06.093144: [APP] startActivation: Entering (tunnel: test3)
2021-01-15 17:39:06.094584: [APP] startActivation: Starting tunnel
2021-01-15 17:39:06.096580: [APP] startActivation: Success
2021-01-15 17:39:06.106419: [APP] Tunnel 'test3' connection status changed to 'connecting'
2021-01-15 17:39:06.145029: [NET] App version: 1.0.12 (22)
2021-01-15 17:39:06.145137: [NET] Starting tunnel from the app
2021-01-15 17:39:06.449353: [NET] DNS64: mapped [my external IP from cut/paste] to itself.
2021-01-15 17:39:06.450127: [NET] Attaching to interface
2021-01-15 17:39:06.453009: [NET] UAPI: Updating private key
2021-01-15 17:39:06.453505: [NET] Routine: decryption worker - started
2021-01-15 17:39:06.455363: [NET] Routine: encryption worker - started
2021-01-15 17:39:06.457352: [NET] Routine: decryption worker - started
2021-01-15 17:39:06.459347: [NET] Routine: handshake worker - started
2021-01-15 17:39:06.461324: [NET] Routine: encryption worker - started
2021-01-15 17:39:06.463330: [NET] Routine: handshake worker - started
2021-01-15 17:39:06.465330: [NET] Routine: handshake worker - started
2021-01-15 17:39:06.467355: [NET] Routine: decryption worker - started
2021-01-15 17:39:06.469606: [NET] Routine: encryption worker - started
2021-01-15 17:39:06.471336: [NET] Routine: TUN reader - started
2021-01-15 17:39:06.473387: [NET] Routine: event worker - started
2021-01-15 17:39:06.475452: [NET] UAPI: Removing all peers
2021-01-15 17:39:06.477352: [NET] UAPI: Transition to peer configuration
2021-01-15 17:39:06.480285: [NET] peer(UdS5…LWHM) - UAPI: Created
2021-01-15 17:39:06.481354: [NET] peer(UdS5…LWHM) - UAPI: Updating endpoint
2021-01-15 17:39:06.482012: [NET] peer(UdS5…LWHM) - UAPI: Updating persistent keepalive interval
2021-01-15 17:39:06.483360: [NET] peer(UdS5…LWHM) - UAPI: Removing all allowedips
2021-01-15 17:39:06.485364: [NET] peer(UdS5…LWHM) - UAPI: Adding allowedip
2021-01-15 17:39:06.487389: [NET] peer(UdS5…LWHM) - UAPI: Adding allowedip
2021-01-15 17:39:06.490089: [NET] UDP bind has been updated
2021-01-15 17:39:06.491359: [NET] peer(UdS5…LWHM) - Starting...
2021-01-15 17:39:06.493424: [NET] Routine: receive incoming IPv4 - started
2021-01-15 17:39:06.493637: [NET] Device started
2021-01-15 17:39:06.495465: [NET] Tunnel interface is utun2
2021-01-15 17:39:06.498858: [NET] Network change detected with satisfied route and interface order [pdp_ip0]
2021-01-15 17:39:06.499834: [NET] DNS64: mapped [my external IP from cut/paste] to itself.
2021-01-15 17:39:06.499945: [NET] Routine: receive incoming IPv6 - started
2021-01-15 17:39:06.500190: [APP] Tunnel 'test3' connection status changed to 'connected'
2021-01-15 17:39:06.505356: [NET] peer(UdS5…LWHM) - Routine: nonce worker - started
2021-01-15 17:39:06.505817: [NET] peer(UdS5…LWHM) - Routine: sequential sender - started
2021-01-15 17:39:06.507416: [NET] peer(UdS5…LWHM) - Routine: sequential receiver - started
2021-01-15 17:39:06.509442: [NET] UAPI: Transition to peer configuration
2021-01-15 17:39:06.511418: [NET] peer(UdS5…LWHM) - UAPI: Updating endpoint
2021-01-15 17:39:06.513680: [NET] Routine: receive incoming IPv4 - stopped
2021-01-15 17:39:06.515386: [NET] Routine: receive incoming IPv6 - stopped
2021-01-15 17:39:06.517638: [NET] UDP bind has been updated
2021-01-15 17:39:06.519651: [NET] Routine: receive incoming IPv4 - started
2021-01-15 17:39:06.521406: [NET] Routine: receive incoming IPv6 - started
2021-01-15 17:39:07.097284: [NET] Network change detected with satisfied route and interface order [pdp_ip0, utun2]
2021-01-15 17:39:07.098742: [NET] DNS64: mapped [my external IP from cut/paste] to itself.
2021-01-15 17:39:07.100664: [NET] UAPI: Transition to peer configuration
2021-01-15 17:39:07.102130: [NET] peer(UdS5…LWHM) - UAPI: Updating endpoint
2021-01-15 17:39:07.104489: [NET] Routine: receive incoming IPv4 - stopped
2021-01-15 17:39:07.106019: [NET] Routine: receive incoming IPv6 - stopped
2021-01-15 17:39:07.117883: [NET] UDP bind has been updated
2021-01-15 17:39:07.123153: [NET] Routine: receive incoming IPv4 - started
2021-01-15 17:39:07.123358: [NET] Routine: receive incoming IPv6 - started
2021-01-15 17:39:11.096424: [APP] Status update notification timeout for tunnel 'test3'. Tunnel status is now 'connected'.
2021-01-15 17:39:16.607815: [NET] peer(UdS5…LWHM) - Sending handshake initiation
2021-01-15 17:39:16.608588: [NET] peer(UdS5…LWHM) - Awaiting keypair
2021-01-15 17:39:21.612013: [NET] peer(UdS5…LWHM) - Sending handshake initiation
2021-01-15 17:39:26.809283: [NET] peer(UdS5…LWHM) - Handshake did not complete after 5 seconds, retrying (try 2)
2021-01-15 17:39:26.809847: [NET] peer(UdS5…LWHM) - Sending handshake initiation
2021-01-15 17:39:32.098586: [NET] peer(UdS5…LWHM) - Handshake did not complete after 5 seconds, retrying (try 2)
2021-01-15 17:39:32.098807: [NET] peer(UdS5…LWHM) - Sending handshake initiation
2021-01-15 17:39:37.285413: [NET] peer(UdS5…LWHM) - Handshake did not complete after 5 seconds, retrying (try 2)
2021-01-15 17:39:37.285967: [NET] peer(UdS5…LWHM) - Sending handshake initiation
2021-01-15 17:39:42.434114: [NET] peer(UdS5…LWHM) - Sending handshake initiation
2021-01-15 17:39:47.768892: [NET] peer(UdS5…LWHM) - Handshake did not complete after 5 seconds, retrying (try 2)
2021-01-15 17:39:47.769503: [NET] peer(UdS5…LWHM) - Sending handshake initiation
2021-01-15 17:39:52.857861: [NET] peer(UdS5…LWHM) - Handshake did not complete after 5 seconds, retrying (try 2)
2021-01-15 17:39:52.858367: [NET] peer(UdS5…LWHM) - Sending handshake initiation
2021-01-15 17:39:58.174875: [NET] peer(UdS5…LWHM) - Handshake did not complete after 5 seconds, retrying (try 2)
2021-01-15 17:39:58.175224: [NET] peer(UdS5…LWHM) - Sending handshake initiation


Anything I should look into?

Thanks for any help.
 

FreeVel

Member
Joined
Feb 28, 2017
Messages
27
Check that you got all the key pair values in the client & server config correct, and you haven't mixed them up accidentally; it is not difficult to happen.

(1) Check your steps around setting the configs for client & server
(2) did you check the nat settings on your wireguard jail ? does it look like step 1c ?
 
Last edited:

SamPool

Neophyte
Joined
Jan 15, 2021
Messages
7
Check that you got all the key pair values in the client & server config correct, and you haven't mixed them up accidentally; it is not difficult to happen.

(1) Check your steps around setting the configs for client & server
(2) did you check the nat settings on your wireguard jail ? does it look like step 1c ?
Ok, going to double check everything and post my steps.

Jail Setup in Jail wizard. Note : The items pointed to by Arrows are what I entered the ones in circles the jail assigned.

JailSetup1.jpg

JailSetup2.jpg

JailSetup3.jpg


Step 3: Set up wireguard & Jail networking (jail <-> wireguard)
a. Enable Wireguard iface, NAT & IP forwarding in "rc.conf"
d. ensure the following lines exist in your rc.conf
c. Create the ipfw.rules file
d. Paste the below lines into the file,

Code:
root@wireguard:~ # more /etc/rc.conf
cron_flags="$cron_flags -J 15"

# Disable Sendmail by default
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Run secure syslog
syslogd_flags="-c -ss"

# Enable IPv6
ipv6_activate_all_interfaces="YES"

# Enable Wireguard
wireguard_enable="YES"
wireguard_interfaces="wg0"

#Enable ip forwarding
gateway_enable="YES"

#Enable Firewall NAT in kernel mode
firewall_enable="YES"
firewall_nat_enable="YES"
#firewall_logging="YES" # Optional
firewall_script="/usr/local/etc/ipfw.rules"

root@wireguard:~ # more /usr/local/etc/ipfw.rules
#!/bin/sh

# ipfw config/rules
# from FBSD Handbook, rc.firewall, et. al.

# Flush all rules before we begin.
ipfw -q -f flush

# Set rules command prefix
cmd="ipfw -q add "
# Internet-facing iface
vif="epair0b"
# Used for outboud NAT rules
skip="skipto 1000"

#### WG-specific Options ####
# Listen Port
wg_port="51820"
# Subnet
wg_subnet="10.0.0.1/32"

# Wireguard interface, matching the name in /etc/wireguard/*.conf
wg_iface="wg0"

# Allow NAT
ipfw disable one_pass
ipfw -q nat 1 config if $vif same_ports unreg_only reset

# allow all for localhost
$cmd 00010 allow ip from any to any via lo0
$cmd 00011 allow ip from any to any via $wg_iface

# NAT-specifig rules
$cmd 00099 reass all from any to any in       # reassamble inbound packets
$cmd 00100 nat 1 ip from any to any in via $vif # NAT any inbound packets

# checks stateful rules.  If marked as "keep-state" the packet has
# already passed through filters and is "OK" without futher
# rule matching
$cmd 00101 check-state


# allow WG
#$cmd 00233 $skip udp from any to any src-port $wg_port out via $vif keep-state
#$cmd 00234 $skip udp from $wg_subnet to any out via $vif keep-state
#$cmd 00235 $skip tcp from $wg_subnet to any out via $vif setup keep-state

#$cmd 00320 $skip udp from any to any out via $vif keep-state
#$cmd 00325 $skip tcp from any to any out via $vif setup keep-state
#$cmd 00330 $skip icmp from any to any out via $vif keep-state

#$cmd 999 deny ip from any to any

# NAT
$cmd 1000 nat 1 ip from any to any out via $vif # skipto location for outbound stateful rules
#$cmd 1001 allow ip from any to any


e. Using in-kernel NAT requires to disable TCP segmentation offloading (TSO). Set the following in Jail's /etc/sysctl.conf
Code:
root@wireguard:~ # more /etc/sysctl.conf
# $FreeBSD: releng/12.1/sbin/sysctl/sysctl.conf 337624 2018-08-11 13:28:03Z brd $
#
#  This file is read when going to multi-user and its contents piped thru
#  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for details.
#

# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
#security.bsd.see_other_uids=0

# Using in-kernel NAT requires to disable TCP segmentation offloading (TSO).
net.inet.tcp.tso="0"


e. Restart the Jail & go get back to the console

f. Confirm firewall settings loaded. Note: The above config of ipfw rules should look like this
Code:
root@wireguard:~ # ipfw list
00010 allow ip from any to any via lo0
00011 allow ip from any to any via wg0
00099 reass ip from any to any in
00100 nat 1 ip from any to any in via epair0b
00101 check-state :default
01000 nat 1 ip from any to any out via epair0b
65535 allow ip from any to any

root@wireguard:~ # ipfw nat show config
ipfw nat 1 config if epair0b same_ports unreg_only reset
root@wireguard:~ # 


I will go over Step 4: Setup Wireguard Server & remote host configs in next post.
 

SamPool

Neophyte
Joined
Jan 15, 2021
Messages
7
Continuing...
Step 4: Setup Wireguard Server & remote host configs

a. Create public/private key pairs for wireguard server and remote host. I will hide part of the keys for obvious reasons.
Code:
root@wireguard:~ # cd /usr/local/etc/wireguard/

root@wireguard:/usr/local/etc/wireguard # more wg.public
UdS5VFjSKChD5R8XXXXXXXXXXXXXXXXXXXXlay9LWHM=

root@wireguard:/usr/local/etc/wireguard # more remote.public
lBCc12e6RWHRCI2XXXXXXXXXXXXXXXXXXXX7jnqUtk0=


b. Create Wireguard server config
c. Add the Server (interface) and remote (peer) by pasting the below config, then press Ctrl+X & "Yes"
Code:
root@wireguard:/usr/local/etc/wireguard # more wg0.conf
[Interface]
Address = 10.0.0.1/32
PrivateKey = UdS5VFjSKChD5R8XXXXXXXXXXXXXXXXXXXXlay9LWHM=
ListenPort = 51820

[Peer]
PublicKey = lBCc12e6RWHRCI2XXXXXXXXXXXXXXXXXXXX7jnqUtk0=
AllowedIPs = 10.0.0.2/32

c. Start Wireguard service
Code:
root@wireguard:/usr/local/etc/wireguard # service wireguard restart
[#] rm -f /var/run/wireguard/wg0.sock
[#] wireguard-go wg0
INFO: (wg0) 2021/01/16 12:37:17 Starting wireguard-go version 0.0.20201118
[#] wg setconf wg0 /tmp/tmp.7aQjiskB/sh-np.ZAmIcX
[#] ifconfig wg0 inet 10.0.0.1/32 10.0.0.1 alias
[#] ifconfig wg0 mtu 1420
[#] ifconfig wg0 up
[#] route -q -n add -inet 10.0.0.2/32 -interface wg0
[+] Backgrounding route monitor
root@wireguard:/usr/local/etc/wireguard #


Step 5: Setup wireguard on remote host
a. Create remote host config. I hide part of the IP.
Code:
root@wireguard:/usr/local/etc/wireguard # more remote.conf
[Interface]
Address = 10.0.0.2/32
PrivateKey = lBCc12e6RWHRCI2XXXXXXXXXXXXXXXXXXXX7jnqUtk0=
DNS = 8.8.8.8

[Peer]
PublicKey = UdS5VFjSKChD5R8XXXXXXXXXXXXXXXXXXXXlay9LWHM=
AllowedIPs = 0.0.0.0/0,::/0
Endpoint = 74.XXX.XXX.148:51820


c. Install wireguard on remote host and load the remote host config file using QR scan code (easier)
I installed on an iPad connected to cellular network.

Step 6: Setup port-forwarding on your router

a. Setup port-forwarding on your router so
<<WAN_IP>>:51820 <---> <<bge0_ip>>:51820
PortForwarding2.jpg


That is what I have and setup..
I really appreciate the time and any help.
 

FreeVel

Member
Joined
Feb 28, 2017
Messages
27
ok it seems you mixed up the keys

your server config wg.conf, shows as value of the private interface key the value in your wg.public...

Make sure key --> config files match in this way

wg.private --> wg.conf / interface private
wg.pubic --> remote.conf / peer public

remote.public --> wg.conf / peer public
remote.private --> remote.conf / interface private
 

SamPool

Neophyte
Joined
Jan 15, 2021
Messages
7
ok it seems you mixed up the keys

your server config wg.conf, shows as value of the private interface key the value in your wg.public...

Make sure key --> config files match in this way

wg.private --> wg.conf / interface private
wg.pubic --> remote.conf / peer public

remote.public --> wg.conf / peer public
remote.private --> remote.conf / interface private
o_Oo_Oo_O Stupid me o_Oo_Oo_O

So obvious. Thank you for pointing that out. I have connection now.
Thank you so much for your time and patience.
 

Glorious1

Neophyte Sage
Joined
Nov 23, 2014
Messages
1,065
Is there a similar guide to this for setting up just a Wireguard client using PIA? Can't find it.
 

volothamp

Member
Joined
Jul 28, 2019
Messages
46
I'm stuck at the beginning.

After having set the ipfw rules, it seems like they're ignored

Code:
 iocage restart Wireguard
* Stopping Wireguard
  + Executing prestop OK
  + Stopping services OK
  + Tearing down VNET OK
  + Removing devfs_ruleset: 16 OK
  + Removing jail process OK
  + Executing poststop OK
Wireguard: nat requires nat_interface, using ix0
No default gateway found for ipv6.
* Starting Wireguard
  + Started OK
  + Using devfs_ruleset: 16
  + Configuring VNET OK
  + Using IP options: vnet
  + Starting services OK
  + Executing poststart OK


devfs_ruleset: 16 instead of 6 from my understanding.

and the rules aren't read

Code:
root@Wireguard:~ # ipfw list
65535 allow ip from any to any


While if execute my rules script directly the rules are loaded

Code:
root@Wireguard:~ # /bin/sh /usr/local/etc/ipwf.rules
root@Wireguard:~ # ipfw list
00010 allow ip from any to any via lo0
00011 allow ip from any to any via wg0
00099 reass ip from any to any in
00100 nat 1 ip from any to any in via epair0b
00101 check-state :default
01000 nat 1 ip from any to any out via epair0b
65535 allow ip from any to any


In /etc/rc.conf the firewall_scripts variable is set

Code:
root@Wireguard:~ # cat /etc/rc.conf
cron_flags="$cron_flags -J 15"

# Disable Sendmail by default
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Run secure syslog
syslogd_flags="-c -ss"

# Enable IPv6
ipv6_activate_all_interfaces="YES"


# Enable Wireguard
wireguard_enable="YES"
wireguard_interfaces="wg0"

#Enable ip forwarding
gateway_enable="YES"

#Enable Firewall NAT in kernel mode
firewall_enable="YES"
firewall_nat_enable="YES"
firewall_logging="YES" # Optional
firewall_script="/usr/local/etc/ipfw.rules"


Any idea?

Thanks
 

SamPool

Neophyte
Joined
Jan 15, 2021
Messages
7
I'm stuck at the beginning.

After having set the ipfw rules, it seems like they're ignored

Code:
 iocage restart Wireguard
* Stopping Wireguard
  + Executing prestop OK
  + Stopping services OK
  + Tearing down VNET OK
  + Removing devfs_ruleset: 16 OK
  + Removing jail process OK
  + Executing poststop OK
Wireguard: nat requires nat_interface, using ix0
No default gateway found for ipv6.
* Starting Wireguard
  + Started OK
  + Using devfs_ruleset: 16
  + Configuring VNET OK
  + Using IP options: vnet
  + Starting services OK
  + Executing poststart OK


devfs_ruleset: 16 instead of 6 from my understanding.

and the rules aren't read

Code:
root@Wireguard:~ # ipfw list
65535 allow ip from any to any


While if execute my rules script directly the rules are loaded

Code:
root@Wireguard:~ # /bin/sh /usr/local/etc/ipwf.rules
root@Wireguard:~ # ipfw list
00010 allow ip from any to any via lo0
00011 allow ip from any to any via wg0
00099 reass ip from any to any in
00100 nat 1 ip from any to any in via epair0b
00101 check-state :default
01000 nat 1 ip from any to any out via epair0b
65535 allow ip from any to any


In /etc/rc.conf the firewall_scripts variable is set

Code:
root@Wireguard:~ # cat /etc/rc.conf
cron_flags="$cron_flags -J 15"

# Disable Sendmail by default
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Run secure syslog
syslogd_flags="-c -ss"

# Enable IPv6
ipv6_activate_all_interfaces="YES"


# Enable Wireguard
wireguard_enable="YES"
wireguard_interfaces="wg0"

#Enable ip forwarding
gateway_enable="YES"

#Enable Firewall NAT in kernel mode
firewall_enable="YES"
firewall_nat_enable="YES"
firewall_logging="YES" # Optional
firewall_script="/usr/local/etc/ipfw.rules"


Any idea?

Thanks
Did you do Step 3: Set up wireguard & Jail networking (jail <-> wireguard)

It should be a, b & c but it was labeled a, d, e by mistake. In the rc.config you should have options for the firewall.

Code:
#Enable Firewall NAT in kernel mode
firewall_enable="YES"
firewall_nat_enable="YES"
#firewall_logging="YES" # Optional
firewall_script="/usr/local/etc/ipfw.rules"

Also, what does this file look like, /usr/local/etc/ipwf.rules?
Can you do a more on it and post the output?
 

volothamp

Member
Joined
Jul 28, 2019
Messages
46
Did you do Step 3: Set up wireguard & Jail networking (jail <-> wireguard)


Also, what does this file look like, /usr/local/etc/ipwf.rules?
Can you do a more on it and post the output?
Ok I started from scratch and now I'm getting the correct firewall rules

Code:
root@wireguard:/usr/local/etc/wireguard # ipfw list

00010 allow ip from any to any via lo0
00011 allow ip from any to any via wg0
00099 reass ip from any to any in
00100 nat 1 ip from any to any in via epair0b
00101 check-state :default
01000 nat 1 ip from any to any out v

root@wireguard:/usr/local/etc/wireguard # ipfw nat show config

ipfw nat 1 config if epair0b same_ports unreg_only reset


Thanks for the support
 

Dan Tudora

Senior Member
Joined
Jul 6, 2017
Messages
269
hello
I do not understand never ever WHY you make some complicated setup with jail/ipfw/redirect port and etc.
wireguard work in FreeNAS/TrueNAS Core on THE host ( I just copy/paste from forum thread and read iX anoucement AND work)
in this thread in post #4 #13 AND other I explain how I make test and work including now. read just dissscusion of that resourcess (all post):smile:
TrueCommand online/cloud already use WireGuard for connect TrueNAS/TrueNAs SCALE
JUSST DO NOT UNDESTEND
I just think I must to write a ressorces about copy/paste from that resources/thread to clarity that "problem"
IS not a problem
IS A NEED of users
we can help them with correct INFO
I am IN
 
Last edited:

keboose

Member
Joined
Mar 5, 2016
Messages
91
I have a question about port forwarding through WireGuard, I figure this is the best place to ask since the ipfw rules in this guide are what I have pretty much verbatim.

I have a service running on my laptop, and it's listening on port 4525, TCP only. I want to access it from outside my LAN, so when I'm home (not using WG), I just forward port 4525 on my router to that PC and it's fine.

However, sometimes I take that laptop with me on the go, and I would still like that service to be accessible. I can connect my laptop to my phone hotspot, but my phone can't forward ports, naturally, and the mobile carrier probably blocks them anyway.

Is there an ipfw line I can add to my WG rules to forward that port from the server to the client? I have tried a few things on my own, but I am a complete dunce with IPFW and none of them worked. Basically I would like to forward the port 4525 from my router to my WG server, then ???? happens to send those packets to my client PC.
 
Top