Jail and ZeroTier: "unable to configure virtual network port: cannot find /dev node"

Joined
Jan 25, 2020
Messages
7
Hi everyone,
I'm new here and new to Freenas.
I'm having troubles setting up a ZeroTier network in a jail.
I can join the network but I'm unable to ping the jail.
If I run "zerotier-one -d" I get this:

"ERROR: unable to configure virtual network port: cannot find /dev node for newly created tap device"

This happens only inside the jail, the same command on the main host runs smoothly.

If I try to list the networks with "zerotier-cli listnetworks" I get a PORT_ERROR as status.

Output of ifconfig inside the jail:

"
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 40:61:86:95:80:c7
hwaddr 02:8e:10:00:05:0b
inet 192.168.0.20 netmask 0xffffff00 broadcast 192.168.0.255
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
tun0: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
nd6 options=1<PERFORMNUD>
groups: tun
tap9993: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 00:bd:b7:19:f8:09
hwaddr 00:bd:b7:19:f8:09
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: no carrier
groups: tap
"

If you need any other information just ask!

Kind regards,
Lorenzo Benevento.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Zerotier isn't installed in jails, so that explains your output.

The right way to do it is to create a bridge on the host and add the zerotier interface to it, then any jail vnet interface you want to work with zerotier to that same bridge.
 
Joined
Jan 25, 2020
Messages
7
Okay perfect, thank you so much, I'm going to try the right way! Just for curiosity, why isn't installing ZeroTier inside the jail itself possible? Why can't I just run a "pkg install zerotier" inside the jail?
 
Joined
Jan 25, 2020
Messages
7
Update: I added both the zerotier interface (after changing the MTU value to match the one of the bridge) and the jail to bridge0. I'm almost sure I'm doing something wrongly but I have no idea what. I still can't ping anything on the zerotier network from within the jail! What am I missing?

Host ifconfig:
"
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=82099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether 40:61:86:c3:e1:ea
hwaddr 40:61:86:c3:e1:ea
inet 192.168.0.18 netmask 0xffffff00 broadcast 192.168.0.255
inet6 fe80::4261:86ff:fec3:e1ea%re0 prefixlen 64 scopeid 0x1
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:25:85:7c:4b:00
nd6 options=1<PERFORMNUD>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: vnet0:3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000
member: zt8gk1jge88tii4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000000
member: vnet0:2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000
member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
vnet0:2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: Plex as nic: epair0b
options=8<VLAN_MTU>
ether 40:61:86:31:41:ea
hwaddr 02:8e:10:00:05:0a
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
zt8gk1jge88tii4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 5000 mtu 1500
options=80000<LINKSTATE>
ether 46:37:4c:a6:23:e5
hwaddr 00:bd:e8:98:22:09
inet 192.168.192.94 netmask 0xffffff00 broadcast 192.168.192.255
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 6826
vnet0:3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: Minecraft_Server as nic: epair0b
options=8<VLAN_MTU>
ether 40:61:86:95:80:c6
hwaddr 02:8e:10:00:04:0a
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair"

Jail ifconfig:
"
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 40:61:86:95:80:c7
hwaddr 02:8e:10:00:07:0b
inet 192.168.0.20 netmask 0xffffff00 broadcast 192.168.0.255
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
tun0: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
nd6 options=1<PERFORMNUD>
groups: tun"
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
You have a lot going on there... a tunnel and a tap interface in addition to what you're doing with the zerotier bridged link...

The gateway setting of the jail might be important as you will need to consider that your zerotier network may not be on the same subnet as the rest of your environment and hence will need to route.

It will depend on what you intend the use of the jail and zerotier to be... do you expect all traffic in that jail to go out through zerotier? (you can set the gateway to your zerotier network/router in that case).
 
Joined
Jan 25, 2020
Messages
7
I mean, it would not be a problem if everything goes through ZeroTier, but I'd rather being able to connect to that jail also on my local subnet!
I'm going to try and replace the gateway of the jail with the one of ZeroTier and see if that works and maybe I'll be able to have some links.
About the tunnel and tap interface, I think they could be residue of my previous attempts to install ZeroTier inside the jail itself. Should I remove them?
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
About the tunnel and tap interface, I think they could be residue of my previous attempts to install ZeroTier inside the jail itself. Should I remove them?
If not needed, they should go.

I'd rather being able to connect to that jail also on my local subnet!
You may be able to play with routing to get what you want... or you can change your zerotier address scheme to extend your LAN subnet, just watch out about where DHCP will come from (you will probably need to manually assign your zerotier devices addresses from your local subnet as DHCP will not pass across, but you shouldn't run DHCP on both sides unless you know what you're doing).
 
Joined
Jan 25, 2020
Messages
7
I'll try to do the fine tuning later, for now I'd just like to be able to get some communication across the jail and the other device on the ZeroTier network... Isn't my bridge correct? If yes, do you see what I'm doing wrong?
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Your bridge looks to contain the 2 key interfaces (although I can't tell from that with of the vnets it should be, you seem to have two of them there, so OK assuming nothing is missed).

Your zerotier interface has an IP address of 192.168.192.94

The rest of your kit is on 192.168.0.0/24... a different subnet.

Bridges don't route, so you have no path at the IP address level to reach the zerotier network.

If your zerotier interface was on 192.168.0.0/24, then you should be able to talk to it (or other devices also connected to it and addressed as 192.168.0.0/24).

Because it's not like that, you either need to add a route from your FreeNAS or jail to find the 192.168.192.0/24 network via the zerotier interface.

To help yopu see what's happening a bit you can run netstat -r in the jail and on the host, which will probably show you that there's a route for 192.168.0.0/24, but the only other sensible option is default... which would be to your internet gateway... which will not get you to the zerotier network/interface.
 
Last edited:
Joined
Jan 25, 2020
Messages
7
Okay, I'm back with more question!
First of all I want to thanks you so much for the time you're spending here!

I ran some traceroute on the jail and the windows machine. It looks like from both the host and my windows machine it goes straight to the other machines! I can see no gateway, nothing. It just go there, while on the jail it is handled like it was a normal ip, so it go over to my gateway, then some stuff from my isp I guess and from there there is no route. Now, if I got this right, I should tell freenas to route the traffic on the zerotier ip through something else like the zerotier gateway. But in order to do that I think I should redirect the traffic on the zerotier interface, which I don't have inside the jail. What should be the right "route add" command for this?

I'm new to networking so sorry about the stupid questions I'm doing here!
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Joined
Jan 25, 2020
Messages
7
No problem, I'm going to try an learn how to get wireguard to work. If I'll have troubles (and I will) maybe it is better if I make another thread isn't it?
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
New thread for a new product...

I'm only new to that one myself, but will help out if I can.
 
Top