Jails with VNET cannot access network

[Tokolozi]

So Quite a new user in the whole FreeNAS situation, but in trying to setup a jail for my plex media server I always run into the same issues described below.

Build(based on stuff I had lying around)
MoBo: MSI Z77 Mpower
CPU: i7 3770k
RAM: 32GB
NIC: built-in realtek (for the time being)
HDD's: 3x8TB in ZFS
OS: FreeNAS-11.2-U7
(Build Date: Nov 19, 2019 0:4)

Host IP: 192.168.0.40/24 (fixed)
Gateway: 192.168.0.1
DNS Set:
1.1.1.1
8.8.8.8
8.8.4.4


Issue(s)

So in trying to set up a plex media server I ran into some issues. I've been through about every thread I could find on the topic but could not find a solution that worked in my case.

Issue 1:
Set up a PMS using using the video guide on the IX systems YT channel.
- Server would launch in browser
- Could configure settings
- Could not fetch metadata
- Could not play media that needed encoding

> Read that enabling VNET solves issues described above. in came issue 2

Issue 2:
Deleted initial jail and started a new jail via GUI, with "correct" VNET settings but could not launch in browser.
So decided to start via SHH and CLI to make a new jail from scratch, thinking the GUI might have caused a glitch. With No avail:

Code:

iocage create -n "plexjail" -r 11.2-RELEASE ip4_addr="vnet0|192.168.0.41/24" defaultrouter="192.168.0.1" vnet="on" allow_raw_sockets="1" boot="on" 

After trying several times, reading several forum posts (here, on plex, FreeBSD help.... basically everywhere) I've run into a dead end with the following issues.

- Host can ping internet

Code:

root@HiveMind[~]# ping google.com
PING google.com (172.217.168.206): 56 data bytes
64 bytes from 172.217.168.206: icmp_seq=0 ttl=56 time=17.696 ms
64 bytes from 172.217.168.206: icmp_seq=1 ttl=56 time=24.759 ms
64 bytes from 172.217.168.206: icmp_seq=2 ttl=56 time=15.534 ms
64 bytes from 172.217.168.206: icmp_seq=3 ttl=56 time=14.270 ms
^C
--- google.com ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 14.270/18.065/24.759/4.054 ms

-Jails cannot ping anything, not even host
To host:

Code:

root@plexjail:/ # ping 192.168.0.40
PING 192.168.0.40 (192.168.0.40): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
^C
--- 192.168.0.40 ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss

To internet:

Code:

root@plexjail:/ # ping google.com
ping: cannot resolve google.com: Host name lookup failure

At this point it points to a wrong network interface or bridging, so the ifconfigs here:
HOST:

Code:

root@HiveMind[~]# ifconfig
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=82099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether d4:3d:7e:4c:ac:0c
        hwaddr d4:3d:7e:4c:ac:0c
        inet 192.168.0.40 netmask 0xffffff00 broadcast 192.168.0.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:3b:15:a5:81:00
        nd6 options=1<PERFORMNUD>
        groups: bridge
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: vnet0:4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
vnet0:4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: plexjail as nic: epair0b
        options=8<VLAN_MTU>
        ether 01:5d:f6:b4:ff:07
        hwaddr 02:7e:d0:00:04:0a
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        groups: epair

JAIL:

Code:

root@plexjail:/ # ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 01:5d:f6:b4:ff:08
        hwaddr 02:7e:d0:00:05:0b
        inet 192.168.0.41 netmask 0xffffff00 broadcast 192.168.0.255
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        groups: epair

maybe my gateways aren't correct I believed:

HOST

Code:

root@HiveMind[~]# cat /etc/resolv.conf
# Generated by resolvconf
search local
nameserver 1.1.1.1
nameserver 8.8.8.8
nameserver 8.8.4.4

JAIL

Code:

root@plexjail:/ # cat /etc/resolv.conf
# Generated by resolvconf
search local
nameserver 1.1.1.1
nameserver 8.8.8.8
nameserver 8.8.4.4

So I thought maybe if I enable DHCP this would solve some things.... NOPE just get an error and the jail won't start.

Long story short. I need help...
If there is anything blatantly obvious I missed please yell at me.
But I just can't seem to get this Jail with VNET (needed for plex) running on the server. Which is sad, since that's the main purpose of this build.

 

[KevDog]

Within your jail, can you post netstat -rn

And btw -- the nameservers aren't your gateways. These are dns resolvers. You check gateway with netstat -rn

 

[Tokolozi]

Within your jail, can you post netstat -rn

And btw -- the nameservers aren't your gateways. These are dns resolvers. You check gateway with netstat -rn

Hey,
thanks for the reply

Here's my netstat:

Code:

root@plexjail:/ # netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.0.1        UGS     epair0b
127.0.0.1          link#1             UH          lo0
192.168.0.0/24     link#2             U       epair0b
192.168.0.41       link#2             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#1                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#1                        U           lo0
fe80::1%lo0                       link#1                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0
root@plexjail:/ #

Additionally I was wondering if 11.3-release would resolve this... Willing to try at least. Not too much set here anyway that works. Can afford losing the "progress"

 

[KevDog]

Those maybe ok: my routing table for IPV4 is a little different (I'm using 10.0.1.1 as gateway/network). I don't know the significance of the link#1, link#2 statements:

Code:

Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.1.1           UGS       vlan1
10.0.1.0/24        link#4             U         vlan1
10.0.1.197         link#4             UHS         lo0
127.0.0.1          link#3             UH          lo0

 

[KevDog]

Could you also post

iocage get all plexjail

 

[Tokolozi]

Those maybe ok: my routing table for IPV4 is a little different (I'm using 10.0.1.1 as gateway/network). I don't know the significance of the link#1, link#2 statements:

Code:

Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.1.1           UGS       vlan1
10.0.1.0/24        link#4             U         vlan1
10.0.1.197         link#4             UHS         lo0
127.0.0.1          link#3             UH          lo0

Those were configured automatically.
Due to my ISP being a PITA I either needed double NAT or I just keep using their router as a gateway which main range I can't change.

as for the get all:

Code:

root@HiveMind[~]# iocage get all plexjail
CONFIG_VERSION:26
allow_chflags:0
allow_mlock:0
allow_mount:0
allow_mount_devfs:0
allow_mount_fusefs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:0
allow_mount_zfs:0
allow_quotas:0
allow_raw_sockets:1
allow_set_hostname:1
allow_socket_af:0
allow_sysvipc:0
allow_tun:0
allow_vmm:0
assign_localhost:0
available:readonly
basejail:0
boot:1
bpf:0
children_max:0
cloned_release:11.3-RELEASE
comment:none
compression:lz4
compressratio:readonly
coredumpsize:off
count:1
cpuset:off
cputime:off
datasize:off
dedup:off
defaultrouter:192.168.0.1
defaultrouter6:auto
depends:none
devfs_ruleset:5
dhcp:0
enforce_statfs:2
exec_clean:1
exec_created:/usr/bin/true
exec_fib:0
exec_jail_user:root
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_system_jail_user:0
exec_system_user:root
exec_timeout:60
host_domainname:none
host_hostname:plexjail
host_hostuuid:plexjail
host_time:1
hostid:BBBBBBBB-BBBB-BBBB-BBBB-D43D7E4CAC0C
hostid_strict_check:0
interfaces:vnet0:bridge0
ip4:new
ip4_addr:vnet0|192.168.0.41/24
ip4_saddrsel:1
ip6:new
ip6_addr:none
ip6_saddrsel:1
ip_hostname:0
jail_zfs:0
jail_zfs_dataset:iocage/jails/plexjail/data
jail_zfs_mountpoint:none
last_started:none
localhost_ip:none
login_flags:-f root
mac_prefix:015df6
maxproc:off
memorylocked:off
memoryuse:off
mount_devfs:1
mount_fdescfs:1
mount_linprocfs:0
mount_procfs:0
mountpoint:readonly
msgqqueued:off
msgqsize:off
nat:0
nat_backend:ipfw
nat_forwards:none
nat_interface:none
nat_prefix:172.16
nmsgq:off
notes:none
nsem:off
nsemop:off
nshm:off
nthr:off
openfiles:off
origin:readonly
owner:root
pcpu:off
plugin_name:none
plugin_repository:none
priority:99
pseudoterminals:off
quota:none
readbps:off
readiops:off
release:11.3-RELEASE-p6
reservation:none
resolver:/etc/resolv.conf
rlimits:off
rtsold:0
securelevel:2
shmsize:off
stacksize:off
state:up
stop_timeout:30
swapuse:off
sync_state:none
sync_target:none
sync_tgt_zpool:none
sysvmsg:new
sysvsem:new
sysvshm:new
template:0
type:jail
used:readonly
vmemoryuse:off
vnet:1
vnet0_mac:none
vnet1_mac:none
vnet2_mac:none
vnet3_mac:none
vnet_default_interface:auto
vnet_interfaces:none
wallclock:off
writebps:off
writeiops:off

This is after transferring to 11.3 but issue persists move back

 

[silverback]

Do you have a DHCP server on your network? I would try a new jail and see if DHCP assigned jail has internet.

 

[Tokolozi]

Do you have a DHCP server on your network? I would try a new jail and see if DHCP assigned jail has internet.

The Router from the ISP is technically the DHCP server as well, no way to change it without double natting the network. something I want to avoid.

But when trying to make a Jail with DHCP enabled I get the following error:

Acquiring DHCP address: FAILED, address received: 0.0.0.0/8 Stopped plex due to DHCP failure Partial plugin destroyed

 

[KevDog]

Your vnet0_mac address is none.

For example mine is :
vnet0_mac:02ff604fca3f 02ff604fca40

Because of the arp tables, I don't think you are going to be able to route properly without a MAC address. The GUI automatically generated one for me on my jail creation. I'm not sure if you are using the GUI or creating jails at the CLI however try adding a MAC address in.

 

[Tokolozi]

Your vnet0_mac address is none.

For example mine is :
vnet0_mac:02ff604fca3f 02ff604fca40

Because of the arp tables, I don't think you are going to be able to route properly without a MAC address. The GUI automatically generated one for me on my jail creation. I'm not sure if you are using the GUI or creating jails at the CLI however try adding a MAC address in.

I'll try this evening to make a new one with Mac.
However made this using GUI and CLI already. At some point I even remembered having a MAC adresss there but seems that with multiple tries it vanished....

 

[Tokolozi]

reinstalled everything, seems to be working now...

 

[KevDog]

I'm glad you found a solution. Sorry I couldn't be more helpful.

 

[Tokolozi]

I'm glad you found a solution. Sorry I couldn't be more helpful.

No problem. No real data was on it yet so sometimes better to just bite the bullet and start from scratch.