Jails on Core 13.0-U5.2 do not connect to the internet

BuffTofu

BuffTofu

Dabbler
Joined
Dec 18, 2021
Messages
35
winnielinnie said:
Can you create a new test jail, and see if DNS works under the test jail?
Click to expand...
To add on to this, one of my jails, as you may have seen, is Homebridge, and was installed via the Plugins page. If I try to install any plugin, it tries to connect it to pkg and fails to, which is the same reason why it can't ping domains. I doubt it is a configuration issue.
 
Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
Whattteva

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
BuffTofu said:
Yes
Click to expand...
So, now you're back from U5 to U3 and it still doesn't work? Then it sounds like it probably wasn't the upgrade and something else that caused it that may have been catalyzed by the reboot.
 
BuffTofu

BuffTofu

Dabbler
Joined
Dec 18, 2021
Messages
35
Whattteva said:
So, now you're back from U5 to U3 and it still doesn't work? Then it sounds like it probably wasn't the upgrade and something else that caused it that may have been catalyzed by the reboot.
Click to expand...
I don’t know. But the screen now just shows this
 

Attachments

  • IMG_8496.jpeg
    IMG_8496.jpeg
    824.3 KB · Views: 252
Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
There is definitely something screw up in that machine or in that network.
 
Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
Can you take a screenshot of your system's network interfaces?
 
Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
BuffTofu

BuffTofu

Dabbler
Joined
Dec 18, 2021
Messages
35
Davvo said:
Your bridge interface doesn't have an IP.
Click to expand...
I saw no change with or without one. Is the bridge its own device, or does it share an IP with the jail? I have had 2 jails running on 1 bridge and it worked fine.
 
Davvo

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
BuffTofu said:
I saw no change with or without one. Is the bridge its own device, or does it share an IP with the jail? I have had 2 jails running on 1 bridge and it worked fine.
Click to expand...
In TN you can assign jails and VMs to a network interface, be it a bridge or a phisical one.
@Patrick M. Hausen has already told you the proper way it should be configured.
 
winnielinnie

winnielinnie

MVP
Joined
Oct 22, 2019
Messages
3,641
I don't have a "bridge0" interface in the TrueNAS Core GUI. It only exists when checking with "ifconfig" in the command-line.

Did you manually create the "bridge0" interface using the GUI?

In my case, the bridge is configured per each jail.

For example, a jail will use re0 as the vnet_default_interface, and then under Network Properties the interfaces is vnet0:bridge0

In summary:
  • Before creating any jails, the TrueNAS Core GUI shows the physical network cards that you can configure for the server itself
  • No need to manually create a bridge interface (i.e, "bridge0") in the GUI
  • When you create and configure a jail to use VNET, and assign it a vnet_default_interface (i.e, "re0"), it should automatically create a bridge interface (which will still not show up in the GUI; only in the command-line)
  • You can then confirm this by checking the jail's Network Properties and even using "ifconfig" to check in an SSH session


EDIT:

This might mean you'd have to stop all your jails, then delete "bridge0" via the GUI -> Network -> Interfaces

Then while your jails are still down, configure them to use "re0" as the vnet_default_interface and to use vnet0:bridge0 in its Network Properties for "interfaces"

Then start your jails to finish the process.
 
Last edited:
S

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
13-u5.3 definitely breaks jail networking.
After upgrade from 5.2 to 5.3 all jails networking stopped working, revert to 5.2 fixes it without any config changes.
in 5.3 newly created jails don't work, not in NAT not VNET,
DHCP fails to acquire address
Manual IP assignment results with no connection at all, can't ping host, can't ping other jails, just a black hole.
All my jails are and always were configured as per winnielinne last post.
only change in networking from fresh install is "gateway_enable" tunable and fixed IP for main network interface
ifconfig in jail both 5.3 and 5.2 show the same:

Code:
ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
epair0b: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=8<VLAN_MTU>
    ether 46:8a:5b:0e:ad:6e
    hwaddr 02:26:2f:2b:f1:0b
    inet 192.168.1.155 netmask 0xffffff00 broadcast 192.168.1.255
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=1<PERFORMNUD>

conf1.png





conf2.png
conf3.png
 
Patrick M. Hausen

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
Output of ifconfig on the host, please.
 
S

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
Patrick M. Hausen said:
Output of ifconfig on the host, please.
Click to expand...
that's from 5.2
Code:
ifconfig         
igb0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP>
    ether 44:8a:5b:f5:a3:ca
    inet 192.168.1.120 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether 44:8a:5b:f5:a3:cb
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 58:9c:fc:10:ff:9b
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 10 priority 128 path cost 2000
    member: vnet0.3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 7 priority 128 path cost 2000
    member: vnet0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 6 priority 128 path cost 2000000
    member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 20000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    ether fe:a0:98:3a:25:8a
    hwaddr 58:9c:fc:10:b9:32
    groups: tap
    media: Ethernet autoselect
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
    Opened by PID 1780
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    inet 10.10.0.1 --> 10.10.0.2 netmask 0xffffff00
    groups: tun
    nd6 options=9<PERFORMNUD,IFDISABLED>
    Opened by PID 2104
tun1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    inet 10.8.0.2 --> 10.8.0.1 netmask 0xffffff00
    groups: tun
    nd6 options=9<PERFORMNUD,IFDISABLED>
    Opened by PID 2065
vnet0.3: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: cloud as nic: epair0b
    options=8<VLAN_MTU>
    ether 44:8a:5b:07:4c:d6
    hwaddr 02:77:9e:58:90:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.4: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: DB as nic: epair0b
    options=8<VLAN_MTU>
    ether 46:8a:5b:99:66:bc
    hwaddr 02:bf:81:c2:32:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>


and 5.3
Code:
 ifconfig                               
igb0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP>
    ether 44:8a:5b:f5:a3:ca
    inet 192.168.1.120 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether 44:8a:5b:f5:a3:cb
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 58:9c:fc:10:ff:9b
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 10 priority 128 path cost 2000
    member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 6 priority 128 path cost 2000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: DB as nic: epair0b
    options=8<VLAN_MTU>
    ether 46:8a:5b:99:66:bc
    hwaddr 02:89:e7:0f:56:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 58:9c:fc:10:ff:fd
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 8 priority 128 path cost 2000000
    member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 20000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    ether fe:a0:98:3a:25:8a
    hwaddr 58:9c:fc:10:b9:32
    groups: tap
    media: Ethernet autoselect
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
    Opened by PID 1823
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    inet 10.10.0.1 --> 10.10.0.2 netmask 0xffffff00
    groups: tun
    nd6 options=9<PERFORMNUD,IFDISABLED>
    Opened by PID 2227
vnet0.2: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: db-copy as nic: epair0b
    options=8<VLAN_MTU>
    ether 46:8a:5b:0e:ad:6d
    hwaddr 02:7a:15:62:34:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
tun1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    inet 10.8.0.2 --> 10.8.0.1 netmask 0xffffff00
    groups: tun
    nd6 options=9<PERFORMNUD,IFDISABLED>
    Opened by PID 2058
vnet0.3: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: cloud as nic: epair0b
    options=8<VLAN_MTU>
    ether 44:8a:5b:07:4c:d6
    hwaddr 02:bf:81:c2:32:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
 
Patrick M. Hausen

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
igb0 is missing from the bridge interface. And your configuration is invalid even with U5.2. The IP address must be on the bridge0, not on igb0.

- reboot with all jails autostart disabled
- create an interface of type bridge named bridge0
- add igb0 as the only member interface
- remove the IP address from igb0, instead put "up" into the options field
- put the IP address on bridge0
- test and save - you should increase the timeout from 60 to e.g. 300 seconds before clicking "test", then clear the ARP cache of your desktop system

If that worked, then for each jail:

- vnet_default_interface: none

The rest looks good. Your jails should work afterwards.
 
Patrick M. Hausen

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
P.S. Temporary fix for U5.3: ifconfig bridge0 addm igb0

Permanent fix: see my last post above.
 
S

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
Patrick M. Hausen said:
igb0 is missing from the bridge interface. And your configuration is invalid even with U5.2. The IP address must be on the bridge0, not on igb0.

- reboot with all jails autostart disabled
- create an interface of type bridge named bridge0
- add igb0 as the only member interface
- remove the IP address from igb0, instead put "up" into the options field
- put the IP address on bridge0
- test and save - you should increase the timeout from 60 to e.g. 300 seconds before clicking "test", then clear the ARP cache of your desktop system

If that worked, then for each jail:

- vnet_default_interface: none

The rest looks good. Your jails should work afterwards.
Click to expand...
thannk you Sir.
its odd it worked for some 5 years since 11u1 and now stopped...
I'll implement changes later and see if it works in 5.3, its a live production system so can't f-about too much ion business hours.
 
You must log in or register to reply here.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "Jails on Core 13.0-U5.2 do not connect to the internet"

Similar threads

Patrick M. Hausen
WARNING - don't upgrade your TrueNAS CORE jails to FreeBSD 13.3 just yet
2 3
Replies
45
Views
21K
victort
victort
P
TrueNAS Core 13.0-U4 to U5 upgrade. Jellyfin plugin quit working not responding on the configured IP address.
Replies
4
Views
3K
pasha-19
P
J
13.0-U3.1: only new users can't connect over SMB
Replies
7
Views
3K
JMY1000
J
J
CORE 13.0-U4 to SCALE - what to consider?
Replies
2
Views
2K
john.doe79
J
B
Unable to connect to SMB Shares on new installation of TRUENAS Core 13.1-MASTER
Replies
4
Views
3K
morganL
M
Top