VNET No Network Connectivity In Jail (Post Migration)

[wooley-x64]

I have migrated to new hardware.

My old system was re-purposed Gaming rig Desktop hardware. I have moved on to much older and bigger things. Dell R710.

Version: FreenNAS 11.2U5

My old configuration topology wise:

Two active Interfaces:
em0: 192.168.1.25 (The FreeNAS interface which I used for file transfers and mgmt)
This interface was onboard integrated to the motherboard

Bridge0
ibg3: This was an addon Intel 4 port gig NIC I had one Port online which I used for the jails solely
vnet0:x 192.168.1.240 (Transmission with OpenVPN)
192.168.1.241 (Sonarr)
192.168.1.242 (Lidarr)
192.168.1.243 (Radarr)
192.168.1.244 (Jackett)

This functioned for the past 8 months without any issues. I know it was a pain to get working, but when it did I was very pleased.

I performed a backup of the config and built FREENAS on a vm on the R710 I connected one port on the Intel NIC in passthru to the FreeNAS VM and I have two interfaces again.

vmx0: 192.168.1.25
igb0:

If I take VNET off the other jails and choose igb0 they come online fine. I do however, need vnet enabled on the transmission jail in order to run OpenVPN.

What I am trying to troubleshoot is why the connectivity is not working.

FREENAS HOST ifconfig:

Code:

igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500

        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>

        ether 00:1b:21:46:88:78

        hwaddr 00:1b:21:46:88:78

        nd6 options=9<PERFORMNUD,IFDISABLED>

        media: Ethernet autoselect

        status: no carrier

vmx0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500

        options=60039b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>

        ether 00:0c:29:c6:13:62

        hwaddr 00:0c:29:c6:13:62

        inet 192.168.1.25 netmask 0xffffff00 broadcast 192.168.1.255

        nd6 options=9<PERFORMNUD,IFDISABLED>

        media: Ethernet autoselect

        status: active

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384

        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>

        inet6 ::1 prefixlen 128

        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3

        inet 127.0.0.1 netmask 0xff000000

        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

        groups: lo

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

        ether 02:90:f2:e4:4d:00

        nd6 options=9<PERFORMNUD,IFDISABLED>

        groups: bridge

        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15

        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200

        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0

        member: vmx0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>

                ifmaxaddr 0 port 2 priority 128 path cost 2000

        member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>

                ifmaxaddr 0 port 1 priority 128 path cost 2000000

Troubled Jail ifconfig: (xmission)

Code:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 94:de:80:22:cf:c2
        hwaddr 02:3f:10:00:06:0b
        inet 192.168.1.240 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        groups: epair

The Tunables:

Code:

cloned_interfaces
bridge0
rc

ifconfig_bridge0
addm igb0 up
rc

Note: That in the old version I had it set to igb3. However, that no longer exists in the new system.

I am thinking the issue might be with how for some reason I have an epair0b on the jail, but epair0a is not defined anywhere. I kind of get lost when I try to think about how this works logically.

This used to work on the desktop and everything else up to this point has been near flawless to get back up and running. I would hate to have to run a second vm just to run transmission and openvpn which has to connect back to this system anyway.

Any troubleshooting steps appreciated,
-Wooley

 

[Jailer]

I think I remember reading a response from someone in the past that to get it work it had something to do with setting up promiscuous mode for the NIC in esxi. I could be wrong on this though.

 

[wooley-x64]

That NIC is passthru to the vm so from my understanding ESX has nothing to do with that device anymore its basically in the hands of FreeNAS as if bare metal

 

[silverback]

The interface igb0 has no carrier reported by ifconfig. Is there a cable from your router.? I don’t see the point in having 2 interface's on the same subnet. But if you want to use the vmx interface, as jailer said you need edit the security settings for you vSwitch to allow promiscuous mode.

 

[wooley-x64]

There is a cable, but it was in the wrong port. I made the dangerous assumption that the lowest vendor ID that ESX showed me was port 0 on the card. Anyway. I got connectivity now. I will have to work on getting the vmx0 interface out of bridge0 as that is causing loops.

TY for your assistance.

 

[wooley-x64]

Update: I have gotten everything up and running now. I have modified the configuration for simplicity sake. 1 interface only now. I think I finally realized that this never worked with two interfaces on the same subnet as mentioned numerous times on this forum that its bad form and can cause problems due to being on the same host.

Despite that, tunnel is up and all the jails are happy.

I finally realized that it was never working the other way and all my traffic on the old box must have been going through the em0 interface. It dawned on me late last night that the original configuration was set for tunable igb3

That would correspond with interface port 4 on the Intel NIC. (0,1,2,3)

Well I know for sure I had Port 3 connected and never port 4.

In the end I get to reclaim an Intel 4 port card and use that in my pfsense box.

Thanks again.