Strange I350-T4 Behavior with Jail Web GUI

[boggie1688]

I'm experiencing some bizarre behavior from my jails after installing a Dell I350 T4. Perhap someone can provide some insight or a suggestion on how to troubleshoot.

Truenas recognizes the card immediately, and I can see all 4 nics on the dashboard. I've connected IGB1 to my switch, and it is dynamically assigned an IP from my DCHP Pfsense box. The whole reason for switching to the I350 is to get away from the onboard Realtek nic (RE0). Next, I hope into my pfsense GUI, delete the static IP assigned to RE0, find IGB1's mac address and statically assign RE0's old IP to it. Apply on the Pfsense box, and reboot the Truenas box.

All in all, I've simply just moved the static IP from RE0 to IGB1. Upon startup, IGB1 is correctly assigned the proper static IP, and RE0 is assigned some dynamic IP.

Everything works. All the jails are accessible from their respective address, and all networked computers can access the NAS box as needed. However, after a few minutes to sometimes an hour, the jail web gui's stop responding. I'm forced to manually go into the Truenas GUI to bring the jail down, and then start it back up. This problem persists, no matter how many times I reboot Truenas or the jail. It also appears the jail is humming along nicely even though the web GUI is inaccessible.

I've tried a different NIC port (IGB0), and the strange behavior continues. However, if I switch everything back to RE0 is fine.

To make things more interesting, during the period while I'm using IGB1 on my static assigned ip, only certain devices on my network can access any given jail's web GUI. I noticed my laptop (WIFI) can, however my android phone can't, yet both of these devices can access the Truenas web GUI.

Is it possible that some settings on the I350-T4 persisted from its pervious installation? If so, how might I go about wiping it clean?

 

[boggie1688]

Someone suggest that I unplug RE0 and actually disable it. While I had it unplugged, I never disabled the onboard nic, which I guess may have been screwing with the jail's bridge?

Everything seems fine now.

 

[boggie1688]

Nevermind, problem is still present. It just took longer to pop up. This is so WEIRD.

 

[Patrick M. Hausen]

If your jails vnet_default_interface is set to "auto", it will create the bridge with the interface that has got the default route. Which should be igb1. You should definitely never plug in two interfaces into the same LAN unless you know very well what you are doing.
You can check if the bridge is really configured the right way with ifconfig on the command line.

if you had explicitly configured re0 for your jails you should change that to "auto" or "igb1" if you have only one flat LAN. If there are multiple networks, possibly VLANs etc. things are a bit more complicated. But first check if the jails really use igb1.

 

[boggie1688]

If your jails vnet_default_interface is set to "auto", it will create the bridge with the interface that has got the default route. Which should be igb1. You should definitely never plug in two interfaces into the same LAN unless you know very well what you are doing.
You can check if the bridge is really configured the right way with ifconfig on the command line.

if you had explicitly configured re0 for your jails you should change that to "auto" or "igb1" if you have only one flat LAN. If there are multiple networks, possibly VLANs etc. things are a bit more complicated. But first check if the jails really use igb1.

Thanks for taking the time to reply.

I quickly learned having two nics on the same lan was a bad bad idea. RE0 was disconnected shortly after the i350 install, but still the problem persisted till someone suggested I completely disable the onboard nic. This resulted in everything working for about 8 hrs.

Here is my ifconfig, from the Truenas shell, with the i350 completely removed from the box. The onboard RE0 is enabled, and all three jails are working. All three jails have their vnet default interface set to "auto", looks like it grabs the RE0 nic. Under network properties for the jail, all three jails have the line vnet0:bridge0.

Code:

root@freenas:~ # ifconfig
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=82099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether 2c:f0:5d:62:43:a2
        inet 192.168.1.133 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:eb:43:21:f3:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: vnet0.6 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 7 priority 128 path cost 2000
        member: vnet0.4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 2000
        member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
        groups: bridge
        nd6 options=1<PERFORMNUD>
vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: Plex as nic: epair0b
        options=8<VLAN_MTU>
        ether 2e:f0:5d:31:41:ea
        hwaddr 02:1b:5f:3f:d3:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
vnet0.4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: zoneminder as nic: epair0b
        options=8<VLAN_MTU>
        ether 2e:f0:5d:21:a9:30
        hwaddr 02:a9:f8:78:e4:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
vnet0.6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: nzb as nic: epair0b
        options=8<VLAN_MTU>

Here is ifconfig's output when RE0 is disabled completely, the i350 is installed, an ethernet is connected to IGB3.

Code:

root@freenas:~ # ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:38
        media: Ethernet autoselect
        status: no carrier
        nd6 options=1<PERFORMNUD>
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:39
        media: Ethernet autoselect
        status: no carrier
        nd6 options=1<PERFORMNUD>
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:3a
        media: Ethernet autoselect
        status: no carrier
        nd6 options=1<PERFORMNUD>
igb3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6>
        ether a0:36:9f:08:01:3b
        inet 192.168.1.133 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:eb:43:21:f3:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: vnet0.3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 10 priority 128 path cost 2000
        member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 9 priority 128 path cost 2000
        member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 8 priority 128 path cost 2000
        member: igb3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 20000
        groups: bridge
        nd6 options=1<PERFORMNUD>
vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: Plex as nic: epair0b
        options=8<VLAN_MTU>
        ether 2e:f0:5d:31:41:ea
        hwaddr 02:2f:a0:b8:14:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
vnet0.2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: nzb as nic: epair0b
        options=8<VLAN_MTU>
        ether 2e:f0:5d:21:a9:29
        hwaddr 02:2c:20:14:5e:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
vnet0.3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: zoneminder as nic: epair0b
        options=8<VLAN_MTU>

To my inexperienced eyes, everything looks fine. But the jails act strangely, sometimes I can access them, sometimes I can't. If I restart the jails, I can access them from my laptop, but not my android phone both of which are on wifi. The router box can individually ping each jail no problem, while the devices that cannot reach the web gui cannot ping them.

I'm genuinely baffled by this working once in awhile situation.

 

[Patrick M. Hausen]

Could you enclose the command output in code tags, please? [CODE]something[/CODE]

 

[boggie1688]

Could you enclose the command output in code tags, please? [CODE]something[/CODE]

Done.

I should also explicitly say that when using the i350, I set all the jail's vnet default interface to IGB3, the port of the nic that has the ethernet connected.

 

[Patrick M. Hausen]

That looks good. You should move the IP address configuration from igb3 to bridge0 as documented in the FreeBSD handbook. If you don't have a bridge0 interface configured but have it autocreated by the jail subsystem, you need to remove it first by

• disabling autostart for all jails
• rebooting

Then you can follow these steps:

• remove the IP address from igb3
• disable hardware offloading for igb3 - important!
• create a bridge interface, name bridge0, assign IP address to that, configure igb3 as member
• change vnet_default_interface to "none" for all jails and make sure to set vnet0:bridge0 down in the network options

Your jails should run stable, then.

 

[boggie1688]

That looks good. You should move the IP address configuration from igb3 to bridge0 as documented in the FreeBSD handbook. If you don't have a bridge0 interface configured but have it autocreated by the jail subsystem, you need to remove it first by

• disabling autostart for all jails
• rebooting

Then you can follow these steps:

• remove the IP address from igb3
• disable hardware offloading for igb3 - important!
• create a bridge interface, name bridge0, assign IP address to that, configure igb3 as member
• change vnet_default_interface to "none" for all jails and make sure to set vnet0:bridge0 down in the network options

Your jails should run stable, then.

I'll try this right now.

 

[boggie1688]

That looks good. You should move the IP address configuration from igb3 to bridge0 as documented in the FreeBSD handbook. If you don't have a bridge0 interface configured but have it autocreated by the jail subsystem, you need to remove it first by

• disabling autostart for all jails
• rebooting

Then you can follow these steps:

• remove the IP address from igb3
• disable hardware offloading for igb3 - important!
• create a bridge interface, name bridge0, assign IP address to that, configure igb3 as member
• change vnet_default_interface to "none" for all jails and make sure to set vnet0:bridge0 down in the network options

Your jails should run stable, then.

Oh boy, I must have misconfigured something because I cannot access the web gui now. Hang on, I'm trying to plug the ethernet into a different nic so I can get back in.

 

[boggie1688]

That looks good. You should move the IP address configuration from igb3 to bridge0 as documented in the FreeBSD handbook. If you don't have a bridge0 interface configured but have it autocreated by the jail subsystem, you need to remove it first by

• disabling autostart for all jails
• rebooting

Then you can follow these steps:

• remove the IP address from igb3
• disable hardware offloading for igb3 - important!
• create a bridge interface, name bridge0, assign IP address to that, configure igb3 as member
• change vnet_default_interface to "none" for all jails and make sure to set vnet0:bridge0 down in the network options

Your jails should run stable, then.

So when configuring the interfaces, I disabled DHCP and remove the IP address from IGB3. I then created a bridge with the ip address 192.168.1.133. Upon rebooting, I cannot access the web gui. I tried plugging the ethernet cable to different ports, re-enabiling re0, but nothing seems be assigned to any of the ports because none of them have DHCP enabled.

 

[Patrick M. Hausen]

You can activate DHCP on the bridge interface. Just make sure to have igb3 as a member, otherwise the bridge will not have a connection to your LAN. And disable hardware offloading (I am repeating myself) on igb3.
I did not get from your output that you were using DHCP. I would not recommend that for a server.

 

[boggie1688]

So i disable DHCP for IGB3, ensure hardware offloading is disable (truenas loves to warn me about this), then create a bridge1, add IGB3 and set the ip to 192.168.1.133.

Everytime I do this and reboot, I lose access to the truenas gui.


Here the thing though, I can ping the truenas box from my pfsense box at 192.168.1.133. Now the entire truenas gui is exhibiting the same issues as the jails.

 

[Patrick M. Hausen]

Is there a router between your desktop and the TrueNAS? In that case you need to set the default gateway in Network --> Global Settings. Or stick to DHCP, which you can use with the bridge just the same.

 

[Patrick M. Hausen]

Plus, you are aware that saving the network config is a two step process, now? First "Test", then "Save" ...

 

[boggie1688]

Now using IGB0, moved the the ethernet cable. Problem is still persisting. I can access it from my laptop, but not my smartphone.

Jails all set to vnet default interface none. All have vnet0:bridge0 under interface.

Code:

root@truenas[~]# ifconfig
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8100b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER>
        ether a0:36:9f:08:01:38
        inet 192.168.1.66 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:39
        media: Ethernet autoselect
        status: no carrier
        nd6 options=9<PERFORMNUD,IFDISABLED>
igb2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:3a
        media: Ethernet autoselect
        status: no carrier
        nd6 options=1<PERFORMNUD>
igb3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:3b
        media: Ethernet autoselect
        status: no carrier
        nd6 options=1<PERFORMNUD>
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether 2c:f0:5d:62:43:a2
        media: Ethernet autoselect (none)
        status: no carrier
        nd6 options=1<PERFORMNUD>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:eb:43:21:f3:00
        inet 192.168.1.133 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: vnet0.3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 11 priority 128 path cost 2000
        member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 10 priority 128 path cost 2000
        member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 9 priority 128 path cost 2000
        member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: nzb as nic: epair0b
        options=8<VLAN_MTU>
        ether 2e:f0:5d:21:a9:29
        hwaddr 02:2c:20:14:5e:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
vnet0.2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: Plex as nic: epair0b
        options=8<VLAN_MTU>
        ether 2e:f0:5d:31:41:ea
        hwaddr 02:92:05:14:13:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
vnet0.3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: zoneminder as nic: epair0b
        options=8<VLAN_MTU>

 

[Patrick M. Hausen]

You need to remove the DHCP setting from igb0. You cannot have two interfaces (igb0 and bridge0) with different addresses in the same network.
The rest looks OK. How do the jails get their IP addresses?

Edit: just to make sure - please put "up" into the "Options" field of igb0. There were versions of FreeNAS/TrueNAS that did not automatically enable bridge members without an IP address.

Yes, it's complicated. TrueNAS violates the documented FreeBSD standard in its default configuration, which clearly states that bridge member interfaces must not have an IP address configured. All IP configuration, IPv4 and IPv6, goes on the bridge interface instead. So I have come not to rely on the automatic creation of bridge interfaces, anymore, and configure everything manually.

 

[Patrick M. Hausen]

To get the bridge interface more stable in terms of ARP (mapping of IP address to MAC address) you can force it to inherit the MAC address from the first member interface. For that you need these two tunables:

For copy & paste:
if_bridge_load
net.link.bridge.inherit_mac

 

[boggie1688]

You need to remove the DHCP setting from igb0. You cannot have two interfaces (igb0 and bridge0) with different addresses in the same network.
The rest looks OK. How do the jails get their IP addresses?

Edit: just to make sure - please put "up" into the "Options" field of igb0. There were versions of FreeNAS/TrueNAS that did not automatically enable bridge members without an IP address.

Yes, it's complicated. TrueNAS violates the documented FreeBSD standard in its default configuration, which clearly states that bridge member interfaces must not have an IP address configured. All IP configuration, IPv4 and IPv6, goes on the bridge interface instead. So I have come not to rely on the automatic creation of bridge interfaces, anymore, and configure everything manually.

Jails are statically assigned IP address by mac address from the pfsense box. When the bridge works, it looks like pfsense is correctly assigning IPs.

Good call on the UP option, my box a[pears to be susceptible to this.

It looks like the bridge is running well, however the issue persists with the web gui for each jail not being accessible by my phone.

Code:

Warning: settings changed through the CLI are not written to
the configuration database and will be reset on reboot.

root@truenas[~]# ifconfig
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8100b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER>
        ether a0:36:9f:08:01:38
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:39
        media: Ethernet autoselect
        status: no carrier
        nd6 options=9<PERFORMNUD,IFDISABLED>
igb2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:3a
        media: Ethernet autoselect
        status: no carrier
        nd6 options=1<PERFORMNUD>
igb3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:08:01:3b
        media: Ethernet autoselect
        status: no carrier
        nd6 options=1<PERFORMNUD>
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether 2c:f0:5d:62:43:a2
        media: Ethernet autoselect (none)
        status: no carrier
        nd6 options=1<PERFORMNUD>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:eb:43:21:f3:00
        inet 192.168.1.133 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: vnet0.3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 11 priority 128 path cost 2000
        member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 10 priority 128 path cost 2000
        member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 9 priority 128 path cost 2000
        member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: Plex as nic: epair0b
        options=8<VLAN_MTU>
        ether 2e:f0:5d:31:41:ea
        hwaddr 02:2c:20:14:5e:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
vnet0.2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: nzb as nic: epair0b
        options=8<VLAN_MTU>
        ether 2e:f0:5d:21:a9:29
        hwaddr 02:92:05:14:13:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
vnet0.3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: zoneminder as nic: epair0b
        options=8<VLAN_MTU>

 

[Patrick M. Hausen]

Only the phone, but the desktop works fine? Possibly a different issue altogether?

SSL involved? IOS does not connect to servers with certificates that exceed 397 days of lifetime, anymore, for example.

Other than that I would now start to do packet traces with tcpdump.