Truenas loses network connectivity when fiddling with jails

[clusty]

I've been plagued by this issue for a while now and I might have gotten to bottom of it (still can't explain)
Randomly, my TrueNAS 12/13 installation is losing network connectivity (as well as the jails).
The "fix" seems to be changing router ports and rebooting the NAS.
The issues seem to be correlated with creating / starting / stopping jails or plugins.
The IPMI interface is very much alive alive throughout the whole process.
My ethernet card is Intel based.

Any idea what could be going on?
I could post more info if it would help getting to the bottom of the problem.

 

[Patrick M. Hausen]

Please post the complete output of ifconfig and the output of iocage get all <jailname> for one of your jails that triggers the problem. Both commands on the NAS host.

Kind regards,
Patrick

 

[clusty]

Please post the complete output of ifconfig and the output of iocage get all <jailname> for one of your jails that triggers the problem. Both commands on the NAS host.

Kind regards,
Patrick

Thanks a lot for the reply:
Here are the info you requested (I masked some Mac addresses).
For some reason now it seems to be working...
Could my UniFi router be to blame for this instability ?

 

[Patrick M. Hausen]

This or a wrong bridge configuration, possibly. I'll look into it tomorrow if I have the time. Bedtime for now ...

 

[Patrick M. Hausen]

You have four virtual network interfaces, so probably four running jails - but only one is a member of the bridge0 interface. Did you configure all jails with VNET enabled?

What you should do in my opionion:

* boot the system with all jails "boot" option disabled - this effectively removes the bridge0 interface
* manually create an interface named bridge0, type BRIDGE, only member ix0
* test, save
* edit ix0: remove IP address, enter "up" in the options field, check "disable hardware offloading"
* edit bridge0: put IP address from previous step on this interface
* test, save
* reboot, check if everything apart from the jails is working

For each jail:

* change vnet_default_interface from "auto" to "none"
* ensure that interfaces is set to "vnet0:bridge0"
* re-enable "boot" option

Reboot the machine one last time and check if all jails are coming up OK.

 

[clusty]

You have four virtual network interfaces, so probably four running jails - but only one is a member of the bridge0 interface. Did you configure all jails with VNET enabled?

What you should do in my opionion:

* boot the system with all jails "boot" option disabled - this effectively removes the bridge0 interface
* manually create an interface named bridge0, type BRIDGE, only member ix0
* test, save
* edit ix0: remove IP address, enter "up" in the options field, check "disable hardware offloading"
* edit bridge0: put IP address from previous step on this interface
* test, save
* reboot, check if everything apart from the jails is working

For each jail:

* change vnet_default_interface from "auto" to "none"
* ensure that interfaces is set to "vnet0:bridge0"
* re-enable "boot" option

Reboot the machine one last time and check if all jails are coming up OK.

Will try now :)
I did set all jails with vnet (default option). Do I need VNET ? :) All jails are set with DHCP rather than NAT since this simplifies the port forwarding (I suck at firewall rules and local host management is a bit easier)

 

[clusty]

You have four virtual network interfaces, so probably four running jails - but only one is a member of the bridge0 interface. Did you configure all jails with VNET enabled?

What you should do in my opionion:

* boot the system with all jails "boot" option disabled - this effectively removes the bridge0 interface
* manually create an interface named bridge0, type BRIDGE, only member ix0
* test, save
* edit ix0: remove IP address, enter "up" in the options field, check "disable hardware offloading"
* edit bridge0: put IP address from previous step on this interface
* test, save
* reboot, check if everything apart from the jails is working

For each jail:

* change vnet_default_interface from "auto" to "none"
* ensure that interfaces is set to "vnet0:bridge0"
* re-enable "boot" option

Reboot the machine one last time and check if all jails are coming up OK.

I've done a simpler setup and things seem to "sorta work". I switched everything to static IPs and stuff comes up more reliably (not 100%).
A fix is to plug/unplug the ethernet on the NAS box.
Could the motherboard NIC be turning to shit ?