igb0 does flood network with mDNS information (~ 50 - 80MB/sec)

[Floppsi]

Dear all,

I am running TrueNAS Core -12.0-U8
I using my NAS since very log, since some month i have the problem that after a restart igb0 is sending out MBs of data and killing the whole network.
In the past I could disconnect the port restart the NAS and everything was OK. But this time, no restart does help.

On My Mac I can see that the mDNS Responder does receive about 50 - 80 MB of data per seconds, as you can imagine in that case nothing else is working anymore on the network. As soon as I unplug the ethernet cable from the port igb0 everything is back to normal.
As said restart does not help. After a restart for very short time everything is normal than it starts again.

Any idea, or any hint how I could stop that is very welcome.
igb1 is always connected and I can access the NAS.

Thanks for the help.
Floppsi

 

[anodos]

Dear all,

I am running TrueNAS Core -12.0-U8
I using my NAS since very log, since some month i have the problem that after a restart igb0 is sending out MBs of data and killing the whole network.
In the past I could disconnect the port restart the NAS and everything was OK. But this time, no restart does help.

On My Mac I can see that the mDNS Responder does receive about 50 - 80 MB of data per seconds, as you can imagine in that case nothing else is working anymore on the network. As soon as I unplug the ethernet cable from the port igb0 everything is back to normal.
As said restart does not help. After a restart for very short time everything is normal than it starts again.

Any idea, or any hint how I could stop that is very welcome.
igb1 is always connected and I can access the NAS.

Thanks for the help.
Floppsi

Run command
service avahi-daemon stop
then
avahi-daemon --debug

And see what interfaces it's binding to on startup (and whether it's battling with itself or another device on the network).

 

[Floppsi]

Run command
service avahi-daemon stop
then
avahi-daemon --debug

And see what interfaces it's binding to on startup (and whether it's battling with itself or another device on the network).

Thanks I did the following
1. "Stopping avahi-daemon"
2. plugged Ethernet cable again into igb0 --> still sending out data
3. Unplug Ethernet cable
4.

Code:

root@freenas:/mnt/ # avahi-daemon --debug
Found user 'avahi' (UID 200) and group 'avahi' (GID 200).
Successfully dropped root privileges.
avahi-daemon 0.7 starting up.
Loading service file /usr/local/etc/avahi/services/ADISK.service.
Loading service file /usr/local/etc/avahi/services/AFPOVERTCP.service.
Loading service file /usr/local/etc/avahi/services/DEV_INFO.service.
Loading service file /usr/local/etc/avahi/services/HTTP.service.
Loading service file /usr/local/etc/avahi/services/HTTPS.service.
Loading service file /usr/local/etc/avahi/services/ISCSITARGET.service.
Loading service file /usr/local/etc/avahi/services/MIDDLEWARE.service.
Loading service file /usr/local/etc/avahi/services/MIDDLEWARE_SSL.service.
Loading service file /usr/local/etc/avahi/services/NFS.service.
Loading service file /usr/local/etc/avahi/services/SFTP_SSH.service.
Loading service file /usr/local/etc/avahi/services/SMB.service.
Loading service file /usr/local/etc/avahi/services/SSH.service.
Loading service file /usr/local/etc/avahi/services/WEBDAV.service.
Joining mDNS multicast group on interface igb0.IPv4 with address 192.168.1.5.
New relevant interface igb0.IPv4 for mDNS.
Joining mDNS multicast group on interface ix1.IPv4 with address 192.168.3.2.
New relevant interface ix1.IPv4 for mDNS.
Joining mDNS multicast group on interface ix0.IPv4 with address 192.168.4.2.
New relevant interface ix0.IPv4 for mDNS.
Network interface enumeration completed.
Registering new address record for 192.168.1.5 on igb0.IPv4.
Registering new address record for 192.168.3.2 on ix1.IPv4.
Registering new address record for 192.168.4.2 on ix0.IPv4.
Server startup complete. Host name is freenas.local. Local service cookie is 2043434534.
Service "freenas" (/usr/local/etc/avahi/services/WEBDAV.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/SSH.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/SMB.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/SFTP_SSH.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/NFS.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/MIDDLEWARE_SSL.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/MIDDLEWARE.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/ISCSITARGET.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/HTTPS.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/HTTP.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/DEV_INFO.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/AFPOVERTCP.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/ADISK.service) successfully established.

5. Plug in Ethernet cable into igb0 --> still sending out data

What confuses me that after stopping the avahi-daemon I though the flooding would stop.

 

[anodos]

Thanks I did the following
1. "Stopping avahi-daemon"
2. plugged Ethernet cable again into igb0 --> still sending out data
3. Unplug Ethernet cable
4.

Code:

root@freenas:/mnt/ # avahi-daemon --debug
Found user 'avahi' (UID 200) and group 'avahi' (GID 200).
Successfully dropped root privileges.
avahi-daemon 0.7 starting up.
Loading service file /usr/local/etc/avahi/services/ADISK.service.
Loading service file /usr/local/etc/avahi/services/AFPOVERTCP.service.
Loading service file /usr/local/etc/avahi/services/DEV_INFO.service.
Loading service file /usr/local/etc/avahi/services/HTTP.service.
Loading service file /usr/local/etc/avahi/services/HTTPS.service.
Loading service file /usr/local/etc/avahi/services/ISCSITARGET.service.
Loading service file /usr/local/etc/avahi/services/MIDDLEWARE.service.
Loading service file /usr/local/etc/avahi/services/MIDDLEWARE_SSL.service.
Loading service file /usr/local/etc/avahi/services/NFS.service.
Loading service file /usr/local/etc/avahi/services/SFTP_SSH.service.
Loading service file /usr/local/etc/avahi/services/SMB.service.
Loading service file /usr/local/etc/avahi/services/SSH.service.
Loading service file /usr/local/etc/avahi/services/WEBDAV.service.
Joining mDNS multicast group on interface igb0.IPv4 with address 192.168.1.5.
New relevant interface igb0.IPv4 for mDNS.
Joining mDNS multicast group on interface ix1.IPv4 with address 192.168.3.2.
New relevant interface ix1.IPv4 for mDNS.
Joining mDNS multicast group on interface ix0.IPv4 with address 192.168.4.2.
New relevant interface ix0.IPv4 for mDNS.
Network interface enumeration completed.
Registering new address record for 192.168.1.5 on igb0.IPv4.
Registering new address record for 192.168.3.2 on ix1.IPv4.
Registering new address record for 192.168.4.2 on ix0.IPv4.
Server startup complete. Host name is freenas.local. Local service cookie is 2043434534.
Service "freenas" (/usr/local/etc/avahi/services/WEBDAV.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/SSH.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/SMB.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/SFTP_SSH.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/NFS.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/MIDDLEWARE_SSL.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/MIDDLEWARE.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/ISCSITARGET.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/HTTPS.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/HTTP.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/DEV_INFO.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/AFPOVERTCP.service) successfully established.
Service "freenas" (/usr/local/etc/avahi/services/ADISK.service) successfully established.

5. Plug in Ethernet cable into igb0 --> still sending out data

What confuses me that after stopping the avahi-daemon I though the flooding would stop.

What do mean that it's sending when avahi is stopped? Does traffic continue if you simply stop avahi daemon (and not start with debugging)?

 

[Floppsi]

What do mean that it's sending when avahi is stopped? Does traffic continue if you simply stop avahi daemon (and not start with debugging)?

exactly if I only execute the "service avahi-daemon stop" and connect the ethernet cable the NAS is still sending out the data.

I am still checking if something else in my network could cause that.

 

[anodos]

Make sure you also have all jails and plugins stopped when you're trying to toubleshoot this stuff.

 

[Floppsi]

Make sure you also have all jails and plugins stopped when you're trying to toubleshoot this stuff.

Thanks for the help, I will try that as next.

Here some more information.
My current setup was running since more than 3 years:
Ethernet: (both are connected to one switch)
- igb0 with IP 192.168.1.5 (web interfaces ans the shares are assigned to it)
- igb1 no IP assigned but the 2 VM have it assigned as their NAT
VM, Jails
- 2 VM with NAT assigned to igb1 (fixed IPs in the VMs)
- 5 Jails (Plex, nextcloud, Maria and postrgres DBs) all with vnet0
Tunable:
- Variable: "ifconfig_bridge0", with Value: "addm igb0 up" and Type: "rc.conf"
- Varianle: "cloned_interfaces", with Value: "bridge0" and Type: "rc.conf"

My additional ethernet card is only used for 10GBIt direct connections.
192.168.3.2 on ix1.IPv4. -> to the Mac
192.168.4.2 on ix0.IPv4. -> to my server for iSCSI connection.

If the the ethernet cable is plugged into both igb0 and igb1 I have the high data flow.
If i have only igb0 or igb1 connected everything works normal. in both case I can access the web interface, but in case igb1 is connected, I can't obviously connect to the shares.

For me it seems that probably, if both cables are plugged in, I create something like a deadlock. Not sure what changed compared to the last years.

The current workaround, just use the igb0 and I assigned the VMs to igb0 too. Everything is working, but I still would like to figure out what's going wrong or what changes compared to the past. Or I was only lucky that it did work the last years without problem.