Network Interfaces UP and DOWN Rapidly, 24/7

[Joseph Sharbutt]

Please see excerpt from my console log. This is happening 24/7. The network still appears to be functioning, fine, but I can't imagine that the adapters are supposed to bounce like that. I have two dual Gb Intel server NICs, one set up as an LACP LAGG connected to a Gig Cisco Switch with LACP configured.

I'm also running two or three jails. Could this have something do do with it? I've been fighting it for awhile now, and I'm about to just remove all of the NICs and start over with a simple network setup and go from there.

11.3 also appeared to have killed my Active Directory connection. I've done nothing the last two days but try to figure out why I can't get FreeNAS to connect to my domain. It has been working for years.

Jul 16 05:25:31 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to UP
Jul 16 05:25:31 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to UP
Jul 16 05:25:31 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to UP
Jul 16 05:25:31 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to UP
Jul 16 05:25:31 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to UP
Jul 16 05:25:31 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to UP
Jul 16 05:25:48 fnas11.jlshq.sharbutt.net nfsd: can't register svc name
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net bridge0: Ethernet address: 02:10:0f:1b:f7:00
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net epair0a: Ethernet address: 02:17:d0:00:09:0a
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net epair0b: Ethernet address: 02:cc:20:00:0a:0b
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: epair0a: link state changed to UP
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: epair0a: link state changed to UP
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: epair0b: link state changed to UP
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: epair0b: link state changed to UP
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: epair0a: changing name to 'vnet0.1'
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: em0: promiscuous mode enabled
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: em1: promiscuous mode enabled
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: lagg0: promiscuous mode enabled
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: bridge0: link state changed to UP
Jul 16 05:26:06 fnas11.jlshq.sharbutt.net kernel: bridge0: link state changed to UP
Jul 16 05:26:07 fnas11.jlshq.sharbutt.net kernel: vnet0.1: promiscuous mode enabled
Jul 16 05:26:07 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to DOWN
Jul 16 05:26:07 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to DOWN
Jul 16 05:26:07 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to DOWN
Jul 16 05:26:07 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to DOWN
Jul 16 05:26:07 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to DOWN
Jul 16 05:26:07 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to DOWN
Jul 16 05:26:09 fnas11.jlshq.sharbutt.net kernel: lo0: link state changed to UP
Jul 16 05:26:09 fnas11.jlshq.sharbutt.net kernel: lo0: link state changed to UP
Jul 16 05:26:10 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to UP
Jul 16 05:26:10 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to UP
Jul 16 05:26:10 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to UP
Jul 16 05:26:10 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to UP
Jul 16 05:26:10 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to UP
Jul 16 05:26:10 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to UP
Jul 16 16:03:01 fnas11.jlshq.sharbutt.net kernel: vnet0.1: link state changed to DOWN
Jul 16 16:03:01 fnas11.jlshq.sharbutt.net kernel: vnet0.1: link state changed to DOWN
Jul 16 16:03:01 fnas11.jlshq.sharbutt.net kernel: epair0b: link state changed to DOWN
Jul 16 16:03:01 fnas11.jlshq.sharbutt.net kernel: epair0b: link state changed to DOWN
Jul 16 16:03:02 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to DOWN
Jul 16 16:03:02 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to DOWN
Jul 16 16:03:02 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to DOWN
Jul 16 16:03:02 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to DOWN
Jul 16 16:03:02 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to DOWN
Jul 16 16:03:02 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to DOWN
Jul 16 16:03:04 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to UP
Jul 16 16:03:04 fnas11.jlshq.sharbutt.net kernel: em0: link state changed to UP
Jul 16 16:03:04 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to UP
Jul 16 16:03:04 fnas11.jlshq.sharbutt.net kernel: lagg0: link state changed to UP
Jul 16 16:03:04 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to UP
Jul 16 16:03:04 fnas11.jlshq.sharbutt.net kernel: em1: link state changed to UP

 

[Patrick M. Hausen]

The other side of your lagg probably does not agree with your FreeNAS on either the physical port settings or the LACP parameters or both. This or the cabling is flaky in a way to come out as byzantine.

 

[Samuel Tai]

Have you checked to ensure your cables are good and seated properly? This is the simplest thing to check and rule out first.

 

[Joseph Sharbutt]

The other side of your lagg probably does not agree with your FreeNAS on either the physical port settings or the LACP parameters or both. This or the cabling is flaky in a way to come out as byzantine.

If that was the case, why would it be happening on all adapters, not just the ones that participate in the LAGG?

I'm not super familiar with how the jails bridge across the adapters to get their own interface, but I do seem to remember something about jails causing similar issues.

This isn't supposed to be happening thought, right? I can't even troubleshoot anything else because there are too many messages about my interfaces going up and down.

 

[Samuel Tai]

Try unplugging one side of the LAGG to see if things stabilize. Also make sure the LACP load-balancing algorithm on the switch hasn't reverted to the default of source/destination MAC. For your application, source/destination IP may be a better algorithm.

 

[Joseph Sharbutt]

Try unplugging one side of the LAGG to see if things stabilize. Also make sure the LACP load-balancing algorithm on the switch hasn't reverted to the default of source/destination MAC. For your application, source/destination IP may be a better algorithm.

Thank you, I will give this a shot.