LACP slowdown, Weird MAC (CAM) Table, and other unsolved mysteries

[justgosh]

FreeNAS 9.10
When you assign a port to a LACP group all of the interface's MAC addresses are changed to be the same MAC address. All the bonded ports have the same MAC address. Checking the CAM (MAC) table on the switch says the same. Ifconfig shows the same.
When you delete the LACP group, the interfaces keep the software assigned software MAC address until you reboot FreeNAS. Ifconfig shows the same. The switch says the same. My switch alerted me to the MAC issues
Apr 25 10:13:14.274: %SW_MATM-4-MACFLAP_NOTIF: Host 001f.2956.dffe in vlan 1 is flapping between port Gi0/5 and port Gi0/6

Should I report this as a bug?

In order to enable LACP:
I deleted the interfaces
Created a port group selecting LACP
Added the interfaces to the aggregate
Configured the network settingson the port group with a static IP, Gateway, and DNS.
On the switch side I created an etherchannel group with mode set to active
The etherchannel group came up with the happy "%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up" and "%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel2, changed state to up"

When I tested the throughput; I used a 2012 server with a LACP NIC team, through a Cisco switch, to a FreeNAS 9.10 server. iPerf connected, but the throughput was somehow lower with LACP enabled. I tested from several single interface clients and the throughput was slower than before I setup LACP.

Whenever I fiddled with the interfaces, I needed to restart the file sharing services SMB and NFS.

Lastly, NFS wouldn't work over the LACP connection to a ESXi server.

Any thoughts, ideas, or comments would be appreciated.

Edit: after 2 days of turning it off and on again, NFS started working over the LACP connection.

 

[jgreco]

You can report it as a bug if you wish, but it isn't clear that this is a real problem. Usually link aggregations are set up and taken down as part of initial system configuration, and a reboot would normally be done before production traffic was involved.

Somewhat reduced throughput with LACP is not unexpected. You're adding a layer. The goal of LACP isn't to make a single connection go faster.

 

[justgosh]

@jgreco it's a 5% slowdown (811-842). 5% is really unexpected.
Any thoughts on why NFS would stop working as soon as it's on a LACP interface?

The reasons I haven't done a bug reported are, I don't even know where to begin and I may have derped setting up LACP on the FreeNAS :)

Can someone confirm they have connected NFS 9.10 over a LACP connection to ESXi 6.0.

 

[jgreco]

@jgreco it's a 5% slowdown (811-842). 5% is really unexpected.

It's a bit higher than I might have guessed, which might have been more like 1-2%.

Any thoughts on why NFS would stop working as soon as it's on a LACP interface?

Did you reboot and clear the ARP cache on the client?

The reasons I haven't done a bug reported are, I don't even know where to begin and I may have derped setting up LACP on the FreeNAS :)

Can someone confirm they have connected NFS 9.10 over a LACP connection to ESXi 6.0.

 

[justgosh]

Did you reboot and clear the ARP cache on the client?

Several reboots and attempts later I'm still getting an error.

Call "HostDatastoreSystem.CreateNasDatastore" for object "ha-datastoresystem" on ESXi "192.168.16.222" failed.
NFS mount 192.168.16.230:/mnt/POOL-01/NFS1 failed: Unable to connect to NFS server.

Update: AS SOON AS I posted this... it worked. I've been getting that error for 2 days now.... and as soon as I tell the world it's a pile of... it works.

 

[jgreco]

Update: AS SOON AS I posted this... it worked. I've been getting that error for 2 days now.... and as soon as I tell the world it's a pile of... it works.

They do make "I Hate Computers" shirts.