LAGG with LACP

[freebsdsa]

That is a good point. I did not think to look as I purchased these servers from iXsystems. I figure they know the correct locations for the cards.

This is what is reported in dmesg.

ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.5.15> port 0xf020-0xf03f mem 0xfb580000-0xfb5fffff,0xfb604000-0xfb607fff irq 58 at device 0.0 on pci132
ix0: Using MSIX interrupts with 9 vectors
ix0: Ethernet address: 90:e2:ba:58:bf:90
ix0: PCI Express Bus: Speed 5.0GT/s Width x8

I do not remember is dmesg reports what the card can do or info about the PCI slot it is connected too right now. Either way, I will do a visual check on Monday when I am in the office again.

 

[Rand]

ix0: PCI Express Bus: Speed 5.0GT/s Width x8

Hm,
PCIe 2.0= 5.0GT/s,
so that would be ok I guess

 

[Rand]

Have you set lro, txcsum, rxcsum and tso4/6 ? That made a lot of difference for me

 

[freebsdsa]

Have you set lro, txcsum, rxcsum and tso4/6 ? That made a lot of difference for me

I have tested with LRO off (got worst performance). I tested with TSO off as well and did not see a difference at all. I did not play with TXCSUM/RXCSUM off yet. I will have to try that out, but I have never seen those make it work better when disabled.

 

[Rand]

No i use them enabled, not disabled.
They are not set automatically after all thats why i mention them.

 

[freebsdsa]

I must be blind as I thought I saw those turned on, but you are right. Both RXCSUM and TXCSUM are not on by default. Well I will enable those and do some more testing when I am back in the office Monday. Thanks for that tip.

 

[Rand]

none of them was set automatically for me... They are listed in capabilities o/c but are not active

 

[freebsdsa]

hmm. I think TSO and LRO was enabled for me, I cannot recall right now.

 

[freebsdsa]

Ok so I enabled everything and it seems to be better now. Thanks for pointing out that all those features are not enabled by default. This is what I pass for options on the interface now.

"tso vlanhwtso vlanhwcsum lro rxcsum txcsum vlanhwtag"

Once I set this, I was able to push the interface to 99.12% utilization from three hosts. Before it topped out at 33%.

Now back to LACP.

 

[Rand]

Great, good luck with that:)

 

[freebsdsa]

I am done with this testing on the network. I cannot get LACP working at all. If I leave the switch config the same, I can boot into an illumos derivative and I get a working LACP link. When I boot into FreeBSD or FreeNAS 9.2 it just does not work. I am happy with just doing LAGG with failover as that gives me what I need right now. On to other testing on this server now. Thanks all.

 

[Rand]

Well you *could* create a bug report to get that sorted out...;)
If its not ok with FreeBSD then there's maybe not much they can do but it would be documented at least...

 

[freebsdsa]

No I agree with you. I will for sure file a bug very soon. Thanks again for all your feed back.

 

[wreedps]

Did you ever get your LACP issue fixed? I just converted from a Cisco switch to Brocade and I cannot get the LAG to come up. I am 100% certain the new Brocade switch config is correct.

 

[Rick Arman]

bump

 

[Ericloewe]

What do you mean, "bump"? This is an ancient thread.