LACP performance question.

[Greg_E]

I've read the post about LACP friend or foe https://www.ixsystems.com/community/threads/lacp-friend-or-foe.30541/ and set up my system accordingly knowing that there were limitations on how it would perform. But I'm seeing something different now that I have one of my servers going through a pair of 10gb NICs.

What I'm seeing is that one NIC will get most of the transmit tasks, and the other NIC will get the receive tasks. This is built with a SuperMicro X10DRi mainboard, consulting the manual it shows that certain PCIe slots are connected to processor 1 and others to processor 2. Given this info, I put one NIC in the slots so that each processor had control over one of them thinking this might give better performance.

Switches are Enterasys C5k125 series.

Has something changed from the older post where a client should be working from a single NIC and the next client might work from the second NIC? I kind of saw the same performance when I had the 1gb NICs in LACP, but it was harder to see because the clients are all at 1GB so everything was more saturated. With the 10GB NICs I'm seeing a larger difference between send and receive. Please see the attached image for a picture of what I'm seeing. You can clearly see that mxge0 is receiving and mxge1 is transmitting, and due the the amount of bandwidth used that this is probably a single client fetching a file.

 

[MatthewSteinhoff]

this is probably a single client fetching a file.

A single client will never benefit from LACP so I'm not sure this graph is relevant to the larger question of if LACP is working or of there are issues with the NIC configuration.

Hit the server with a couple dozen heavy clients then post some graphs. That'll give a truer picture of what's happening.

Cheers,
Matt

 

[Greg_E]

Matt,

That's the plan when the semester is over. I want to test how many streams of DNxHD video at 145mbps can get out of this server at once. I know it will handle our needs at 35mbps XDCam EX, but want to try the better compression of the DNxHD. 16 computers in the lab and probably 2 or 3 streams per computer which should give me a good heavy load.

But I have seen similar before where it appears a client is sending on one of those NICs and receiving from the other. Might be a function of the switch too.

 

[Jessep]

LACP isn't about single client performance.

What you describe is round-robin, where each request goes to the next port in the Team. LACP can be setup with several different methods.