Network bridge 10Gb-1Gb slower than without bridge

[EtienneB]

Hello
I am running Truenas Core 13.0U5.2.

I have configured my network as follows:
mac ----10Gb----->Truenas----1Gb---->switch---->modem/router--->Internet

where the 10Gb and the 1Gb are bridged, so that i would have 10Gb connection to my server and then 1Gb to the router for internet access on my mac.

However, when I bridge these 2 Intel adapters (ix1 and igb1), my 10Gb connection only achieves around 2.5Gbit with iperf3.
When I remove the ix1 from the bridge, I get 7.5Gbit on my 10Gb NIC (ix1) with iperf3.
(that it is not hitting 10Gb is another issue, perhaps cable).

I can't find anywhere why my 10Gb NIC would drop because of the bridge to 1/3 of its pre-bridge speed.
When I run iperf, the cpu on the server only hits 18% max.

Any ideas? Or is this expected behaviour of mixing NIC speeds in a bridge?

Thanks,
Etienne

 

[Patrick M. Hausen]

Did you disable hardware offloading for the two member interfaces? Did you move all IP address configuration to the bridge interface leaving only "Options: up" in the two members?

 

[EtienneB]

I had 1 interface with UP, not the 10Gb I saw. So I added UP to it (my mistake there, now I understand why sometimes I needed ipconfig ix1 up).

I had no IP on the 10Gb nic (ix1), only my main server IP on the 1Gb (igb1), that one is moved to the bridge.
So ix1 and igb1 don't have their own IPs.

Hardware offloading was not disabled, and now it is disabled. But then my speed goes down to 2.01Gb.

Also MTU is blank on ix1 and igb1, the bridge defaults to 1500.

BUT, after unchecking the disabling hardware offloading tick marks again, the speed went up to 7.4Gbit on the bridge.
Somehow it needed a reset with this setting it seems. (Hardware offloading is enabled then).

I had already rebooted the machine before, but that didn't help.

Thank you for triggering this
Now I will see if I can get it to 10Gb

 

[Davvo]

This resource might help you then.

Resource - High Speed Networking Tuning to maximize your 10G, 25G, 40G networks

Both FreeBSD and Linux come by default highly optimized for classic 1Gbps ethernet. This is by far the most commonly deployed networking for both clients and servers, and a lot of research has been done to tune performance especially for local...

www.truenas.com

 

[EtienneB]

This resource might help you then.

Resource - High Speed Networking Tuning to maximize your 10G, 25G, 40G networks

Both FreeBSD and Linux come by default highly optimized for classic 1Gbps ethernet. This is by far the most commonly deployed networking for both clients and servers, and a lot of research has been done to tune performance especially for local...

www.truenas.com

Thanks, will do!