Can't change network options

[Mlovelace]

What is the best approach when I start with the tunables again?

I guess:

1. deactivate autotune if it's on and remove all tunables from the system tab. --> reboot

With which tunable do I start? Is there a "strategy"?
Do I need to reboot after each tunable or is there a way to do it without reboot?

Don't use autotune, if you have, as you stated delete any of the settings and reboot. Here are some of the tunables I've done on one of our system with a chelsio T520-CR card.

You can ignore the "hw.mps.max_chains" loader as it's related to an LSI HBA card. Typically if it's a sysctl your system doesn't require a reboot, if it's a loader the system will require a reboot. I've found the htcp congestion control to be better than the default newreno, which is the top loader cc_htcp_load. Start with these setting, though the window sizes might be bigger than you need. The value for the window sizes are in bytes (33554432 = 32MB) just fyi.

 

[Nik]

Don't use autotune, if you have, as you stated delete any of the settings and reboot. Here are some of the tunables I've done on one of our system with a chelsio T520-CR card.
View attachment 24842

You can ignore the "hw.mps.max_chains" loader as it's related to an LSI HBA card. Typically if it's a sysctl your system doesn't require a reboot, if it's a loader the system will require a reboot. I've found the htcp congestion control to be better than the default newreno, which is the top loader cc_htcp_load. Start with these setting, though the window sizes might be bigger than you need. The value for the window sizes are in bytes (33554432 = 32MB) just fyi.

First of all, thanks for helping me.

I've removed the old sysctls and added the ones from you.

This is the outcome:



Tested with: iperf.exe -c 10.16.1.34 -P 1 -i 1 -p 5001 -f k -t 40

Before it was pretty linear around 3.75GB not it's going higher but flipping wild around.

How would I know where to start getting the rate more stable?

 

[Elliot Dierksen]

Before it was pretty linear around 3.75GB not it's going higher but flipping wild around.

That graph looks kind of like a tail drop, but it could also be that you need a larger still window size. It seems plausible that the upticks in the graph are as it bursts up to the window size, and then it ticks back down as it waits for the acknowledgement. Try throttling that up, and see if the performance graph follows the increase in window. Sorry if you already know this, but the window size is how much data it will leave unacknowledged. This is particularly an issue over greater distances where the link is fast, but the window size is small so the hosts in question won't use all the bandwidth because they stop sending because the window if full and they haven't yet received an acknowledgement. I haven't had to use iperf in some time, but does it (or can it) tell you the TCP values a hosts is using? If so, you could draw some inferences about the values to use for FreeNAS based on the values from hosts that can fill the link.

 

[Nik]

That graph looks kind of like a tail drop, but it could also be that you need a larger still window size. It seems plausible that the upticks in the graph are as it bursts up to the window size, and then it ticks back down as it waits for the acknowledgement. Try throttling that up, and see if the performance graph follows the increase in window. Sorry if you already know this, but the window size is how much data it will leave unacknowledged. This is particularly an issue over greater distances where the link is fast, but the window size is small so the hosts in question won't use all the bandwidth because they stop sending because the window if full and they haven't yet received an acknowledgement. I haven't had to use iperf in some time, but does it (or can it) tell you the TCP values a hosts is using? If so, you could draw some inferences about the values to use for FreeNAS based on the values from hosts that can fill the link.

I can set transport layer options in iperf.



but it doesn't tell you the values the host is using.

I have played a bit around with the tunable values, but I can't see any change. Unless a reboot after each change is mandatory, then this would be the reason. But I checked in the console for the sysctl values and the ones I set in the tunables were set correctly in the sysctls.

Any more ideas guys?

 

[Elliot Dierksen]

I don't know if a reboot is required for a change to the TCP window size. If it is possible, try it just to see if it changes anything.