11.3 Replications, bottlenecking at 1G, iperf3 hits 8.7Gb/s

[bsodmike]

Hi all,

I currently have 2x FreeNAS boxes connected to a UniFi 16XG 10G Switch. I've enabled Jumbo frames and configured both interfaces (on the primary node and the receiving backup node) with MTU set to 9000. iperf3 doesn't seem to have any issues pushing ~8.7Gb/s. However, notice the 1Gb/s traces below (UniFi Dashboard).

On the receiving box:

What more can be done to troubleshoot this further? FYI I only enabled Jumbo frames and made the MTU change after noticing the 1Gb/s speed...

Thanks!

 

[Johnny Fartpants]

The first thing to look at is encryption type and replication compression. Try using a 'Fast' Cipher and no compression on replication.

 

[bsodmike]

The first thing to look at is encryption type and replication compression. Try using a 'Fast' Cipher and no compression on replication.

My results were quoted with 'Standard' Cipher and no compression. Will test with 'Fast' Cipher, in due course - Thanks!!

 

[bsodmike]

I believe I ran into something similar a while back with rsync (over SSH), and it needed options to speed things up; will see if I can find my notes.

 

[Johnny Fartpants]

NETCAT is another option to try.

 

[bsodmike]

Seems `sshd` is running as a single-threaded task and it's pegging the receiving thread to ~98% CPU utilisation (on the 'Standard' Cipher, no compression).