Network Speed Concern

lmannyr · Jun 16, 2021

Home gigabit network setup.
FreeNAS 11.3
RAID Z2 with 37% used of 40TB

Im transferring an iPhoto directory via rsync from the server onto a local desktop. The network interface reporting page says rate is at an average of 600kb/s. Does this speed over a gigabit LAN seem right?

Thanks

JohnDigital · Jun 16, 2021

No, you have something a miss id estimate. Pictures notoriously transfer slower because they are smaller, but 640kb is definitely low. You have two network ports in that board id try the other one along with different cables, verify connections etc..

JohnDigital · Jun 16, 2021

iperf is a common tool to test network bandwidth

lmannyr · Jun 16, 2021

Iperf says about 70mbits/sec.

Restarted the rsync transfer and its running in the 40-60kbits/s range......

is it rsync???

Code:

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  8.49 MBytes  71.2 Mbits/sec                 
[  4]   1.00-2.00   sec  8.49 MBytes  71.2 Mbits/sec                 
[  4]   2.00-3.00   sec  8.31 MBytes  69.7 Mbits/sec                 
[  4]   3.00-4.00   sec  8.53 MBytes  71.6 Mbits/sec                 
[  4]   4.00-5.00   sec  8.40 MBytes  70.4 Mbits/sec                 
[  4]   5.00-6.00   sec  8.55 MBytes  71.7 Mbits/sec                 
[  4]   6.00-7.00   sec  8.48 MBytes  71.2 Mbits/sec                 
[  4]   7.00-8.00   sec  8.51 MBytes  71.4 Mbits/sec                 
[  4]   8.00-9.00   sec  8.27 MBytes  69.4 Mbits/sec                 
[  4]   9.00-10.00  sec  8.29 MBytes  69.6 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  84.3 MBytes  70.7 Mbits/sec                  sender
[  4]   0.00-10.00  sec  84.3 MBytes  70.7 Mbits/sec                  receiver

Spearfoot · Jun 16, 2021

lmannyr said:

It's not just rsync. You should be getting more than 900 Mbits/sec over a Gigabit connection with iperf, so something is definitely wrong with your network setup.

You have a Supermicro X10SL7-F -- so do I! -- which has Intel Ethernet ports. Tell us more about the rest of your network setup; what kind of Ethernet port does the 'local desktop' have? Are the systems connected to a switch? If so, what brand and model?

lmannyr · Jun 16, 2021

Spearfoot said:
It's not just rsync. You should be getting more than 900 Mbits/sec over a Gigabit connection with iperf, so something is definitely wrong with your network setup.

You have a Supermicro X10SL7-F -- so do I! -- which has Intel Ethernet ports. Tell us more about the rest of your network setup; what kind of Ethernet port does the 'local desktop' have? Are the systems connected to a switch? If so, what brand and model?

FN connected to a Netgear GS308 switch. At this switch the smart TV, XBOX, Apple TV, UPS, and 3 ports from FN is connected to it. This switch is connected to a TP-LINK Switch in the office. The office switch is connected to the ONLY LAN connected desktop (the one receiving the files from the server), pfSense, and wifi router. The desktop has a Speedbyte Gigabit Ethernet PCI Card.

iperf on a wifi connected laptop. with FN as server... it averaged 70Mbits/s.

iperf client on wifi connected laptop to desktop iperf server...it averaged 630Mbits/s

Something is up.

lmannyr · Jun 16, 2021

Why fast in only one direction?

Desktop as iperf server
FN as iperf client.

Code:

# iperf3 -c 192.168.desktop
Connecting to host 192.168.desktop, port 5201
[  5] local 192.168.FN port 23459 connected to 192.168.desktop port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   115 KBytes   939 Kbits/sec   15   5.69 KBytes       
[  5]   1.00-2.00   sec  90.5 KBytes   740 Kbits/sec   12   5.69 KBytes       
[  5]   2.00-3.01   sec  63.6 KBytes   520 Kbits/sec    9   5.69 KBytes       
[  5]   3.01-4.01   sec  97.6 KBytes   798 Kbits/sec   12   5.68 KBytes       
[  5]   4.01-5.01   sec  59.4 KBytes   485 Kbits/sec    9   4.27 KBytes       
[  5]   5.01-6.01   sec  41.0 KBytes   336 Kbits/sec    7   5.69 KBytes       
[  5]   6.01-7.01   sec  25.5 KBytes   208 Kbits/sec    6   2.84 KBytes       
[  5]   7.01-8.01   sec  43.8 KBytes   359 Kbits/sec    6   5.69 KBytes       
[  5]   8.01-9.01   sec   117 KBytes   961 Kbits/sec   16   5.68 KBytes       
[  5]   9.01-10.00  sec  39.6 KBytes   327 Kbits/sec    9   1.41 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   693 KBytes   568 Kbits/sec  101             sender
[  5]   0.00-10.00  sec   662 KBytes   542 Kbits/sec                  receiver

iperf Done.

Next I tried it the other way around again..
FN is iperf server
Desktop is iperf client

Code:

% ./iperf3 -c 192.168.FN
Connecting to host 192.168.FN, port 5201
[  4] local 192.168.desktop port 60011 connected to 192.168.FS port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  8.56 MBytes  71.7 Mbits/sec                 
[  4]   1.00-2.00   sec  8.23 MBytes  69.1 Mbits/sec                 
[  4]   2.00-3.00   sec  8.14 MBytes  68.2 Mbits/sec                 
[  4]   3.00-4.00   sec  8.54 MBytes  71.7 Mbits/sec                 
[  4]   4.00-5.00   sec  8.42 MBytes  70.7 Mbits/sec                 
[  4]   5.00-6.00   sec  8.51 MBytes  71.3 Mbits/sec                 
[  4]   6.00-7.00   sec  8.56 MBytes  71.8 Mbits/sec                 
[  4]   7.00-8.00   sec  8.50 MBytes  71.4 Mbits/sec                 
[  4]   8.00-9.00   sec  8.46 MBytes  70.9 Mbits/sec                 
[  4]   9.00-10.00  sec  8.38 MBytes  70.3 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  84.3 MBytes  70.7 Mbits/sec                  sender
[  4]   0.00-10.00  sec  84.2 MBytes  70.7 Mbits/sec                  receiver

iperf Done.

sretalla · Jun 17, 2021

OK, how about we look at dmesg | grep 100 on your FN server and see if the NIC is connecting at 100 Mbits.

You could be looking at a duplex/speed mismatch between server and switch.

lmannyr · Jun 17, 2021

sretalla said:
OK, how about we look at dmesg | grep 100 on your FN server and see if the NIC is connecting at 100 Mbits.

You could be looking at a duplex/speed mismatch between server and switch.

Outputs this...

Code:

AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
Timecounter "TSC-low" frequency 1750032454 Hz quality 1000
Event timer "i8254" frequency 1193182 Hz quality 100
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xc000-0xc01f mem 0xf7400000-0xf747ffff,0xf7480000-0xf7483fff irq 18 at device 0.0 on pci5
igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xb000-0xb01f mem 0xf7300000-0xf737ffff,0xf7380000-0xf7383fff irq 19 at device 0.0 on pci6
umass0:  SCSI over Bulk-Only; quirks = 0x8100
umass1:  SCSI over Bulk-Only; quirks = 0x8100
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>

sretalla · Jun 17, 2021

OK, we might just want to confirm it and run dmesg | grep igb but it looks like that may not be the case at the FN end.

Check on the switch (maybe with a green/yellow light indication) if it thinks it's 1Gbit or 100Mbit.

Are you using both NICs?

lmannyr · Jun 17, 2021

sretalla said:
OK, we might just want to confirm it and run dmesg | grep igb but it looks like that may not be the case at the FN end.

Outputs this....

Code:

# dmesg | grep igb
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xc000-0xc01f mem 0xf7400000-0xf747ffff,0xf7480000-0xf7483fff irq 18 at device 0.0 on pci5
igb0: Using MSIX interrupts with 5 vectors
igb0: Ethernet address: 0c:c4:7a:6a:e0:be
igb0: Bound queue 0 to cpu 0
igb0: Bound queue 1 to cpu 1
igb0: Bound queue 2 to cpu 2
igb0: Bound queue 3 to cpu 3
igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xb000-0xb01f mem 0xf7300000-0xf737ffff,0xf7380000-0xf7383fff irq 19 at device 0.0 on pci6
igb1: Using MSIX interrupts with 5 vectors
igb1: Ethernet address: 0c:c4:7a:6a:e0:bf
igb1: Bound queue 0 to cpu 4
igb1: Bound queue 1 to cpu 5
igb1: Bound queue 2 to cpu 6
igb1: Bound queue 3 to cpu 7
igb0: link state changed to UP
igb0: link state changed to DOWN
igb0: link state changed to UP
igb1: promiscuous mode enabled
igb0: link state changed to DOWN
igb0: promiscuous mode enabled
igb0: link state changed to UP
igb1: link state changed to UP
igb1: link state changed to DOWN
igb0: link state changed to DOWN
igb0: link state changed to UP
igb1: link state changed to UP
igb1: link state changed to DOWN
igb1: link state changed to UP
igb0: link state changed to DOWN
igb1: link state changed to DOWN
igb1: link state changed to UP
igb0: link state changed to UP
igb0: link state changed to DOWN
igb0: link state changed to UP
igb1: link state changed to DOWN
igb0: lagg_port_destroy: lp_ifflags unclean
igb0: link state changed to DOWN
igb1: lagg_port_destroy: lp_ifflags unclean
igb0: link state changed to UP
igb1: link state changed to UP
igb1: link state changed to DOWN
igb1: link state changed to UP
igb0: link state changed to DOWN
igb0: link state changed to UP
igb0: link state changed to DOWN
igb0: link state changed to UP
igb1: link state changed to DOWN
igb1: link state changed to UP

"Pro/1000 Network Connection" is Gigabit right?

sretalla said:
Check on the switch (maybe with a green/yellow light indication) if it thinks it's 1Gbit or 100Mbit.

All lights on both switches are GREEN.

sretalla · Jun 17, 2021

And there we have it... your LAGG is flapping (or are you just unplugging cables?).

We need to look at the config you're using on FN and on the switch for that.

Spearfoot · Jun 17, 2021

lmannyr said:

Your FreeNAS interfaces seem to be going up and down, which isn't good. And it looks like you've configured a LAGG group? Try removing the LAGG group and see if you can get reasonable performance from a single interface.

EDIT: definitely remove the LAGG group; you're using unmanaged switches and LAGG requires LACP support and configuration on the switch.

If will be helpful if you provide us with all of the details of your network setup for both your FreeNAS server and desktop workstation: static/DHCP? Netmask? Gateway? Nameserver(s)? Have you installed any FreeNAS tunables? Have you optimized the NIC settings on your desktop? etc.

Lastly, it may not be the source of your problem... but Intel NICs are reasonably priced these days, so I suggest replacing the no-name NIC you're using in your desktop.

lmannyr · Jun 17, 2021

sretalla said:
And there we have it... your LAGG is flapping.

We need to look at the config you're using on FN and on the switch for that.

Just so I understand you correctly, your asking for the "System --> General -->" backup Config file?

Mlovelace · Jun 17, 2021

lmannyr said:
FN connected to a Netgear GS308 switch.

You have a lagg configured and connected to an unmanaged switch. Problem solved...

lmannyr · Jun 17, 2021

Mlovelace said:
You have a lagg configured and connected to an unmanaged switch. Problem solved...

The LAGG was added to help solve this issue. LAGG was setup as "Failover" as recommended in the manual. It didn't help. For S&G, I deleted the LAGG from interface, and disconnected one of the ethernet cables from the FN box. Same results with iperf.

sretalla · Jun 17, 2021

lmannyr said:
"System --> General -->" backup Config file?

No, we would need to see a screenshot of what you have under Network | Interfaces... but maybe you're already changing that to remove the LAGG, so no need if you've done that.

sretalla · Jun 17, 2021

lmannyr said:
The LAGG was added to help solve this issue

That wouldn't have been the case even with a managed switch, so it's good that it's now off the table.

I think we should still look at how it's set then.

lmannyr · Jun 17, 2021

sretalla said:
No, we would need to see a screenshot of what you have under Network | Interfaces... but maybe you're already changing that to remove the LAGG, so no need if you've done that.

Screen Shot 2021-06-17 at 8.44.31 AM.png

sretalla · Jun 17, 2021

If you're not using the server for anything, I think it might be time for a reboot after having messed with LAGG.

All of that UP/DOWN noise has happened since the last reboot, so it would be nice to see dmesg come up clean.

Important Announcement for the TrueNAS Community.

Network Speed Concern

Contributor

Guru

Guru

Contributor

He of the long foot

Contributor

Contributor

Powered by Neutrality

Contributor

Powered by Neutrality

Contributor

Powered by Neutrality

He of the long foot

Contributor

Guru

Contributor

Powered by Neutrality

Powered by Neutrality

Contributor

Powered by Neutrality

Similar threads