Slow SMB Transfer Speeds Around 5MB/s

[Simon Sparks]

The network card is an INTEL X520-SR2

 

[SweetAndLow]

The network card is an INTEL X520-SR2

Dude you are on thin ice here. You keep avoiding questions that are being asked very directly. On these forums we expect cooperation from people asking questions. Providing the information people ask for is just helping yourself and by avoiding these questions it makes you seem like you don't really care to fix this problem.

provide iperf results and detail description of your network layout. It is clearly your network that is messed up.

 

[Simon Sparks]

iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------

I am attempting to learn iPerf as I write this, its now running on FreeNAS and I have got a copy for my windows client.

 

[m0nkey_]

A wild documentation page appeared! :D http://doc.freenas.org/9.10/cli.html#iperf

 

[SweetAndLow]

iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------

I am attempting to learn iPerf as I write this, its now running on FreeNAS and I have got a copy for my windows client.

OK thanks for running ipef. It's a little worrisome that you don't know how to use it. It's a basic networking tool that should be the first thing used whenever attaching a network device or having any kind of speed problems. This also leads me to believe that there are lots of other networking problems with your system.

 

[Simon Sparks]

FREENAS

iperf -s -w 128k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte
------------------------------------------------------------
[ 4] local 192.168.1.203 port 5001 connected with 192.168.1.31 port 44027
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.3 sec 28.6 MBytes 23.3 Mbits/sec

Windows PC

C:\>netsh advfirewall show allprofiles state

Domain Profile Settings:
----------------------------------------------------------------------
State OFF

Private Profile Settings:
----------------------------------------------------------------------
State OFF

Public Profile Settings:
----------------------------------------------------------------------
State OFF
Ok.


C:\Users\Admin\Downloads\iperf-2.0.5b-win32>iperf.exe -c 192.168.1.203 -p 5001 -f m -w 128k
------------------------------------------------------------
Client connecting to 192.168.1.203, TCP port 5001
TCP window size: 0.12 MByte
------------------------------------------------------------
[ 3] local 192.168.1.31 port 44027 connected with 192.168.1.203 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.3 sec 28.6 MBytes 23.3 Mbits/sec

 

[Simon Sparks]

To prove that this is NOT my Windows PC I did the same test to my Backup FreeNAS box.

FREENAS

[root@freenas-backup] ~# iperf -s -w 128k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte
------------------------------------------------------------
[ 4] local 192.168.1.205 port 5001 connected with 192.168.1.31 port 44059
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 1.10 GBytes 946 Mbits/sec

Windows PC

C:\Users\Admin\Downloads\iperf-2.0.5b-win32>iperf.exe -c 192.168.1.205 -p 5001 -f m -w 128k
------------------------------------------------------------
Client connecting to 192.168.1.205, TCP port 5001
TCP window size: 0.12 MByte
------------------------------------------------------------
[ 3] local 192.168.1.31 port 44059 connected with 192.168.1.205 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1129 MBytes 947 Mbits/sec

 

[m0nkey_]

FREENAS

iperf -s -w 128k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte
------------------------------------------------------------
[ 4] local 192.168.1.203 port 5001 connected with 192.168.1.31 port 44027
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.3 sec 28.6 MBytes 23.3 Mbits/sec

Windows PC

C:\>netsh advfirewall show allprofiles state

Domain Profile Settings:
----------------------------------------------------------------------
State OFF

Private Profile Settings:
----------------------------------------------------------------------
State OFF

Public Profile Settings:
----------------------------------------------------------------------
State OFF
Ok.


C:\Users\Admin\Downloads\iperf-2.0.5b-win32>iperf.exe -c 192.168.1.203 -p 5001 -f m -w 128k
------------------------------------------------------------
Client connecting to 192.168.1.203, TCP port 5001
TCP window size: 0.12 MByte
------------------------------------------------------------
[ 3] local 192.168.1.31 port 44027 connected with 192.168.1.203 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.3 sec 28.6 MBytes 23.3 Mbits/sec

OK, this is a network issue on your FreeNAS box. Just for comparison, I ran the same iperf test as you with the same parameters, to make sure they're not a cause:

Code:

[root@tardis] ~# iperf -s -w 128k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte
------------------------------------------------------------
[  4] local 192.168.32.5 port 5001 connected with 192.168.32.200 port 53265
[ ID] Interval	   Transfer	 Bandwidth
[  4]  0.0-10.0 sec  1.01 GBytes   866 Mbits/sec

Code:

C:\iperf-2.0.9-win64>iperf -c tardis.localnet -p 5001 -f m -w 128k
------------------------------------------------------------
Client connecting to tardis.localnet, TCP port 5001
TCP window size: 0.12 MByte
------------------------------------------------------------
[  3] local 192.168.32.200 port 53265 connected with 192.168.32.5 port 5001
[ ID] Interval	   Transfer	 Bandwidth
[  3]  0.0-10.0 sec  1033 MBytes   867 Mbits/sec

 

[Simon Sparks]

interface Port-channel9
description Uplink to FreeNAS Primary 4x1GbE
switchport mode access
end

interface GigabitEthernet1/0/9
description FreeNAS - Primary Management
switchport mode access
channel-protocol lacp
channel-group 9 mode active
end

interface GigabitEthernet1/0/21
description FreeNAS - Primary Management
switchport mode access
channel-protocol lacp
channel-group 9 mode active
end

interface GigabitEthernet2/0/9
description FreeNAS - Primary Management
switchport mode access
channel-protocol lacp
channel-group 9 mode active
end

interface GigabitEthernet2/0/21
description FreeNAS - Primary Management
switchport mode access
channel-protocol lacp
channel-group 9 mode active
end

interface Port-channel10
description Uplink to FreeNAS Backup 2 x 1GbE
switchport mode access
end

interface GigabitEthernet1/0/10
description FreeNAS - Backup Management
switchport mode access
channel-protocol lacp
channel-group 10 mode active
end

interface GigabitEthernet2/0/10
description FreeNAS - Backup Management
switchport mode access
channel-protocol lacp
channel-group 10 mode active
end

 

[m0nkey_]

interface Port-channel9
description Uplink to FreeNAS Primary 4x1GbE
switchport mode access
end

interface GigabitEthernet1/0/9
description FreeNAS - Primary Management
switchport mode access
channel-protocol lacp
channel-group 9 mode active
end

interface GigabitEthernet1/0/21
description FreeNAS - Primary Management
switchport mode access
channel-protocol lacp
channel-group 9 mode active
end

interface GigabitEthernet2/0/9
description FreeNAS - Primary Management
switchport mode access
channel-protocol lacp
channel-group 9 mode active
end

interface GigabitEthernet2/0/21
description FreeNAS - Primary Management
switchport mode access
channel-protocol lacp
channel-group 9 mode active
end

interface Port-channel10
description Uplink to FreeNAS Backup 2 x 1GbE
switchport mode access
end

interface GigabitEthernet1/0/10
description FreeNAS - Backup Management
switchport mode access
channel-protocol lacp
channel-group 10 mode active
end

interface GigabitEthernet2/0/10
description FreeNAS - Backup Management
switchport mode access
channel-protocol lacp
channel-group 10 mode active
end

I'm not a Cisco IOS expert, but first thing to try is disable the LAGG and only run with one NIC attached. See if you get the same result or any improvement. If it improves, then focus on the LAGG. If there is no improvement, then it's probably a bad NIC, cable or switch port.

 

[Spearfoot]

To prove that this is NOT my Windows PC I did the same test to my Backup FreeNAS box.

FREENAS

[root@freenas-backup] ~# iperf -s -w 128k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte
------------------------------------------------------------
[ 4] local 192.168.1.205 port 5001 connected with 192.168.1.31 port 44059
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 1.10 GBytes 946 Mbits/sec

Windows PC

C:\Users\Admin\Downloads\iperf-2.0.5b-win32>iperf.exe -c 192.168.1.205 -p 5001 -f m -w 128k
------------------------------------------------------------
Client connecting to 192.168.1.205, TCP port 5001
TCP window size: 0.12 MByte
------------------------------------------------------------
[ 3] local 192.168.1.31 port 44059 connected with 192.168.1.205 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1129 MBytes 947 Mbits/sec

Very good! I was going to suggest you do this as the next step.

So there is definitely a problem with the networking setup on this server.

Just to be clear: the NIC in question is bound to the bce driver, so is it a BroadCom NetXtreme II?

You might try removing the LAGG/LACP setup and test with an individual port.

EDIT: I see that @m0nkey_ has beaten me to it! :)

 

[Simon Sparks]

The hardware is a DELL PowerEdge R810 - dmesg output

bce0: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0xf2000000-0xf3ffffff irq 36 at device 0.0 on pci1
bce0: Using defaults for TSO: 65518/35/2048
bce0: Ethernet address: d0:67:e5:f9:fc:4b
bce0: ASIC (0x57092003); Rev (C0); Bus (PCIe x2, 5Gbps); B/C (7.10.0); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)

bce1: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0xf4000000-0xf5ffffff irq 48 at device 0.1 on pci1
bce1: Using defaults for TSO: 65518/35/2048
bce1: Ethernet address: d0:67:e5:f9:fc:4d
bce1: ASIC (0x57092003); Rev (C0); Bus (PCIe x2, 5Gbps); B/C (7.10.0); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)

bce2: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0xf6000000-0xf7ffffff irq 37 at device 0.0 on pci2
bce2: Using defaults for TSO: 65518/35/2048
bce2: Ethernet address: d0:67:e5:f9:fc:4f
bce2: ASIC (0x57092003); Rev (C0); Bus (PCIe x2, 5Gbps); B/C (7.10.0); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)

bce3: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0xf8000000-0xf9ffffff irq 49 at device 0.1 on pci2
bce3: Using defaults for TSO: 65518/35/2048
bce3: Ethernet address: d0:67:e5:f9:fc:51
bce3: ASIC (0x57092003); Rev (C0); Bus (PCIe x2, 5Gbps); B/C (7.10.0); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)

 

[Simon Sparks]

For Comparison this is the backup box which is a DELL PowerEdge R510

bce0: <QLogic NetXtreme II BCM5716 1000Base-T (C0)> mem 0xda000000-0xdbffffff irq 36 at device 0.0 on pci1
bce0: Using defaults for TSO: 65518/35/2048
bce0: Ethernet address: 84:2b:2b:48:6a:5e
bce0: ASIC (0x57092008); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (7.10.0); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)

bce1: <QLogic NetXtreme II BCM5716 1000Base-T (C0)> mem 0xdc000000-0xddffffff irq 48 at device 0.1 on pci1
bce1: Using defaults for TSO: 65518/35/2048
bce1: Ethernet address: 84:2b:2b:48:6a:5f
bce1: ASIC (0x57092008); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (7.10.0); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.13)

 

[Simon Sparks]

dd if=/dev/zero of=/mnt/Data/Homes/SparksS/testfile bs=4M count=10000
10000+0 records in
10000+0 records out
41943040000 bytes transferred in 70.777951 secs (592600371 bytes/sec)

(592600371 bytes/sec) / 1024 = ( 578711 kilobytes/sec )
( 578711 kilobytes/sec ) / 1024 = ( 565 megabytes/sec )

 

[Simon Sparks]

dd if=/dev/zero of=/mnt/VMware/testfile bs=4M count=10000
10000+0 records in
10000+0 records out
41943040000 bytes transferred in 70.954523 secs (591125670 bytes/sec)

(591125670 bytes/sec) / 1024 = ( 577271 kilobytes/sec)
( 577271 kilobytes/sec) / 1024 = ( 563 megabytes/sec )

 

[Simon Sparks]

I am just in the process of getting access back after deleting the lagg I lost access so I am accessing the iDRAC remote console to set an IP on a single interface

 

[Spearfoot]

dd if=/dev/zero of=/mnt/Data/Homes/SparksS/testfile bs=4M count=10000
10000+0 records in
10000+0 records out
41943040000 bytes transferred in 70.777951 secs (592600371 bytes/sec)

(592600371 bytes/sec) / 1024 = ( 578711 kilobytes/sec )
( 578711 kilobytes/sec ) / 1024 = ( 565 megabytes/sec )

Yes, your disk transfer rates seem okay. But we've established that the problem lies with the network setup.

This is an old post, but a BCM5709 NIC user reported problems until s/he disabled "the management firmware with uxdiag":

https://forums.freenas.org/index.php?threads/broadcom-5709-support.8244/

 

[Simon Sparks]

FREENAS

iperf -s -w 128k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 128 KByte
------------------------------------------------------------
[ 4] local 192.168.1.202 port 5001 connected with 192.168.1.31 port 44567
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.3 sec 30.5 MBytes 24.8 Mbits/sec

Windows PC

iperf.exe -c 192.168.1.202 -p 5001 -f m -w 128k
------------------------------------------------------------
Client connecting to 192.168.1.202, TCP port 5001
TCP window size: 0.12 MByte
------------------------------------------------------------
[ 3] local 192.168.1.31 port 44567 connected with 192.168.1.202 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 30.5 MBytes 25.6 Mbits/sec

 

[Simon Sparks]

Just read the user guide for UXDiag and based on my current config I do NOT have the management firmware enabled. It is only enabled if you want to access the iDRAC via a VLAN on the onboard NICs instead of the dedicated port.

 

[Simon Sparks]

So the last iPerf result was for a single NIC not the 4 NIC LACP

The windows transfer speed is still limited to 5MB/s