Not Getting 10G Network Speeds on Windows 10 Client

adeelleo

Dabbler
Joined
Mar 28, 2021
Messages
38
I am facing issues of not getting 10G Speeds when connecting from Windows to TrueNAS.

I did some testing using iperf3 and it seems that i never get more the 2 Gbps on a 10G NIC when i am communicating between TrueNAS and the Windows 10 Machine.

Below are some Performance stats using iPerf3:

When testing the local Network Throughput on the Windows 10 Machine:

1637828151385.png

1637828205197.png


When Testing the local Network Throughput on the TrueNAS Machine:

1637828299007.png

1637828337752.png


When Testing Network Throughput from the Windows 10 Machine to TrueNAS:

1637828470063.png

1637828511217.png



When Testing Network Throughput from TrueNAS to Windows 10 Machine:

1637828719069.png

1637828744127.png


Below is the current hardware Setup:

Windows 10 Machine:
------------------------
Dell T320 Server
Intel(R) Xeon(R) CPU E5-2420 v2 @ 2.20GHz
8GB ECC DDR3 RAM
2 x 146GB 15k SAS Disks for TrueNAS OS
Intel Pro/1000 PT Quad Port Gigabit Adapter
Intel X520 Dual Port 10Gigabit Adapter

TrueNAS Machine:
------------------------
Dell T320 Server
Intel(R) Xeon(R) CPU E5-2420 v2 @ 2.20GHz
12GB ECC DDR3 RAM
2 x 146GB 15k SAS Disks for TrueNAS OS
Intel Pro/1000 PT Quad Port Gigabit Adapter
Intel X520 Dual Port 10Gigabit Adapter
LSI SAS 9201-16e Quad Port 6Gbps SAS HBA
EMC 60 Disks Expansion
40 x 3TB NL-SAS Disks
TrueNAS Version: TrueNAS-12.0-U6.1

Note: I understand that i am using a lot less RAM that required for the setup with respect to the storage but i haven't yet reached to that stage of testing the actual Disk Performance. I seem to have a bottleneck at the Network layer.

I Tried enabling Jumbo Frames on Windows but that seems to have no effect. So i disabled it again.
 

Kailee71

Contributor
Joined
Jul 8, 2018
Messages
110
Ok so suprisingly, for me, the "official" latest iperf binary from iperf.fr (3.1.3 I think) show me a max of ~6Gbit/s on Win10, regardless of what's on the other side (Windows, Truenas, BSD, Linux etc etc). As soon as I used 3.10.1 the speed went to wire speed (near enough 10Gbit/s). That should at least boost your local speeds in Windows. I don't think it will resolve your problem but at least you'll be able to test and show max speed when you have it available.

Of course, finding pre-compiled binaris is trickier, and I can't recommend just downloading and trusting, but I got mine from here

Hope this helps sorting your issues.

Kai.
 

adeelleo

Dabbler
Joined
Mar 28, 2021
Messages
38
Ok so suprisingly, for me, the "official" latest iperf binary from iperf.fr (3.1.3 I think) show me a max of ~6Gbit/s on Win10, regardless of what's on the other side (Windows, Truenas, BSD, Linux etc etc). As soon as I used 3.10.1 the speed went to wire speed (near enough 10Gbit/s). That should at least boost your local speeds in Windows. I don't think it will resolve your problem but at least you'll be able to test and show max speed when you have it available.

Of course, finding pre-compiled binaris is trickier, and I can't recommend just downloading and trusting, but I got mine from here

Hope this helps sorting your issues.

Kai.

Oh. I was using the latest 3.1.3 version as well. I was also curious why it was not reporting the full 10Gbps Speeds. So i guess it was an issue with the latest iperf3 binary. Another strange thing that i noticed was that with the same 10Gbps card installed in the TrueNAS server along with a Quad Port Gigabit Card iperf3 was reporting 34 Gbps of Throughput. Which made no sense to me.

Anyhow changing iperf3 version wont have any impact on my current situation since i am not even reaching anywhere near the current 6 Gbps Throughput that iperf is incorrectly reporting.

I even tried creating a Mirrored Pool using 48 Disks with 24 Mirrors and tested copying data onto it. I was getting at max 140 MB/s speed. Which is consistent with the iperf numbers reported when i run it between the windows 10 machine and TrueNAS server. Which confirms that i am being bottle-necked at the network layer.
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,945
Turn off Antivirus and the firewall on the windows PC and retest.
Previous threads have pointed at these - in particular the AV as being responsible.

Can you test from another machine?
 

Kailee71

Contributor
Joined
Jul 8, 2018
Messages
110
I'd fire up ubuntu from a stick on your troublemaker and see how that fares. Just to exclude hardware issues. Ubuntu is usually pretty good at being quick OOB. Also, is it writes or reads, or both that are slow?

Also - with 48 disks - how are these attached? Maybe you're swamping your PCIe with all the disk activity and nothing's left for your 10GbE.
 

adeelleo

Dabbler
Joined
Mar 28, 2021
Messages
38
Turn off Antivirus and the firewall on the windows PC and retest.
Previous threads have pointed at these - in particular the AV as being responsible.

Can you test from another machine?
I turned off the antivirus and the windows firewall but that only marginally improved the throughput. about 10% improvement only.

When testing Network Throughput locally on the windows 10 machine:

1637911242320.png

1637911260432.png
 

adeelleo

Dabbler
Joined
Mar 28, 2021
Messages
38
Ok so suprisingly, for me, the "official" latest iperf binary from iperf.fr (3.1.3 I think) show me a max of ~6Gbit/s on Win10, regardless of what's on the other side (Windows, Truenas, BSD, Linux etc etc). As soon as I used 3.10.1 the speed went to wire speed (near enough 10Gbit/s). That should at least boost your local speeds in Windows. I don't think it will resolve your problem but at least you'll be able to test and show max speed when you have it available.

Of course, finding pre-compiled binaris is trickier, and I can't recommend just downloading and trusting, but I got mine from here

Hope this helps sorting your issues.

Kai.
Tested the version 3.10.1 from the link that you suggested but for me it did not show any improvement.
Disabling Windows Firewall and Antivirus however did have 10% improvement.
 

adeelleo

Dabbler
Joined
Mar 28, 2021
Messages
38
I'd fire up ubuntu from a stick on your troublemaker and see how that fares. Just to exclude hardware issues. Ubuntu is usually pretty good at being quick OOB. Also, is it writes or reads, or both that are slow?

Also - with 48 disks - how are these attached? Maybe you're swamping your PCIe with all the disk activity and nothing's left for your 10GbE.
Ok I used an Ubuntu Boot CD and tested iperf3 on that using the same hardware and below were the results:

When testing Network Throughput locally on the Ubuntu Machine (Using Same machine which had windows 10 Installed):

1637995575204.png

1637995591735.png


When Testing Network Throughput from the Ubuntu Machine to TrueNAS:

1637996051220.png

1637996068509.png


So there is nothing wrong with the hardware. Also the TrueNAS side of the hardware seems to be working fine.

Is there any additional configuration i need to do at Windows 10 end to improve the performance?
 

Kailee71

Contributor
Joined
Jul 8, 2018
Messages
110
As I thought. Glad to see nowt's wrong with your hardware, but I'm surprised the newer iperf didn't help. The only thing I can now think of is LRO/TSO, I needed to enable these manually when I switched to Intel 10GbE on my clients. You can en-/disable these in the network interface settings;

1638212333467.png

(yes, this host is running on ESXi, with all necessary precautions) Have a play with these and see if it helps.

Cheerio,

Kai.

++++EDIT+++++
This is where the info came from
+++/EDIT+++++
 

Morris

Contributor
Joined
Nov 21, 2020
Messages
120
What is between your Windows computer and the NAS? Have you tried setting both cards to 10-GB rather than auto?
 

adeelleo

Dabbler
Joined
Mar 28, 2021
Messages
38
What is between your Windows computer and the NAS? Have you tried setting both cards to 10-GB rather than auto?
Hi, nothing is in between the two systems. They are connected directly. I have tried both with a DAC Cable and with SFP+ modules and LC-LC Cable. The interface shows up at 10G Link speed so I don't think auto negotiate is the issue.
 

Morris

Contributor
Joined
Nov 21, 2020
Messages
120
Check that the slots are at least 8x PCIe v2 or better. I saw its server hardware so less likely yet worth checking. Also try reseating and then swapping cards.
 

hescominsoon

Patron
Joined
Jul 27, 2016
Messages
449
For maximum performance there's a few tweaks in windows i always do:
1. I leave the firewall on
2. disable UAC
3. disable power management on the nic.(network and internet settings-ethernet-change ethernet options-properties-power management- uncheck all boxes from bottom to top)
4. in the advanced tab turn off anything about green Ethernet, Ethernet lite, energy efficient..blah blah blah.
 

adeelleo

Dabbler
Joined
Mar 28, 2021
Messages
38
Check that the slots are at least 8x PCIe v2 or better. I saw its server hardware so less likely yet worth checking. Also try reseating and then swapping cards.
Already tested that. The Quad Port SAS HBA is installed in PCI Gen3 x16 Slot and the 10G Cards are installed in PCI Gen3 x8 Slots. So PCI Buss bandwidth is also not an issue.
Plus as already highlighted in my previous posts. The issue is specific to windows 10. Since with the exact same hardware setup i am getting full 10G speeds on Linux.
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,945
This is unfortunately not unknown from windows. There have been several threads with exactly this issue. Some fixed by removing AV software firewall software, some unfortunately not
 
Top