Ridiculously slow transfer speeds (2-3MBit/s)

[GriffODoodle]

I am new to TrueNAS just over the last few months set up a system and learning as I go. I had hardware issues with the motherboard on the (15 year old) PC I was using, so I got a slightly newer one and the network speeds are so slow I can't do anything useful with it.

General Hardware:
Intel i7 4770 machine
16GB RAM
512GB SSD for OS etc
4x4GB HDD raid-z1

Network Hardware (hardwired):
Network Cable: Cat6
Network Switch: 1GBit
Ethernet controller (per lspci): Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)

Troubleshooting steps:
iperf results: 1.48Mbits/sec (click for more details)

Show

------------------------------------------------------------
root@truenas[~]# iperf -c 192.168.0.100 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.0.100, TCP port 5001
TCP window size: 32.8 KByte (default)
------------------------------------------------------------
[ 1] local 192.168.0.158 port 62364 connected with 192.168.0.100 port 5001
[ ID] Interval Transfer Bandwidth
[ 1] 0.00-11.97 sec 12.8 MBytes 8.99 Mbits/sec
[ 2] local 192.168.0.158 port 5001 connected with 192.168.0.100 port 62031
[ ID] Interval Transfer Bandwidth
[ 2] 0.00-16.30 sec 2.88 MBytes 1.48 Mbits/sec

I tried replacing the network cable, connecting directly to the router, iperf between other devices on the network, putting another computer in the same place as the NAS on the same cable etc
Essentially everything I could to confirm it's definitely the NAS that is getting poor speeds.

What can I try next?
I know the Realtek NICs are generally not great with TrueNAS, I see other comments about them being limited to 100Mbit/sec, but right now that would be a huge jump up for me! I am going to get another one when I can, but I don't know if that's the bottleneck and would welcome any advice on anything else I can try until then (or even just confirmation that replacing the NIC will solve the issue).

 

[Arwen]

You don't list the drive manufacturer and model. This can make a serious difference because some HD vendors polluted their NAS disk lines with SMR, (Shingled Magnetic Recording), drives.

You list "512GB SSD for OS etc", what does that mean?
Did you partition it manually?

Further, it is a known FACT that Realtek Ethernet chips are lower performing than server style from Chelsio or Intel. Even a cheap desktop Intel 1Gbps Ethernet card performs better than Realtek Ethernet chips. Now will this solve all your problems? Probably not as you rightfully point out.

Last, on rare occasions the other end also plays a factor. What is the other end, a desktop?

 

[GriffODoodle]

Hi Arwen, Thanks for the reply, I'm definitely happy to provide more info, just wasn't sure what was needed.

You don't list the drive manufacturer and model. This can make a serious difference because some HD vendors polluted their NAS disk lines with SMR, (Shingled Magnetic Recording), drives.

These are HP 4TB drives as the cheapest SATA drives I could find on ebay (HPE Model MB4000GCWDC).

You list "512GB SSD for OS etc", what does that mean?
Did you partition it manually?

This is an SSD that I pulled from an old laptop. I was wring it is 240GB: Sandisk SDSSDA240g

Last, on rare occasions the other end also plays a factor. What is the other end, a desktop?

The other end for the above test was a desktop and I tried a hardwired laptop as well. I also tried iperf between those 2 devices and was able to achieve much higher speeds (up to 800Mbit/s).

 

[GriffODoodle]

The other end for the above test was a desktop and I tried a hardwired laptop as well. I also tried iperf between those 2 devices and was able to achieve much higher speeds (up to 800Mbit/s).

I can't edit my post (maybe because I'm new) but that was supposed to be 100Mbit/s

 

[Davvo]

If iperf gives you that speed it's likely a networking issue. Can you please run another with iperf3?

What is the route from your NAS to your system? Any switches between them?

Also, please read the following resource.

Is my Realtek ethernet really that bad?

One of the longest running issues faced here in the forums may be people who come to these forums, wanting to recycle an old PC into a NAS. In general, this is a bad idea because typical PC hardware isn't optimized towards the task. However...

www.truenas.com

TL;DR: It can work, but if it doesn't you shouldn't be surprised. In this case however it looks like the issue lies elsewhere.

Please specify you motherboard model as well.

 

[Arwen]

...
These are HP 4TB drives as the cheapest SATA drives I could find on ebay (HPE Model MB4000GCWDC).
...

Hmm, it has been a long time since HP made hard drives. These appear to be OEMed from Western Digital, (WDC). Please get the exact model & firmware from SMART for them. Because Western Digital is one of the vendors that polluted their NAS hard drive line with SMR drives. Plus, some other lines also have SMR drives.

I tried looking up the model, MB4000GCWDC, but was not successful in finding a reference about CMR verses SMR. Perhaps the internal model or firmware version will help.

 

[Davvo]

These appear to be OEMed from Western Digital, (WDC).

This was made from SEAGATE though.

HP MB4000GCWDC - 4TB 7.2K RPM 6G 128MB Cache SATA 3.5" Hard Drive

HP MB4000GCWDC - 4TB 7.2K RPM 6G 128MB Cache SATA 3.5" Hard Drive

www.cpumedics.com

 

[Arwen]

This was made from SEAGATE though.

HP MB4000GCWDC - 4TB 7.2K RPM 6G 128MB Cache SATA 3.5" Hard Drive

HP MB4000GCWDC - 4TB 7.2K RPM 6G 128MB Cache SATA 3.5" Hard Drive

www.cpumedics.com

Hmm, looks like standard Enterprise practice. As long as they are not SMR, that won't be part of the problem.

(Standard Enterprise practice is to use internal naming for external Vendor hardware. Like HPE 4TB drives which might be Seagate or Western Digital. Sun Microsystems did this to have both redundancy in the supply chain. And to keep internal pricing down.)

 

[GriffODoodle]

Thank you for all the help, I opened the device up to find more model numbers, I also changed which port in the switch was being used and then tested with iperf while I was looking up how to install iperf3. When I did, it ran at full speed, and is transferring at ~112MB/s.
I started a larger backup transfer last night and it's still going at full speed today.

So I would like to say it's the port in the switch that it was plugged into that is bad, but I know I had another device in that port and it worked fine. So I'm honestly not convinced I understand the problem. I think just jostling it and turning it off, pulling and replacing all the cables and turning it back on seem to have resolved it for now!

 

[Arwen]

I do have a partial theory, though it does not account for the extreme low speed.

A Gigabit Ethernet connection requires 4 pairs in the cable AND connectors. On the other hand, 10/100Mbit/ps Ethernet only uses 2 of the 4 pairs, one pair for transmit and one pair for receive.

I have personally seen a RJ-45 / 8p8c female jack with bent pins. In that case, it affected 10/100Mbps transmit from the workstation. But, it is possible that if another pair were affected it could drop the amount of usable pairs from 4 to 3, and still allow 10/100Mbit/ps Ethernet.

This does not account for why the speed would be so low. Normally if you lost a pair that is only used by Gigabit Ethernet, your computer would re-negotiate to 100Mbit/ps. Yours seemed to get close to 10Mbit/sp;

[ 1] 0.00-11.97 sec 12.8 MBytes 8.99 Mbits/sec

Anyway, glad the problem is solved.