NFS share - abysmally slow speeds (<1MB/s) over Ethernet

[Inspecteur]

Hello TrueNAS community!

I've run into an issue that is beyond my knowledge of troubleshooting, and was hoping to get some advice or ideas that may solve it.

I've been using my TrueNas as a backup server for my windows desktop pc for years and had no problems regarding transfer speeds during that time (100MB/s Read and write speeds on average over SMB share).

I recently switched over to linux (Manjaro 21.1.1 Pahvo - Kernel 5.12.19-1-MANJARO if it matters) and followed this guide to set up an NFS share for my existing dataset: https://www.truenas.com/docs/core/sharing/nfs/nfsshare/
I successfully created the NFS share, mounted and accessed my files OK. I noticed that my read and write speeds while connected via ethernet to the NFS share was a terribly slow 0.5MB/s (please see attached perf network tests). It was too slow to transfer large files and too slow to watch videos that were above this bandwidth.

I am running a USB ethernet 100MBit/s adapter (ICS Advent DM9601 Fast Ethernet Adapter) from the linux computer directly into the TrueNAS computer with a Cat5 ethernet cable (I know 100MBit isn't very fast, I ordered a linux compatible Gigabit adapter, but I will not get it for a while and I suspect the problem will continue since the adapter I have has hit 100Mbit/s before via internet usage).

I can confirm I am indeed using this ethernet adapter to connect to my NFS share as my wifi adapter is disabled and I have no other network devices connected to my linux computer. Upon searching the internet and forums for anything similar, I couldn't find anything that showed promise. One suggestion was to delete the volume/dataset in TrueNAS and try creating the share with a fresh new volume. Unfortunately, this is not an option for me as I do not have spare drives or enough free space to copy my data and perform this.

Show : my zpool status

Last login: Sun Aug 29 08:25:07 on pts/2
FreeBSD 12.2-RELEASE-p6 df578562304(HEAD) TRUENAS

TrueNAS (c) 2009-2021, iXsystems, Inc.
All rights reserved.
TrueNAS code is released under the modified BSD license with some
files copyrighted by (c) iXsystems, Inc.

For more information, documentation, help or support, go here:
http://truenas.com
Welcome to FreeNAS

Warning: settings changed through the CLI are not written to
the configuration database and will be reset on reboot.

root@freenas[~]# zpool status
pool: TrueNAS
state: ONLINE
scan: scrub repaired 0B in 14:05:48 with 0 errors on Sun Aug 8 14:05:49 2021
config:

NAME STATE READ WRITE CKSUM
TrueNAS ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/7632ad05-9047-11eb-a775-4ccc6a2ed144 ONLINE 0 0 0
gptid/76c8da3a-9047-11eb-a775-4ccc6a2ed144 ONLINE 0 0 0

errors: No known data errors

pool: boot-pool
state: ONLINE
scan: scrub repaired 0B in 00:00:32 with 0 errors on Sun Aug 29 03:45:32 2021
config:

NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
da0p2 ONLINE 0 0 0

errors: No known data errors
root@freenas[~]#

My TrusNAS system specs are in my signature below as well as a debug log exported from TrueNAS dashboard. (I hope .tgz is okay, let me know if this should be re-exported to a .zip or something)

Are there any settings in TrueNAS that I should be looking into? Any ways to locate where the bottleneck might be? Or is this looking like a problem that should be posted in the manjaro forums?

Thank you for spending the time to read and help me,
- Inspecteur

 

[JohnDigital]

I am running a USB ethernet 100MBit/s adapter (ICS Advent DM9601 Fast Ethernet Adapter) from the linux computer directly into the TrueNAS computer with a Cat5 ethernet cable (I know 100MBit isn't very fast, I ordered a linux compatible Gigabit adapter, but I will not get it for a while and I suspect the problem will continue since the adapter I have has hit 100Mbit/s before via internet usage).

Why are you using this USB adapter? Almost 91% certain this will be your issue. Whats wrong with the onboard ethernet adapter? Its certainly better than anything plugged into a usb port.

The most you can ever expect to get from that adapter is 12.5MBs and that is very generous, if you got more than 11MB id say you were doing good.

 

[Inspecteur]

Hi John,

Thanks for the response. The motherboard I have was given to me in a non-working condition. It turned out a transistor that was in parallel with the bios flashback feature and ethernet controller was the problem. I removed the transistor which made the board useable, but unfortunately disabled my gigabit network controller. Since the motherboard is Mini-ITX and the pci express slot is populated with a dedicated GPU, I don't see any other option but to use a usb ethernet adapter. I heard that hdmi supports ethernet, though I assume that would pass through the on-board network controller, which I assume is disabled with that transistor removed.

This is the usb network adapter I purchased. It says it supports linux and has ok reviews. It should arrive in a couple of days, hopefully this fixes things!

Amazon.ca

 

[JohnDigital]

Ok i figured. Now what do you need a dedicated GPU for? Your CPU supports video. Id 86 that GPU card and throw a dual port intel NIC in there.

 

[Inspecteur]

Sorry, I should have clarified, the usb ethernet adapter is plugged into the linux pc, which is my personal desktop I use for everything. Unfortunately I cannot remove the dedicated graphics card to install a NIC in that computer since I rely on dedicated graphics processing. Although that would solve the slow transfer speed issue, I just can't see it as a feasible solution at this time.

I'm at the point where if a usb network adapter cannot work with this configuration, I'll just have to put up with the slow speeds until I can get a new motherboard (which might not be so easy to come across), or try a couple cheap adapters until one of them works.

Unfortunately, Amazon contacted me yesterday saying they couldn't deliver the usb ethernet adapter I linked in my previous post (never had this happen before heh), so I will try purchasing a different one when I get the chance.

 

[Arwen]

My very old laptop had a 10/100Megabits per second builtin Ethernet port, which while it worked, I thought I might get faster speed from a 1Gigabit per second USB Ethernet device attached to the USB 3 port. So I tried it, and sure enough, I got 20-25Mega BYTES per second data transfers, (about 200/250Mega BITS per second). A good improvement.

In my opinion, unless cost is the deciding factor, no one should buy 10/100Megabits per second USB Ethernet adapters today.

The old 10/100Megabits per second USB devices may only have USB 2 support. That could be a limiting factor. Even if plugged into a USB 3 port on the computer, the USB 3 port on the computer would drop down to USB 2 backward compatible speeds.

Last, make sure your 10/100Megabits per second USB attached Ethernet port is connecting to the switch at 100Megabits per second. In Linux that would be the "ethtool" program. For example, my desktop has;

Code:

> sudo ethtool enp3s0 | egrep "Speed:|Duplex:"
    Speed: 1000Mb/s
    Duplex: Full