Abysmal file transfer speed

[Shane_O]

Hi iX Community,

Excited to be here as I've just completed my first FreeNAS build.

The issue: When transferring files from my old external hard drives to the new build, experiencing crawling file transfer speeds. The current external hard drive is USB 3.1. Files in this folder are primarily photos and videos.

What I've done so far to mitigate any bottlenecks:

• Replaced all network cables with Cat-6
• Made sure to check that router is Gigabit-capable
• Using a Gigabit switch for all devices
• After experiencing a re0 watchdog timeout error with the onboard Realtek NIC, installed an Intel Pro 1000 CT (HP 632710-001)

Apologies in advance for the noobness, and I appreciate any insight y'all might be able to provide.

 

[MikeyG]

What are your iperf speeds? What kind of speeds do you get locally from the USB drive to your laptop/desktop? I assume you are doing the transfer via smb?

 

[Shane_O]

What are your iperf speeds? What kind of speeds do you get locally from the USB drive to your laptop/desktop? I assume you are doing the transfer via smb?

Transferring from a Macbook Pro via UNIX. From the USB drive to my laptop, a 1.5GB video file will transfer in a matter of seconds (~5 sec).

Unsure about iperf. Was attempting to dig in late last night, but didn't get very far before calling it quits. What is the most direct way to relay my speeds to you?

 

[Shane_O]

What are your iperf speeds? What kind of speeds do you get locally from the USB drive to your laptop/desktop? I assume you are doing the transfer via smb?

Just ran and got:

[ ID] Interval Transfer Bandwidth
[ 13] 0.0-15.0 sec 166 MBytes 92.9 Mbits/sec
[ 12] 0.0-15.0 sec 166 MBytes 93.0 Mbits/sec
Segmentation fault: 11

 

[MikeyG]

I usually run iperf3 -s on my nas, and then iperf3 -c <nasip> on my client. add -R to test the other direction. It should get you a printout of the speeds over the course of 10 seconds.

Mine looks like this from Windows:

Code:

C:\iperf-3.1.3-win64>iperf3.exe -c nas
Connecting to host nas, port 5201
[  4] local 2604:5500:c078:xxxxxxx port 51827 connected to 2604:5500:c078:xxxxxxx port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   937 MBytes  7.86 Gbits/sec
[  4]   1.00-2.00   sec   903 MBytes  7.58 Gbits/sec
[  4]   2.00-3.00   sec   903 MBytes  7.57 Gbits/sec
[  4]   3.00-3.07   sec  63.1 MBytes  7.27 Gbits/sec

 

[MikeyG]

If you are using NFS, might just want to try an SMB share then. Also do some speed tests directly on the FreeNas just to be sure the drives can keep up, though with 1Gb it shouldn't be an issue.

 

[Shane_O]

If you are using NFS, might just want to try an SMB share then. Also do some speed tests directly on the FreeNas just to be sure the drives can keep up, though with 1Gb it shouldn't be an issue.

Tried setting up an SMB share, estimating about a day for 643 GB transfer.

How would you recommend speed testing on FreeNAS?

 

[MikeyG]

There should be a lot of info out there, but I usually turn off compression on a dataset and use dd which should at least test streaming data read/writes.

I use

Code:

dd if=/dev/zero of=test.dat bs=2048k count=50000

let that run, then when finished:

dd of=/dev/null if=test.dat bs=2048k count=50000

This writes 100GB of data and then reads it back.
Might open a separate window and look at gstat -p in the meantime to monitor disks.

 

[Elliot Dierksen]

Just ran and got:

[ ID] Interval Transfer Bandwidth
[ 13] 0.0-15.0 sec 166 MBytes 92.9 Mbits/sec
[ 12] 0.0-15.0 sec 166 MBytes 93.0 Mbits/sec
Segmentation fault: 11

If iperf is crashing with a segmentation fault, that sounds like some kind of hardware problem to me. Not only that, but you are getting less than 100Mb transfers. What is your LAN infrastructure, and what was running the other side of that iperf test?

 

[MikeyG]

Ah thanks @Elliot Dierksen I didn't look at the iperf closely enough. My brain inserted and extra "9" and I thought he was getting close to gigabit.

 

[Shane_O]

If iperf is crashing with a segmentation fault, that sounds like some kind of hardware problem to me. Not only that, but you are getting less than 100Mb transfers. What is your LAN infrastructure, and what was running the other side of that iperf test?

Just ran iperf again and received:
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 870 MBytes 728 Mbits/sec

• TP-Link 8 Port Gigabit Ethernet Network Switch
• Asus RT-N66U Router
• Cat 6 cables to all devices
• My main computer is a 2017 Macbook Pro

 

[Elliot Dierksen]

Just ran iperf again and received:
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 870 MBytes 728 Mbits/sec

That is certainly better, but still not great. My rule of thumb is if you can get 90-95% of the theoretical speed, you are doing well. What was the other side of the iperf connection? As a frame of reference, here is what iperf looks like between my two FreeNAS boxes.
This is on the 1gig side

Code:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.10 GBytes   942 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec                  receiver

And this is on the 10gig side

Code:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.9 GBytes  9.36 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  10.9 GBytes  9.36 Gbits/sec                  receiver

It looks like there is something less than ideal on the LAN side. I would also follow the advice from @mgittelman regarding a speed test locally on the pool.