SOLVED Slow SMB/NFS speed

Tomasz Elendt

Dabbler
Joined
Jan 22, 2014
Messages
20
Hi. I have a custom FreeNAS build that serves me well for the last five years. All of the clients in my home network use WiFi so didn't notice any issues until I plugged my Macbook to LAN cable the other day. I noticed transfers of 30-40 MB/s when transfering large files via SMB. That should be fine for WiFi but for LAN I would expect to see values closer to 1Gb/s (so 3-4x times that). I ran quick iperf3 test and this one looked fine (I got around 900 Mb/s). I thought that maybe there's something wrong with one of the disks but dd tests look fine too (235 MB/s write and 272 MB/s read). I thought that maybe this is something related to SMB so I tried NFS -- I got only slightly better results of ~40-50 MB/s (I would still expect to see over 2x this). I'm out of ideas myself - does anyone have an idea what's going on?

My setup:
CPU: Atom C2750 (Supermico A1SAi-2750F)
RAM: 32GB
ZFS Pool: two mirrors striped (2 x 3TB and 2 x 8TB, all WD Reds) with encryption
Network: 4 x 1Gbe LACP LAGG connected to my switch (HP OfficeConnect 1820-24G); Macbook connected via USB to a single 1Gbe LAN cable.
 
Last edited:

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
653
Hi! In which direction you see slow speeds? From client to NAS or vice versa. Or both ways? Also what is the configured compression alg of the particular dataset (if any)
 

Tomasz Elendt

Dabbler
Joined
Jan 22, 2014
Messages
20
Hi! In which direction you see slow speeds? From client to NAS or vice versa. Or both ways? Also what is the configured compression alg of the particular dataset (if any)

Ah yes, forgot to mentioned that. The transfer speeds I shared (30-40 MB/s for SMB and 40-50 MB/s for NFS) are from NAS to Mac client.
The speeds from client to NAS are also poor I believe -- I will test it in the evening.

I don't have any compression nor dedup enabled on this dataset. I have encryption enabled on the pool though (but that should not matter much I think given the CPU and disk speeds).

The dd tests I ran with the following commands:

Code:
[telendt@freenas /mnt/data/test]$ dd if=/dev/zero of=/mnt/data/test/tmp.dat bs=2048k count=50k
51200+0 records in
51200+0 records out
107374182400 bytes transferred in 435.670977 secs (246457047 bytes/sec)
[telendt@freenas /mnt/data/test]$ dd if=/mnt/data/test/tmp.dat of=/dev/null bs=2048k count=50k
51200+0 records in
51200+0 records out
107374182400 bytes transferred in 375.556793 secs (285906644 bytes/sec)


(that writes to/reads from 100GB file, which is much more than available RAM in my system).
 

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
653
C2750 has AES-NI so encryption is not an issue here (i mean to criple the speeds that much).

I am not an Apple person so i need to ask... Do you have any antivirus or anti-<whatever> software with some sort of "real-time" protection? For example Windows bitdefender can cripple the "incomming" speeds significantly as it does realtime scan/heuristic of the incoming files. There is similar thread about that recently posted.

And yes please let us know the speeds from client to NAS. It will help to pinpoint the bottleneck.
 

Tomasz Elendt

Dabbler
Joined
Jan 22, 2014
Messages
20
I don't think so... I don't have any other hardware at hand so will try with some live-USB Linux distro on that Mac in the evening to rule software/OS issues out. If it won't help next thing I'm gonna try it so destroy that LAGG (and use single LAN connection). Thanks, I will keep you updated.
 

Tomasz Elendt

Dabbler
Joined
Jan 22, 2014
Messages
20
So I've just ran test on 2 MacBooks that I have at home, with the same 3rd party USB to Ethernet adapter.
First, I should probably describe my testing methodology - since macOS doesn't display transfer speed in the copy dialogs I used rsync --progress /smb_mount/surce_file /local_dir/target_file. That was a mistake. I switched to Finder (macOS default file manager) and observed transfer speeds reported by FreeNAS in dashboard widget and "Reporting/Network" graph (side note: I see Tx line on if_octets graph, but no Rx). And it looked much better - I was able to get to get ~85 MB/s (with a crazy high CPU utilization by kernel_task). Still not perfect though.
I checked my drawers and found old Thunderbolt to Ethernet adapter. Plugged that into the old MB and voila: I was able to get up to 110 MB/s of download and up to 90 MB/s (still with high CPU utilization though).
I also booted Ubuntu Live CD from that older Mac and were able to get ~70 MB/s of download, although with much lower CPU usage.

So it's not FreeNAS, it's my clients' OS/hardware. (Why I'm getting perfectly fine iperf scores on that crappy 3rd party adapter despite of low SMB transfers remains mystery.)
I should probably test my shares speed with another desktop/server with high quality NIC.
 

Attachments

  • Screenshot 2019-09-26 at 22.41.45.png
    Screenshot 2019-09-26 at 22.41.45.png
    42.1 KB · Views: 480
  • Screenshot 2019-09-26 at 22.52.31.png
    Screenshot 2019-09-26 at 22.52.31.png
    44.7 KB · Views: 471
Top