Slow SMB read performance while having great write performance

[Zofoor]

Hi all!
I am trying to improve my SMB read performance as currently they are terrible.

Some points:

• iperf works without issues in both directions (850-950 mbit).
• writing to a SMB share is very stable 111 MB/sec for a single large file (4.5 GB)
• reading the same large files is very unstable, starting at 110 MB/sec but after a few seconds it goes down to 15-33 MB/sec using a single 4.5 GB file (with no jail/vm or any other kind of activity). Even doing the same thing with the same file multiple times does not help (I would expect the file would be on the ZFS cache). This test has been done using a Windows 10 client writing the data on an SSD drive (50% free space)
• the system has plenty of memory (64 Gb ECC, 8 of them dedicated to a single VM. Currently no jails running for this purpose)
• the pool has 11 disks raidz2, used 65% of the space
• TrueNAS 12.0 U1 (upgrade today, but had the same issues with 11.3 U5)

Those are the additional settings for the SMB service:

Code:

store dos attributes = no
ea support = no
map archive = no
map hidden = no
map system = no
map readonly = no
server min protocol = SMB3
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=131072 SO_SNDBUF=131072 SO_KEEPALIVE
aio read size = 16384
aio write size = 16384
aio max threads = 10

The last 3 lines of those settigs (AIO settings) has been added today to try fixing the issue, but seems they didn't made any improvement.

And those are the settings for the samba share:

Code:

ea support = no
store dos attributes = no
map archive = no
map hidden = no
map readonly = no
map system = no
vfs objects = dirsort

Read performance fo the disks seems good reading a 4,64 GB file:

Code:

dd if=/mnt/main_volume/media/OS/Windows\ 2019\ Essentials/17763.737.190906-2324.rs5_release_svc_refresh_SERVERESSENTIALS_OEM_x64FRE_en-us_1.iso of=/dev/null bs=4M count=100
100+0 records in
100+0 records out
419430400 bytes transferred in 1.364571 secs (307371682 bytes/sec)

Any idea?

 

[Zofoor]

I run the following commands to test the local speed:
smbclient //127.0.0.1/<SHARE> -U <my user>
get large_file /dev/null

Changed a few options and I reached 430mb/sec on a local loop transfer.
#socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=131072 SO_SNDBUF=131072 SO_KEEPALIVE
aio read size = 0
aio write size = 0

So, as iperf has good results, and a local transfer has also good results, then the problem probably is on the client side.
I need to investigate on my Windows clients... Net interface drivers are updated, LSO disabled... the testing Windows client has very good hardware (Dell T330 server).

 

[anodos]

vfs objects = dirsort
^^^ remove this. It's an unsupported parameter, and I've seen it crash samba 4.12.

socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=131072 SO_SNDBUF=131072 SO_KEEPALIVE
^^^ also remove this

Code:

aio read size = 16384
aio write size = 16384

perhaps also remove these.