SMB Slow Transfers over 10Gbe but FTP fast

Redmist

Cadet
Joined
May 12, 2016
Messages
9
Hi,

I am considering upgrading for FreeNAS 9.3 to 11.3 but the only thing holding me back is slow SMB speed to my Windows 10 PC. With 9.3 I get about 600 MBytes/s but when I install 11.3, SMB performance drops to about 150-200MBytes/s. I tried benchmarking my pool with dd and it was getting the usual 600MB/s so I tried enabling FTP and it was able to transfer at the full 600 MBytes/s. Are there any SMB settings in 11.3 that could affect large transfers so drastically compared to 9.3?
 

c77dk

Patron
Joined
Nov 27, 2019
Messages
467
It could be different use of "sync" feature - how is the dataset setup regaring sync? If anything else than disabled, you can try looking with "zilstat 2" when doing transfers with ftp and smb and compare the results.
 

piperfect

Dabbler
Joined
Jan 22, 2015
Messages
12
I was getting ~700MBytes/s on 11.3 with a cached SMB transfer to a Windows 10 NVMe drive.
I upgraded to TruNAS 12 now I am only getting ~500MBytes/s
........I should not have upgraded the ZFS on the pool. :(
 

Redmist

Cadet
Joined
May 12, 2016
Messages
9
It could be different use of "sync" feature - how is the dataset setup regaring sync? If anything else than disabled, you can try looking with "zilstat 2" when doing transfers with ftp and smb and compare the results.
I just get all zeros in zilstat with FTP & SMB
 

Redmist

Cadet
Joined
May 12, 2016
Messages
9
Interestingly, if I copy a 5GB file with FTP and then copy it with SMB, it goes extremely fast (~950MBytes/s) To me, that suggests that the problem lies with data getting from the disks to SMB if cache to SMB is so fast.
 

c77dk

Patron
Joined
Nov 27, 2019
Messages
467
I just get all zeros in zilstat with FTP & SMB
Just realized I wasn't thinking when I wrote that - ZIL isn't used in reading. Sorry for that.
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
You can try to set aio read size =0 as an aux parameter for the share 9.3 -> 11.3 is a huge change. In 11.2 samba began to default to using samba's built in AIO, which can have some scaling issues in some cases. Alternatively, you can try out 12.0-RC1 (assuming non-production workload). General reports are that performance is much improved.
 

Redmist

Cadet
Joined
May 12, 2016
Messages
9
I tried 12.0-RC1 but the issue remained. aio read size = 0 in the SMB aux parameters appears to have fixed the problem though! Much appreciated.
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
I have gone through the headache of the 12.0 upgrade a well and had issues with read speeds, went down a hardware rabbit hole! I just tried this smb aux setting and it did seem to solve the issue, read speeds were hitting the max 2.5Gbps which is expected for me. Curious are their any issues with running aio read size =0?
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
I have gone through the headache of the 12.0 upgrade a well and had issues with read speeds, went down a hardware rabbit hole! I just tried this smb aux setting and it did seem to solve the issue, read speeds were hitting the max 2.5Gbps which is expected for me. Curious are their any issues with running aio read size =0?
Depends on the context and what share configuration there is. Typically no downside. It just means you are no longer using AIO for reads (which depending on what aio implementation you're using in samba, may be same config as in 11.3).
 

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
Depends on the context and what share configuration there is. Typically no downside. It just means you are no longer using AIO for reads (which depending on what aio implementation you're using in samba, may be same config as in 11.3).

Thanks. I am probably going to stick with 11.3 when I rebuild my Main NAS due to this bug, even with the fix I feel like 12 just isn't fully baked :( That said, my Backup NAS is on 12 as well and doesn't need this setting to saturate my 2.5Gbps link, it never had issues after the upgrade... only the Main server had this slowdown that was solved with the aio 0 setting... Very strange....

EDIT: I just found an issue on my backup NAS, while copying from a SMB share isn't an issue. I noticed when I tried to extract a zip file (Only 600MB) to my PC it was extremely slow and seemed to do some then stop, then do more... so I figured why not try this setting and it fixe the issue... so seems like the issue was still there just presented itself in a different way on my backup server. Glad the same fix worked for both servers though. Still I plan to stick with 11.3 on my main server until 12 fixes the issue itself.
 
Last edited:

Kuro Houou

Contributor
Joined
Jun 17, 2014
Messages
193
So I saw 12u2 just came out, said it addresses network transfer speeds.. think it fixed this issue I and many others had??
 

Redmist

Cadet
Joined
May 12, 2016
Messages
9
I'm having issues once again with SMB transfer speeds. FTP is typically 500MB/s and SMB is less than 200MB/s. Setting [aio read size = 0] no longer seems to fix the problem. This time around, caching the file makes no difference. Transfers are slow even if I copy a 5GB file twice.
 
Top