Slow Network Performance

[JR Gonzalez]

Freenas 11.1-U6
Dual Xeon 5670
48 GB
4TBx6x2 raidz2
3TBx6 raidz2
em

Other machine has
FreeNAS 11.1-U6
Avoton 2750
32GB
4x1TB raidz
igb

Both Supermicro motherboards.


I’m getting awful transfer speeds using cifs/smb and sftp (so far). FreeNAS can copy locally around 200-300MB/s.

iperf
------------------------------------------------------------
Client connecting to beast, TCP port 5001
TCP window size: 32.8 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.10 port 43370 connected with 192.168.1.20 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 73.9 MBytes 620 Mbits/sec
[ 3] 1.0- 2.0 sec 92.2 MBytes 774 Mbits/sec
[ 3] 2.0- 3.0 sec 95.8 MBytes 803 Mbits/sec
[ 3] 3.0- 4.0 sec 97.2 MBytes 816 Mbits/sec
[ 3] 4.0- 5.0 sec 92.1 MBytes 773 Mbits/sec
[ 3] 5.0- 6.0 sec 88.5 MBytes 742 Mbits/sec
[ 3] 6.0- 7.0 sec 86.6 MBytes 727 Mbits/sec
[ 3] 7.0- 8.0 sec 85.8 MBytes 719 Mbits/sec
[ 3] 8.0- 9.0 sec 94.4 MBytes 792 Mbits/sec
[ 3] 9.0-10.0 sec 98.2 MBytes 824 Mbits/sec
[ 3] 10.0-11.0 sec 99.4 MBytes 834 Mbits/sec
[ 3] 11.0-12.0 sec 102 MBytes 858 Mbits/sec
[ 3] 0.0-12.4 sec 1.11 GBytes 774 Mbits/sec

Looks OK. I’m getting 23MB/s with sftp and a little faster with smb. Am I missing something here?

 

[joeschmuck]

Am I missing something here?

Maybe. Things that come to mind is what type of data are you moving (large fiels, small files). How full is your destination pool? And most importantly try to roll back to U5 or U4 as I have seen a few coplaints about slow SMB on U6, maybe you are affected by this as well.

 

[JR Gonzalez]

Mostly larger files. Neither pool is close to full. Unfortunately a roll back is a bit out of the question. I can’t allow one of them to go out of service. I will just grind through it and keep looking and maybe it is something that is known and may be fixed.

Also. It was sftp as well. It doesn’t seem to be a samba issue.

 

[joeschmuck]

Too bad a rollback is out of the question, it likely will fix the problem if you never had this problem before.

 

[JR Gonzalez]

I will test with a live image using something else and see what happens to absolutely rule out hardware when I have a chance. I did have this issue before though with 9.10 on the same avoton. I’ll keep digging.

 

[rungekutta]

Related to this?
https://forums.freenas.org/index.php?threads/network-issues-after-upgrading-to-11-1-u6.69506/

 

[JR Gonzalez]

Could be. Still investigating. So far the only thing i can think of is zfs / disk issue? No idea yet. I just know my speeds are awful.

 

[JR Gonzalez]

So. Seems like disabling HT on both systems (the FreeNAS and SmartOS) box seemed to work. I have 90-113MB/s transfers in the VMs now. When disabling HT on the FreeNAS box and testing it with a Windows laptop it sustained 113MB/s. Odd. Not sure where the problem is there?

 

[JR Gonzalez]

Ok. That didn't work. Know this is a bit old. Just in case though.... the system had sync on the datasets. I added a slog and it helped tons. I also noticed (with Linux) that mount.cifs uses ntlmv1 auth. This is disabled by default by FreeNAS. You have to explicitly mount it with sec=ntlmv2 in fstab on the Linux machines otherwise (and you don't really notice this unless you have a console open because it doesn't log in Linux syslog for some reason) you will receive signature issues. After adding a slog as well as mounting the smb shares properly: The system easily achieves 100MB/s speeds (full gbit). I know this is old but hopefully someone comes across this forum post when they have the same issues. (Note: I also swapped from SmartOS to ESXi... the issue rested in VMs using sec=ntlm by default across the board).