Moving directories within the same share and dataset is really slow...?

[Stilez]

My server (clean install 9.10.2-u1, hardware is Supermicro with a fast 1620-v3 CPU and 96GB 2400 ECC) has a single dataset, with a single SMB share on one directory in it. The NIC for this link is an onboard Intel Gb with MTU=9000. The datastore is on Enterprise 6tb spinners and is mostly empty, with an Intel NVMe ZIL (although I don't think that's relevant to samba?)

I've tried to optimise basic SMB LAN performance, so speeds between my old Windows server and FreeNAS aren't too bad (80MB/s for large files, 30-50MB/s for large collections of mixed size and small files) - I will try to improve this but for now at least it's not dragging its feet too much, and in both cases routinely hit 600-800 mbps on Gb ethernet while copying across the LAN.

Now I'm trying to organise the files copied to the server, and it's a complete nightmare. I just tried to move 4 top-level folders in the share (/movies, /tv, /music, /books) to be subdirs of a new top level folder /media (i.e., /media/movies, /media/tv, etc) using Explorer on Windows 8.1 x64, and suddenly it's crawling or freezing. It took about 5 minutes to get going beyond "calculating" and another 5-10 minutes to move the 4 dirs. I gather than with Win8+ samba allows moves to be done purely "server side", but if so I didn't see any sign of server-side speed. There were no open files or stale SMB sessions which could have been locking content.

Relevant non-default configs are:

** SMB service**

Log level: minimum
Unix extensions
Versions: min=SMB2, max=SMB3
Auxiliary parameters:
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE
getwd cache = yes
case sensitive = yes
preserve case = yes
short preserve case = yes
csc policy = disable
reject md5 clients = yes
reject md5 servers = yes
store dos attributes = no
ea support = no
map hidden = no

** Tunables (all set as sysctls, comments added) **

kern.ipc.maxsockbuf =33554432 - using 10G and plenty of memory
kern.random.sys.harvest.ethernet = 0 - using network for entropy said to slow SMB if using 10G
kern.random.sys.harvest.interrupt = 0
kern.random.sys.harvest.point_to_point = 0
net.inet.raw.maxdgram = 57344
net.inet.tcp.recvbuf_max = 16777216
net.inet.tcp.recvspace = 1048576
net.inet.tcp.sendbuf_max = 16777216
net.inet.tcp.sendspace = 1048576
net.inet.tcp.blackhole = 2
net.inet.udp.blackhole = 1

Any ideas what's going on, and how to improve it? Are there trace or debug functions I can use to get a sense where the holdup is, if it's not obvious to someone with more experience?

 

[JohnDigital]

Can you rule out Windows as a factor? SSH in and move the directories using CLI? Should be nearly instant I could imagine. Possible move to windows temp directory and then move back to FreeNAS that is causing it??. Not much of a Windows user so I cant help alot there. Also I thought I read somewhere that setting MTU to 9000 is all but over. Certainly ALL MTU's have to be set to 9000 on EVERY LINK to see any benefit from a MTU of 9k, if only the one Intel NIC is set, that is almost certainly the problem. I for a fact have a much less powered (mostly default) system and my (large) files move/copy at 116MB/s with MTU's all at default of 1500. Something for you to think about. I dont think 80 or 90MB is too bad over a GB link, but 116 is at the very max on a lower powered system. Try running the network at defaults if all else fails. That is my best guess. Good luck!