Why is shared NFS faster than SMB share?

Status
Not open for further replies.

Arch Willingham

Dabbler
Joined
Feb 21, 2017
Messages
27
I have a FreeNAS 11 server set up to share a RAID1 array. For the most part, I use it to have various Windows virtual machines run a backup with the FreeNAS box as the SMB target and they also backup to local drives on a local datastore on the esxi server. It works great and all is well.

A few days ago, I had to kill local drive in the esxi server so I moved the two local datastores to an NFS share on the FreeNAS box and re-added the datastore to the two virtual machines. Those two machines still do their windows backups: one to a "local" drive (which is actually now a datastore on the NFS share and one to a SMB network share on the FreeNAS box.

The weird part is they are vastly different in speed even though they are both traveling over the same network, to the same FreeNAS box using the same FreeNAS drives.


Code:
Computer  Minutes Data -GB			 GB/Minute			 Share
VM1			104			 334.78			 54.93825641			 "local drive" on NFS datastore
VM1			 278			331.96			20.37931894			 SAMBA netwrk share
VM2			 192			696.67		   61.92622222			 "local drive" on NFS datastore
VM2			 432		   696.65		   27.52197531			 SAMBA netwrk share


In each case, NFS is almost double what SMB is. Is that normal?

Arch

System info:

Hostname freenas
Build FreeNAS-11-MASTER-201707070510 (b563983)
Platform Intel(R) Xeon(R) CPU E31240 @ 3.30GHz
Memory 16346MB

Pool info:

Code:
  pool: vol0001
state: ONLINE
  scan: scrub repaired 0 in 7h32m with 0 errors on Sun Jun  4 07:33:01 2017
config:

	NAME											STATE	 READ WRITE CKSUM
	vol0001										 ONLINE	   0	 0	 0
	  mirror-0									  ONLINE	   0	 0	 0
		gptid/7db60122-0b56-11e7-b406-000c29b5a6d4  ONLINE	   0	 0	 0
		gptid/7ec8a881-0b56-11e7-b406-000c29b5a6d4  ONLINE	   0	 0	 0



Output from testparm -vvv
Code:
# Global parameters
[global]
	bind interfaces only = No
	config backend = file
	dos charset = CP437
	enable core files = Yes
	interfaces =
	multicast dns register = Yes
	netbios aliases =
	netbios name = FREENAS
	netbios scope =
	realm =
	server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc, drepl, winbindd, ntp_signd, kcc, dnsupdate, dns
	server string = FreeNAS Server
	share backend = classic
	unix charset = UTF-8
	workgroup = FREENASWRKGRP
	browse list = Yes
	domain master = Auto
	enhanced browsing = Yes
	lm announce = Yes
	lm interval = 60
	local master = Yes
	os level = 20
	preferred master = Auto
	ads dns update = Yes
	allow dns updates = secure only
	dns forwarder =
	dns update command = /usr/local/sbin/samba_dnsupdate
	machine password timeout = 604800
	nsupdate command = /usr/local/bin/samba-nsupdate -g
	rndc command = /usr/sbin/rndc
	spn update command = /usr/local/sbin/samba_spnupdate
	mangle prefix = 1
	mangling method = hash2
	max stat cache size = 256
	stat cache = Yes
	client ldap sasl wrapping = sign
	ldap admin dn =
	ldap connection timeout = 2
	ldap delete dn = No
	ldap deref = auto
	ldap follow referral = Auto
	ldap group suffix =
	ldap idmap suffix =
	ldap machine suffix =
	ldap page size = 1000
	ldap passwd sync = no
	ldap replication sleep = 1000
	ldap server require strong auth = Yes
	ldap ssl = start tls
	ldap ssl ads = No
	ldap suffix =
	ldap timeout = 15
	ldap user suffix =
	lock spin time = 200
	oplock break wait time = 0
	smb2 leases = Yes
	debug class = No
	debug hires timestamp = Yes
	debug pid = No
	debug prefix timestamp = No
	debug uid = No
	ldap debug level = 0
	ldap debug threshold = 10
	log file =
	logging = file
	log level = 2
	max log size = 51200
	syslog = 1
	syslog only = No
	timestamp logs = Yes
	abort shutdown script =
	add group script =
	add machine script =
	add user script =
	add user to group script =
	allow nt4 crypto = No
	delete group script =
	delete user from group script =
	delete user script =
	domain logons = No
	enable privileges = Yes
	init logon delay = 100
	init logon delayed hosts =
	logon drive =
	logon home = \\%N\%U
	logon path = \\%N\%U\profile
	logon script =
	reject md5 clients = No
	set primary group script =
	shutdown script =
	add share command =
	afs token lifetime = 604800
	afs username map =
	allow insecure wide links = No
	async smb echo handler = No
	auto services =
	cache directory = /var/db/samba4
	change notify = Yes
	change share command =
	cluster addresses =
	clustering = No
	config file =
	ctdbd socket =
	ctdb locktime warn threshold = 0
	ctdb timeout = 0
	default service =
	delete share command =
	homedir map = auto.home
	kernel change notify = No
	lock directory = /var/lock
	log writeable files on exit = No
	message command =
	nbt client socket address = 0.0.0.0
	ncalrpc dir = /var/run/samba4/ncalrpc
	NIS homedir = No
	nmbd bind explicit broadcast = Yes
	panic action = /usr/local/libexec/samba/samba-backtrace
	perfcount module =
	pid directory = /var/run/samba
	registry shares = No
	remote announce =
	remote browse sync =
	reset on zero vc = No
	smbd profiling level = off
	state directory = /var/db/samba4
	usershare allow guests = No
	usershare max shares = 0
	usershare owner only = Yes
	usershare path = /var/db/samba4/usershares
	usershare prefix allow list =
	usershare prefix deny list =
	usershare template share =
	utmp = No
	utmp directory =
	wtmp directory =
	addport command =
	addprinter command =
	cups connection timeout = 30
	cups encrypt = No
	cups server =
	deleteprinter command =
	disable spoolss = Yes
	enumports command =
	iprint server =
	load printers = No
	lpq cache time = 30
	os2 driver map =
	printcap cache time = 750
	printcap name = /dev/null
	show add printer wizard = Yes
	cldap port = 389
	client ipc max protocol = default
	client ipc min protocol = default
	client max protocol = default
	client min protocol = CORE
	client use spnego = Yes
	dcerpc endpoint servers = epmapper, wkssvc, rpcecho, samr, netlogon, lsarpc, drsuapi, dssetup, unixinfo, browser, eventlog6, backupkey, dnsserver
	defer sharing violations = Yes
	dgram port = 138
	disable netbios = No
	enable asu support = No
	eventlog list =
	large readwrite = Yes
	lsa over netlogon = No
	max mux = 50
	max ttl = 259200
	max wins ttl = 518400
	max xmit = 16644
	min receivefile size = 0
	min wins ttl = 21600
	name resolve order = lmhosts wins host bcast
	nbt port = 137
	nt pipe support = Yes
	nt status support = Yes
	read raw = Yes
	rpc big endian = No
	rpc server port = 0
	server max protocol = SMB3
	server min protocol = LANMAN1
	server multi channel support = No
	smb2 max credits = 8192
	smb2 max read = 8388608
	smb2 max trans = 8388608
	smb2 max write = 8388608
	smb ports = 445 139
	svcctl list =
	time server = Yes
	unicode = Yes
	unix extensions = Yes
	use spnego = Yes
	web port = 901
	write raw = Yes
	algorithmic rid base = 1000
	allow dcerpc auth level connect = No
	allow trusted domains = Yes
	auth methods =
	check password script =
	client ipc signing = default
	client lanman auth = No
	client NTLMv2 auth = Yes
	client plaintext auth = No
	client schannel = Auto
	client signing = default
	client use spnego principal = No
	dedicated keytab file =
	encrypt passwords = Yes
	guest account = nobody
	kerberos encryption types = all
	kerberos method = default
	kpasswd port = 464
	krb5 port = 88
	lanman auth = No
	log nt token command =
	map to guest = Bad User
	map untrusted to domain = No
	ntlm auth = Yes
	ntp signd socket directory = /var/run/samba4/ntp_signd
	null passwords = No
	obey pam restrictions = Yes
	old password allowed period = 60
	pam password change = No
	passdb backend = tdbsam
	passdb expand explicit = No
	passwd chat = *new*password* %n\n *new*password* %n\n *changed*
	passwd chat debug = No
	passwd chat timeout = 2
	passwd program =
	password hash gpg key ids =
	password server = *
	preload modules =
	private dir = /var/db/samba4/private
	raw NTLMv2 auth = No
	rename user script =
	restrict anonymous = 0
	root directory =
	samba kcc command = /usr/local/sbin/samba_kcc
	security = USER
	server role = standalone server
	server schannel = Auto
	server signing = default
	smb passwd file = /var/db/samba4/private/smbpasswd
	tls cafile = tls/ca.pem
	tls certfile = tls/cert.pem
	tls crlfile =
	tls dh params file =
	tls enabled = Yes
	tls keyfile = tls/key.pem
	tls priority = NORMAL:-VERS-SSL3.0
	tls verify peer = as_strict_as_possible
	unix password sync = No
	username level = 0
	username map =
	username map cache time = 0
	username map script =
	aio max threads = 100
	deadtime = 15
	getwd cache = Yes
	hostname lookups = Yes
	keepalive = 300
	max disk size = 0
	max open files = 470738
	max smbd processes = 0
	name cache timeout = 660
	socket options = TCP_NODELAY
	use mmap = Yes
	get quota command =
	host msdfs = Yes
	set quota command =
	create krb5 conf = Yes
	idmap backend = tdb
	idmap cache time = 604800
	idmap gid =
	idmap negative cache time = 120
	idmap uid =
	include system krb5 conf = Yes
	neutralize nt4 emulation = No
	reject md5 servers = No
	require strong key = Yes
	template homedir = /home/%D/%U
	template shell = /bin/false
	winbind cache time = 300
	winbindd privileged socket directory = /var/run/samba4/winbindd_privileged
	winbindd socket directory = /var/run/samba4/winbindd
	winbind enum groups = No
	winbind enum users = No
	winbind expand groups = 0
	winbind max clients = 200
	winbind max domain connections = 1
	winbind nested groups = Yes
	winbind normalize names = No
	winbind nss info = template
	winbind offline logon = No
	winbind reconnect delay = 30
	winbind refresh tickets = No
	winbind request timeout = 60
	winbind rpc only = No
	winbind sealed pipes = Yes
	winbind separator = \
	winbind trusted domains only = No
	winbind use default domain = No
	dns proxy = No
	wins hook =
	wins proxy = No
	wins server =
	wins support = No
	idmap config *: range = 90000001-100000000
	idmap config * : backend = tdb
	comment =
	path =
	administrative share = No
	browseable = Yes
	case sensitive = Auto
	default case = lower
	delete veto files = No
	hide dot files = Yes
	hide files =
	hide special files = No
	hide unreadable = No
	hide unwriteable files = No
	mangled names = Yes
	mangling char = ~
	map archive = Yes
	map hidden = No
	map readonly = yes
	map system = No
	preserve case = Yes
	short preserve case = Yes
	store dos attributes = Yes
	veto files =
	veto oplock files =
	blocking locks = Yes
	csc policy = manual
	fake oplocks = No
	kernel oplocks = No
	kernel share modes = Yes
	level2 oplocks = Yes
	locking = Yes
	oplock contention limit = 2
	oplocks = Yes
	posix locking = Yes
	strict locking = No
	afs share = No
	available = Yes
	copy =
	delete readonly = No
	dfree cache time = 0
	dfree command =
	directory name cache size = 0
	dmapi support = No
	dont descend =
	dos filemode = Yes
	dos filetime resolution = No
	dos filetimes = Yes
	fake directory create times = No
	follow symlinks = Yes
	fstype = NTFS
	include =
	magic output =
	magic script =
	postexec =
	preexec =
	preexec close = No
	root postexec =
	root preexec =
	root preexec close = No
	spotlight = No
	volume =
	wide links = No
	cups options =
	default devmode = Yes
	force printername = No
	lppause command =
	lpq command = lpq -P'%p'
	lpresume command =
	lprm command = lprm -P'%p' %j
	max print jobs = 1000
	max reported print jobs = 0
	printable = No
	print command = lpr -r -P'%p' %s
	printer name =
	printing = bsd
	printjob username = %U
	print notify backchannel = No
	queuepause command =
	queueresume command =
	use client driver = No
	acl allow execute always = Yes
	acl check permissions = Yes
	acl map full control = Yes
	durable handles = Yes
	ea support = Yes
	map acl inherit = No
	nt acl support = Yes
	profile acls = No
	access based share enum = No
	acl group control = No
	admin users =
	create mask = 0666
	directory mask = 0777
	force create mode = 0000
	force directory mode = 0000
	force group =
	force unknown acl user = No
	force user =
	guest ok = No
	guest only = No
	hosts allow =
	hosts deny =
	inherit acls = No
	inherit owner = no
	inherit permissions = No
	invalid users =
	read list =
	read only = Yes
	smb encrypt = default
	valid users =
	write list =
	aio read size = 0
	aio write behind =
	aio write size = 0
	allocation roundup size = 1048576
	block size = 1024
	max connections = 0
	min print space = 0
	strict allocate = No
	strict rename = No
	strict sync = No
	sync always = No
	use sendfile = No
	write cache size = 0
	msdfs proxy =
	msdfs root = No
	msdfs shuffle referrals = No
	ntvfs handler = unixuid, default
	vfs objects =


[smbdata]
	path = "/mnt/vol0001/smbdata"
	veto files = /.snapshot/.windows/.mac/.zfs/
	read only = No
	vfs objects = zfs_space zfsacl streams_xattr aio_pthread
	zfsacl:acesort = dontcare
	nfs4:chown = true
	nfs4:acedup = merge
	nfs4:mode = special
 
Last edited by a moderator:

millst

Contributor
Joined
Feb 2, 2015
Messages
141
My general experience has been that NFS outperforms SMB, especially as file size decreases and file count increases. That seems to be the trend from what I've seen online, but some people have the opposite experience.

At home, my C2758 system only uses SMB and has no problem saturating GBE with large files from a mirror. Your results are much less, but that's probably a mix of files. For comparison, trying some tests with large files might be worthwhile.

SMB is single threaded per connection. Your CPU is way more powerful than mine. Make sure your VMs are not limiting CPU usage too much.

-tm
 
Status
Not open for further replies.
Top