Poor CIFS write performance - Freenas 9.3

[m4rv1n]

Hello,
I've a freenas 9.3 box with 4 x 2TB disk in raidZ, connected to my pc with a gigabit connection.
I've created the datastore and shared it by CIFS with LZ4. After slow write performance, i was tried with a new datastore with LZ4 is not active, but the result is the same.
In read access, the speed is about 105MB/s, so it saturate the gigabit connection, but write speed is about 40MB with 50MB maximum speed.
What is wrong? Thank you.

 

[faulerbeamter]

please post your hardware, as it might be the reason for this issue (especially if you don't have enough RAM or power on a single cpu core/thread).
Also, did you choose encryption when you created the zfs pool?

 

[anodos]

Also post /usr/local/etc/smb4.conf

 

[m4rv1n]

BuildFreeNAS-9.3-STABLE-201412240734
CPU G1610T @ 2.30GHz , 16GB RAM

No encryption is used.

[root@storage] /mnt/VolRaidZ# dd if=/dev/zero of=/mnt/VolRaidZ/testfile bs=1G count=1
1+0 records in
1+0 records out
1073741824 bytes transferred in 3.214587 secs (334021706 bytes/sec)

[root@storage] ~# cat /usr/local/etc/smb4.conf
[global]
server max protocol = SMB3_00
interfaces = 127.0.0.1 192.168.1.100
bind interfaces only = yes
encrypt passwords = yes
dns proxy = no
strict locking = no
oplocks = yes
deadtime = 15
max log size = 51200
max open files = 353117
load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes
getwd cache = yes
guest account = nobody
map to guest = Bad User
obey pam restrictions = no
directory name cache size = 0
kernel change notify = no
panic action = /usr/local/libexec/samba/samba-backtrace
server string = FreeNAS Server
ea support = yes
store dos attributes = yes
unix extensions = no
time server = yes
acl allow execute always = false
acl check permissions = true
dos filemode = yes
domain logons = no
local master = yes
idmap config *: backend = tdb
idmap config *: range = 90000001-100000000
server role = standalone
netbios name = STORAGE
workgroup = WORKGROUP
security = user
pid directory = /var/run/samba
smb passwd file = /var/etc/private/smbpasswd
private dir = /var/etc/private
create mask = 0666
directory mask = 0777
client ntlmv2 auth = yes
dos charset = CP437
unix charset = UTF-8
log level = 0

[VolRaidZ]
path = /mnt/VolRaidZ
printable = no
veto files = /.snapshot/.windows/.mac/.zfs/
writeable = yes
browseable = yes
recycle:repository = .recycle/%U
recycle:keeptree = yes
recycle:versions = yes
recycle:touch = yes
recycle:directory_mode = 0777
recycle:subdir_mode = 0700
vfs objects = zfsacl
hide dot files = yes
guest ok = no
nfs4:mode = special
nfs4:acedup = merge
nfs4:chown = true
zfsacl:acesort = dontcare

 

[anodos]

Drop max protocol to SMB2. SMB3 is not entirely stable at this point. You may want to check CPU usage during the writes. Your CPU may be a bottleneck.

 

[Jailer]

poor write speed from what? Windows machine? local hard drive?

 

[m4rv1n]

Switch to SMB2 cause error on access (I can't access, like samba is not enabled)

Jan 9 19:49:38 storage smbd[70778]: [2015/01/09 19:49:38.250159, 0] ../libcli/smb/smb2_signing.c:170(smb2_signing_check_pdu)
Jan 9 19:49:38 storage smbd[70778]: Bad SMB2 signature for message
Jan 9 19:49:38 storage smbd[70778]: [2015/01/09 19:49:38.250350, 0] ../lib/util/util.c:556(dump_data)
Jan 9 19:49:38 storage smbd[70778]: [0000] AE FD E1 2B C7 F9 39 06 67 09 08 5B 53 18 09 C0 ...+..9. g..[S...
Jan 9 19:49:38 storage smbd[70778]: [2015/01/09 19:49:38.250458, 0] ../lib/util/util.c:556(dump_data)
Jan 9 19:49:38 storage smbd[70778]: [0000] 86 25 3F 74 3C 55 12 42 6B 78 4C BE 0F 29 15 3C .%?t.d6 ...N."..
-------------
This is the CPU use
CPU: 6.7% user, 0.0% nice, 45.3% system, 1.2% interrupt, 46.9% idle

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
72240 root 1 52 0 300M 18264K select 0 2:57 56.59% smbd

But sometimes I have this
73098 root 1 26 0 191M 65800K zio->i 0 0:03 6.98% python2.7

or this
72890 root 1 84 0 32780K 10920K CPU1 1 0:14 39.89% bsdtar
------------
Poor write from windows machine with direct connection on dedicated card, from local hard drive I have no problem

[root@storage] /mnt/VolRaidZ# dd if=/dev/zero of=/mnt/VolRaidZ/testfile bs=1G count=1
1+0 records in
1+0 records out
1073741824 bytes transferred in 3.214587 secs (334021706 bytes/sec)

 

[Black Ninja]

Drop max protocol to SMB2. SMB3 is not entirely stable at this point. You may want to check CPU usage during the writes. Your CPU may be a bottleneck.

Is SMB 2.1 stable ? Is it recommended ?

 

[anodos]

Is SMB 2.1 stable ? Is it recommended ?

SMB2 is 2.1

 

[Black Ninja]

SMB2 is 2.1

I am confused with your answer ?! Freenas 9.3 defaults are set to SMB 2.0, before go to SMB 3.0 they are several other version of SMB 2.x you can select. I read the SMB 2.1 has some performance adjustments and is supported by Windows 7.

Are you sure you know what are you talking about ? Perhaps one of us is not !

 

[cyberjock]

See table 8.3b here.. http://olddoc.freenas.org/index.php/CIFS

 

[Black Ninja]

This is what Microsoft says :

SMB 2 was introduced to enable faster communication between computers running Windows Vista® and Windows Server® 2008. SMB 2.1 brought important performance enhancements to the protocol in Windows Server 2008 R2 and Windows® 7.


This is what Freenas read you refer to says :

Table 8.3b: Description of SMB Protocol Versions

Value Description
CORE used by DOS
COREPLUS used by DOS
LANMAN1 used by Windows for Workgroups, OS/2, and Windows 9x
LANMAN2 used by Windows for Workgroups, OS/2, and Windows 9x
NT1 used by Windows NT
SMB2 used by Windows 7; same as SMB2_10
SMB2_02 used by Windows Vista
SMB2_10 used by Windows 7
SMB2_22 used by early Windows 8
SMB2_24 used by Windows 8 beta
SMB3 used by Windows 8
SMB3_00 used by Windows 8, mostly the same as SMB2_24



I don't think this could be understood without "interpreter" from freenas. They trying to make things simpler and easy , but in fact it's quite the opposite results.

They say: "SMB2 - used by Windows 7; same as SMB2_10"

Is this mean that Freenas consider all SMB2.x versions as SMB2.0 ? If that is the case why they will give you 4 versions of smb2.x to select if it doesn't make a difference ?! I doubt that anybody could understand it from this read. No wonder the guy above said "SMB2 is 2.1"
He is confused like the rest of us.

 

[cyberjock]

The chart makes sense to me... SMB2_0 is the Samba default, which also is the same as SMB2_10.

The SMB2_22 and SMB2_24 are basically "beta versions" of SMB2 that was later made into SMB3.

Makes perfect sense to me.

 

[anodos]

The chart makes sense to me... SMB2_0 is the Samba default, which also is the same as SMB2_10.

The SMB2_22 and SMB2_24 are basically "beta versions" of SMB2 that was later made into SMB3.

Makes perfect sense to me.

There is no subprotocol of SMB2 called SMB2_0. If you set server max protocol to SMB2:

• Windows 8 clients will negotiate a connection using the SMB2_10 subprotocol (alternatively referred to as SMB2.1)
• Windows 7 clients will negotiate a connection using the SMB2_10 subprotocol (alternatively referred to as SMB2.1)
• Windows Vista clients will negotiated a connection using the SMB2_04 subprotocol (alternatively referred to as SMB2.04)
• Windows XP, FreeBSD, and most embedded systems will negotiate an SMB1 (or NT1) connection
• Linux flavors that use a modern kernel will use the Linux kernel CIFS client, which can negotiate an SMB2_10 connection.

SMB3 was originally an extension / expansion of SMB2, hence early versions of the protocol are SMB2_24. Then MS decided to be hipster and call it SMB3, which is a good thing because it would be really confusing if there were multiple actively-used subprotocols of SMB2.

To get back to my original point,
(1) In real life any time that someone refers to SMB2 they mean SMB2.1 because no one uses Vista (excluding the possible world where talking unicorns use windows vista to play Half-Life 3 while reading cyberjock's permissions guide). Or to be more direct, SMB2 -like many other things in Vista- was half-baked. Samba follows this convention.
(2) You have now discovered the black hole of code known as samba, where commits are made, modules are added, but nothing escapes (or is removed). :D

 

[Black Ninja]

The chart makes sense to me... SMB2_0 is the Samba default, which also is the same as SMB2_10.

If it's the same why web gui give you an options to chose SMB2.0, SMB2.02 or SMB2.1 ? Just to confuse you ? How that make sense ?

If you set server max protocol to SMB2:

• Windows 8 clients will negotiate a connection using the SMB2_10 subprotocol (alternatively referred to as SMB2.1)
• Windows 7 clients will negotiate a connection using the SMB2_10 subprotocol (alternatively referred to as SMB2.1)
• Windows Vista clients will negotiated a connection using the SMB2_04 subprotocol (alternatively referred to as SMB2.04)
• Windows XP, FreeBSD, and most embedded systems will negotiate an SMB1 (or NT1) connection
• Linux flavors that use a modern kernel will use the Linux kernel CIFS client, which can negotiate an SMB2_10 connection.

SMB3 was originally an extension / expansion of SMB2, hence early versions of the protocol are SMB2_24. Then MS decided to be hipster and call it SMB3, which is a good thing because it would be really confusing if there were multiple actively-used subprotocols of SMB2.

To get back to my original point,
(1) in real life any time that someone refers to SMB2 they mean SMB2.1 because no one uses Vista (excluding the possible world where talking unicorns use windows vista to play Half-Life 3 while reading cyberjock's permissions guide). Or to be more direct, SMB2 -like many other things in Vista- was half-baked.
(2) You have now discovered the black hole of code known as samba, where commits are made, modules are added, but nothing escapes (or is removed). :D

What you are saying make sense. The question is if this is the way freenas guys had in mind when they did it.

I just want my Windows 7 client to negotiate SMB2.1 with Freenas server.
Logically I have to set my max server maximum protocol in freenas to :SMB2_10 to achieve that. But you are saying if I leave freenas to default option which is SMB2, it will still still negotiate SMB2.1 with my Windows 7 client , despite it max set limit to SMB2 ( which I assume is SMB2.0) ?

 

[Black Ninja]

The chart makes sense to me... SMB2_0 is the Samba default, which also is the same as SMB2_10.

If it's the same why web gui give you an options to chose SMB2.0, SMB2.02 or SMB2.1 ? Just to confuse you ? How that make sense ?

 

[anodos]

If it's the same why web gui give you an options to chose SMB2.0, SMB2.02 or SMB2.1 ? Just to confuse you ? How that make sense ?

There is no SMB2.0. I think you are easily confused.

 

[Black Ninja]

There is no SMB2.0. I think you are easily confused.

Right , it says SMB2 not SMB2.0 , I don''t want to repeat it from above, but I already reply with question on you answer above, check and tell me what you response is ?

 

[anodos]

Right , it says SMB2 not SMB2.0 , I don''t want to repeat it from above, but I already reply with question on you answer above, check and tell me what you response is ?

SMB2.1 and SMB2.04 are sub-protocols of SMB2. Selecting "SMB2" has the exact same effect as selecting "SMB2.10".

By the way, this is how samba works. The freenas devs have just exposed the relevant portions of the smb.conf file. See the manpage for smb.conf here: https://www.samba.org/samba/docs/man/manpages-3/smb.conf.5.html.

If this is tripping you up with samba, then I think you're going to have some rough times ahead when you have to set permissions. :D

 

[Black Ninja]

SMB2.1 and SMB2.04 are sub-protocols of SMB2. Selecting "SMB2" has the exact same effect as selecting "SMB2.10".

By the way, this is how samba works. The freenas devs have just exposed the relevant portions of the smb.conf file. See the manpage for smb.conf here: https://www.samba.org/samba/docs/man/manpages-3/smb.conf.5.html.

If this is tripping you up with samba, then I think you're going to have some rough times ahead when you have to set permissions. :D

It wasn't the point to understanding how samba works. The point what to understand the freenas settings:
Why you have 3 options that will do the same thing ?

If you set max protocol "SMB2" in freenas , what is the max SMB protocol that will actually use according to you ?