CIFS directory browsing slow? Try this

[Durkatlon]

The #4 suggestion in the original post (re: DOS attributes) gave my NAS a new lease on life. My production systems run an ancient version of FreeNAS (9.2) and directory listings had gotten progressively slower over the years to the point of major annoyance. After entering the aux params, the problem immediately disappeared. Amazing...

 

[SweetAndLow]

The #4 suggestion in the original post (re: DOS attributes) gave my NAS a new lease on life. My production systems run an ancient version of FreeNAS (9.2) and directory listings had gotten progressively slower over the years to the point of major annoyance. After entering the aux params, the problem immediately disappeared. Amazing...

If you really want to speed things up you can do this, specific to small files but will improve things overall at the same time.

Slow CIFS performance (small files)

 

[cyberjock]

So I'd like to add a little something here... iXsystems is currently tracking a potential bug in FreeBSD causing poor performance with directory listings. I'm working with a few people to hopefully narrow down the issue. I don't want to say too much because I don't know how much I should (or shouldn't) say and I don't want to provide inaccurate information. But one person seems to have noticed it https://forums.freenas.org/index.php?threads/performance-9-3-vs-9-10-folder-browsing.50744/

If the information I have is correct, the issue potentially exists on all systems running FreeBSD 10.3+ (potentially earlier 10 releases, but as the first FreeNAS 9.10 was based on 10.3, we don't really care if it happens on FreeBSD 10.2 or not). I heard from someone that the current builds of FreeNAS 10 has this issue as well.

The fixes in this thread do NOT mitigate effectively the bug that we're potentially tracking, nor do I have an ETA.

I don't seem to have directory listings that are big enough to have this issue on my server, and we've had a hell of a time trying to reproduce the issue, but someone was able to reproduce it over the weekend. ;)

So if you're directory listings are slow, you should definitely try upgrading to future releases as I hope they will be fixed.

I'll try to update this thread when we have a fix as this is hitting quite a few people.

 

[anodos]

So I'd like to add a little something here... iXsystems is currently tracking a potential bug in FreeBSD causing poor performance with directory listings. I'm working with a few people to hopefully narrow down the issue. I don't want to say too much because I don't know how much I should (or shouldn't) say and I don't want to provide inaccurate information. But one person seems to have noticed it https://forums.freenas.org/index.php?threads/performance-9-3-vs-9-10-folder-browsing.50744/

If the information I have is correct, the issue potentially exists on all systems running FreeBSD 10.3+ (potentially earlier 10 releases, but as the first FreeNAS 9.10 was based on 10.3, we don't really care if it happens on FreeBSD 10.2 or not). I heard from someone that the current builds of FreeNAS 10 has this issue as well.

The fixes in this thread do NOT mitigate effectively the bug that we're potentially tracking, nor do I have an ETA.

I don't seem to have directory listings that are big enough to have this issue on my server, and we've had a hell of a time trying to reproduce the issue, but someone was able to reproduce it over the weekend. ;)

So if you're directory listings are slow, you should definitely try upgrading to future releases as I hope they will be fixed.

I'll try to update this thread when we have a fix as this is hitting quite a few people.

A quick note regarding the parameter ea support = no. The samba manpage entry for this parameter is badly outdated. In the version of samba shipping with FreeNAS 9.10.2 (4.5.5 as of this writing), this parameter is actually coupled with Samba being able to handle SMB requests related to extended attributes (storage backend behavior).

TL;DR, ea support = yes must be set in order for streams_xattr to work properly. This means that if you want to use vfs_fruit / have Mac clients, you should leave this parameter enabled. In a future version of Samba (probably 4.6 branch?) vfs_fruit will no longer require "ea support = yes" (the client-facing aspect of ea support will be de-coupled from the storage backend aspect of it).

See discussion in samba-technical here: http://marc.info/?l=samba-technical&m=148848371604944&w=2

There has been some fairly significant rewriting of dosmode.c in samba as of late (changes are in Samba 4.5.5 / FreeNAS 9.10.2-U2). I'm not entirely sure if this "workaround" still applies. Some testing will need to be done (don't have free time for it myself), and if it doesn't "work" anymore we may need to modify the sticky and give notice that it is voodoo for future FreeNAS versions.

 

[cyberjock]

For those that have been waiting for a fix, the very poor and unfixable directory listing issues that have plagued FreeNAS for the last 8 months or so *should* be fixed in the current release (9.10.2-U3, 4/21/2017).

If the issue is not fixed please create a bug ticket at bugs.freenas.org and add me as a watcher so I will know it was created.

Thanks!

 

[marian78]

I read posts about settings case sensitivity (dataset + smb).
When i checked my datasets (very old) i saw, that all have seted his property casesensitivity to sensitive (i use dataset for "windows" sharing).
I want ask, if i can convert dataset property to insensitive on existing dataset with data and next set in smb share auxiliary parameter "case sensitive = true".

 

[anodos]

I read posts about settings case sensitivity (dataset + smb).
When i checked my datasets (very old) i saw, that all have seted his property casesensitivity to sensitive (i use dataset for "windows" sharing).
I want ask, if i can convert dataset property to insensitive on existing dataset with data and next set in smb share auxiliary parameter "case sensitive = true".

"Case sensitivity = insensitive" needs to be set at dataset creation. Nothing to worry about. I have quite a few datasets that I set up in a less than ideal manner (sometimes you don't know what you don't know), and I'm slowly migrating them to case insensitive datasets. I'd say in most cases the performance difference between the two is negligible. The problem is that you have o(n^2) with name lookups, and as the amount of files inside a folder increases to thousands or tens of thousands then the performance penalty gets very noticeable.

 

[marian78]

I will test on my data. Create 3 datasets with identical data (40GiB, 1000 folders):

1. dataset "unix" - insensitive + unix acl + samba share sensitiv true
2. dataset "win" - insensitive + win acl + samba share sensitiv true
3. dataset "win_sense" - sensitiv + win acl + samba share sensitiv false

 

[anodos]

I will test on my data. Create 3 datasets with identical data (40GiB, 1000 folders):

1. dataset "unix" - insensitive + unix acl + samba share sensitiv true
2. dataset "win" - insensitive + win acl + samba share sensitiv true
3. dataset "win_sense" - sensitiv + win acl + samba share sensitiv false

Perhaps post the results in a new thread? ;) The case sensitivity issue is pretty well documented: http://marc.info/?l=samba-technical&m=149399569504088&w=2

 

[marian78]

results on MAC OSX El Capitan:
case 1: 4s to display share
case 2: 5s to display share
case 3: over 40s to display share

My problem is, that my production datasets was creating in Freenas version, where was bug - windows dataset was creating as sensitive (bug 8002).... Now i thing, that only solution is to create new dataset insensitive and copy all data in terminal.....

In SMB share i have this aux parm:

Code:

hide files = /~*/.windows/
veto files = /Thumbs.db/thumbs.db/*.exe/*.com/*.dll/*.bat/*.vbs/*.mp3/*.avi/*.mp4/*.wmv/*.wma/
delete veto files = yes
vfs object = full_audit
valid users = @group1 user1
write list = @group1 user1
admin users = user1
vfs objects = recycle
recycle:repository = .recycle
recycle:keeptree = yes
recycle:versions = yes
recycle:exclude = *.tmp, *~, ~*.*
case sensitive = false/true (in production - false)
store dos attributes = no

I will test on my data. Create 3 datasets with identical data (40GiB, 1000 folders):

1. dataset "unix" - insensitive + unix acl + samba share sensitiv true
2. dataset "win" - insensitive + win acl + samba share sensitiv true
3. dataset "win_sense" - sensitiv + win acl + samba share sensitiv false

 

[anodos]

results on MAC OSX El Capitan:
case 1: 4s to display share
case 2: 5s to display share
case 3: over 40s to display share

My problem is, that my production datasets was creating in Freenas version, where was bug - windows dataset was creating as sensitive (bug 8002).... Now i thing, that only solution is to create new dataset insensitive and copy all data in terminal.....

In SMB share i have this aux parm:

Code:

hide files = /~*/.windows/
veto files = /Thumbs.db/thumbs.db/*.exe/*.com/*.dll/*.bat/*.vbs/*.mp3/*.avi/*.mp4/*.wmv/*.wma/
delete veto files = yes
vfs object = full_audit
valid users = @group1 user1
write list = @group1 user1
admin users = user1
vfs objects = recycle
recycle:repository = .recycle
recycle:keeptree = yes
recycle:versions = yes
recycle:exclude = *.tmp, *~, ~*.*
case sensitive = false/true (in production - false)
store dos attributes = no

You realize that samba will only interpret the last "vfs objects" line in your share definition, right? Basically "vfs objects = recycle" nukes all the default FreeNAS vfs objects for your share as well as "vfs objects = full_audit".

 

[marian78]

Thx, i will correct it...

 

[abramsgunner]

Just wanted to say that adding the auxiliary parameters listed in the original post made a huge difference in browsing the SMB shares for me. I am new to FreeNas and was disappointed in how long it was taking to browse shares... enough so that I was considering going back to sharing files from a Win 7 workstation. I added the auxiliary parameters and now when I browse, the folder is populated before I can let go of the mouse button (even opening the files seems faster). Thank you!

 

[baodad]

Is anyone else finding that disabling DOS attributes wreaks havoc with Windows' robocopy?

 

[anodos]

Is anyone else finding that disabling DOS attributes wreaks havoc with Windows' robocopy?

You can specify in the robocopy command not to copy attributes. /copy:dt

 

[emk2203]

I know that this is not about the FreeNAS server side of things, but has anyone experience with browsing shares in Linux Ubuntu? I have a medium-sized share with 10k+ files and dirs sitting on it, and Windows 10 browses are almost instantaneous.

I have taken the time-tested method of counting seconds, and with Windows 10 clients, I can barely count "21" until the dir opens.

Ubuntu's Nautilus file manager, on the other hand, takes a whopping 30 seconds to open the same share over the network. What can I improve here?

 

[SweetAndLow]

I know that this is not about the FreeNAS server side of things, but has anyone experience with browsing shares in Linux Ubuntu? I have a medium-sized share with 10k+ files and dirs sitting on it, and Windows 10 browses are almost instantaneous.

I have taken the time-tested method of counting seconds, and with Windows 10 clients, I can barely count "21" until the dir opens.

Ubuntu's Nautilus file manager, on the other hand, takes a whopping 30 seconds to open the same share over the network. What can I improve here?

Make sure your using a cifs mount and not the built in smbclient in the file browser.

Sent from my XT1096 using Tapatalk

 

[emk2203]

Make sure your using a cifs mount and not the built in smbclient in the file browser.

Sent from my XT1096 using Tapatalk

This seems to be the most popular answer, also from other forums. If only cifs mounts were not so inconvenient if you just want to browse a dozen or more shares from different servers.

 

[Pezo]

I don't think this has come up yet: is there any reason not to use a case-sensitive dataset and set Samba to be case-sensitive as well?

 

[emk2203]

I don't think this has come up yet: is there any reason not to use a case-sensitive dataset and set Samba to be case-sensitive as well?

Interesting idea; since the dataset is already case-sensitive, it was just a matter of adding the option to the SMB share config and restarting samba.

Unfortunately, the effect is negligible with <10% speedup, if any. This is most likely just a measuring error.

The idea also suffers from the fact that Win 10 access is near-instantaneous - it is possible without changing server options.