SOLVED Browsing directories slow

Status
Not open for further replies.

mstang1988

Contributor
Joined
Aug 20, 2012
Messages
102
It's not the file count that tanks mine, it's the physical size of the directory or perhaps the physical size of the files in them. 760 subfolders should be an instant population IMO (there is no files in the parent dir). I still need to take wireshark traces and see if the client is requesting extra data and perhaps put on a debug version of samba but I don't want to waste that much time when I have great performance with NFS>
 

fcueto

Dabbler
Joined
Aug 10, 2014
Messages
26
Hi guys. Go to services, CIFS and uncheck the box for EA support and Support dos file attributes.:)

You saved my day !!

For FreeNAS 9.2.1.7 users, the settings are not available in the GUI anymore.
You have to go to services, CIFS and add the 2 following lines to the Auxiliary Parameters text field:
ea support = no
store dos attributes = no



Please note that this should be the default setting as it simply doesn't work "out of the box"....
 

c0re

Dabbler
Joined
Feb 11, 2013
Messages
26
I actually had never gotten this issue resolved - this seems to work, so thanks flo_eu and fcueto! Performance is great like this.

That said, when DOS file attributes are disabled, while performance is good, hidden files don't remain hidden and clutter up a lot of directories. Folder icons also get lost and I get a lot of corrupt recycle bin errors (Win 8.1).

I'll do some searching now, but is there any way to tweak the performance so that DOS file attributes can be enabled without causing this issue? Is this a known issue that already has a bug report, or should this be reported?
 

indy

Patron
Joined
Dec 28, 2013
Messages
287

socos

Dabbler
Joined
Aug 30, 2014
Messages
11
Been chasing this one for some time now, this did the trick! Thanks!!

You saved my day !!

For FreeNAS 9.2.1.7 users, the settings are not available in the GUI anymore.
You have to go to services, CIFS and add the 2 following lines to the Auxiliary Parameters text field:
ea support = no
store dos attributes = no



Please note that this should be the default setting as it simply doesn't work "out of the box"....
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
A bit about DOS modes / attributes.

DOS modes are visible in Windows if you right-click on a file or folder and hit 'properties'. They consist of 'Read-only, Hidden, System, and Archive'. As is implied in their name, these attributes are from the good old days of DOS. Samba has two different ways of dealing with them. By default and when not configured as an AD domain controller (when an AD DC samba will automatically enable 'store DOS attributes') samba uses the "map" parameters mentioned in the smb.conf manpage. The "map*" parameters work by manipulating the unix file permissions in the following ways:
dos readonly is represented in unix by removing everyone's write bit
dos archive is represented in unix by the user's execute bit
dos system is represented in unix by the group's execute bit
dos hidden is represented in unix by the other's execute bit
The above operations are carried out through chmod. Unfortunately, this has a tendency to mess up ZFS access control lists (ACLs). Since FreeBSD 9.2.1.6 the zfs aclmode is set to "restricted" on CIFS shares with ACL type set to "windows". The "restricted" aclmode prevents people from nuking their ACEs through a drive-by chmod. It also prevents the "map *" parameters from working. Simply put, the "map *" parameters are a no-go if you're using windows ACLs.

Alternatively, samba can store this information in EAs and SECDESCs. This is the way I believe that samba development is headed. Comments in the samba source regarding the "map *" parameters say things like
I'd like to see the following options go away and always use EAs and SECDESCs
Which makes sense because EAs and SECDESCs are able to store much more information than just these DOS modes. For instance, samba also stores create time in the DOSATTRIB EA. Extended attributes are necessary for the vfs_streams_xattr module, which provides compatibility with NTFS alternate datastreams, to work (user.DosStream). They may also be necessary in order for the upcoming vfs_fruit module (which may fix a lot of problems that Mac users have) to work properly.

Unfortunately, extended attributes are sometimes treated like second class citizens. XATTR operations appear to be quite slow on FreeBSD 9.x. Every time that you open a directory, samba will retrieve the user.DOSATTRIB EA of every file and folder in that directory. This results in very slow uncached browsing via Windows explorer.

Which means that if you have slow browsing performance you have the following options to mitigate it:
1) Add ARC or L2ARC. Standard advice applies here. It is best to max out RAM before considering L2ARC. Adding L2ARC on a system with 8GB RAM is a bad idea.
2) As mentioned in this thread disable "ea support" and "store dos attributes". Note that the DOS modes mentioned above will cease working, which may cause strange behaviour and prevent backup software from functioning properly. Programs, embedded devices, and network scanners / multi-function units may attempt to utilize the archive bit and fail unpredictably. Additionally, it may break some functionality for Mac users.
3) Run a cronjob to cache the extended attributes. An example of such a cronjob would be running the following command
Code:
find /mnt/Tank/Media/ -exec getextattr -x -q user DOSATTRIB {} \;

Note that this command assumes that you have enough ARC / L2ARC to keep the EA cached. It's hard to gauge how much is needed. I've seen between 8KB -18KB per file. If you scale this to 500,000 files, then you're looking at about 3.7GiB- 8.35GiB of metadata. Since by default 25% of the ARC is dedicated to metadata, you're looking at a minimum arc size of 15-34GB to keep this cached.

(3) is theoretical. I also don't know how frequently you need to run it (since it is a rather intensive process, the less the better). If you're having this problem and feel like being a guinea pig, test it and report back your findings. :)

Edit: my experience with (3) is that it works okay for a few days to a week, but then the data gets flushed from ARC. Not a good solution.
 
Last edited:
  • Like
Reactions: Maq

Paranoiak

Dabbler
Joined
Jun 3, 2014
Messages
46
Hi guys. Go to services, CIFS and uncheck the box for EA support and Support dos file attributes.:)
Thank you very much, it solved all my CIFS browsing issues !
 

Chris Tobey

Contributor
Joined
Feb 11, 2014
Messages
114
Does anyone know if this will solve NFS -> CIFS lag issues? I do not want to derail the thread, but when I create a new file accessing a share via NFS (on a linux system with that drive mounted), then try to access the file via CIFS (on my Windows system), it can take up to 15 minutes for the CIFS system to be able to access the file. If I restart the CIFS service it's available immediately, but I can't just restart CIFS every time I want a file transfer to be instantly available.
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Does anyone know if this will solve NFS -> CIFS lag issues? I do not want to derail the thread, but when I create a new file accessing a share via NFS (on a linux system with that drive mounted), then try to access the file via CIFS (on my Windows system), it can take up to 15 minutes for the CIFS system to be able to access the file. If I restart the CIFS service it's available immediately, but I can't just restart CIFS every time I want a file transfer to be instantly available.
Don't share the same directory with NFS and CIFS simultaneously. Just use CIFS.
 

Chris Tobey

Contributor
Joined
Feb 11, 2014
Messages
114
Don't share the same directory with NFS and CIFS simultaneously. Just use CIFS.
I do not see how that is helpful. Sharing my directories with both CIFS and NFS works very well. Only sometimes do I have the lag issue and I am wondering if there is a solution.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
I do not see how that is helpful. Sharing my directories with both CIFS and NFS works very well. Only sometimes do I have the lag issue and I am wondering if there is a solution.

I'm sure there's a solution. I'm sure you'd need to code something complex enough to handle Samba and nfsd and make them work nicely together. That is not a fight worth fighting, so the best thing to do is to stick with a single service for each share.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
I do not see how that is helpful. Sharing my directories with both CIFS and NFS works very well. Only sometimes do I have the lag issue and I am wondering if there is a solution.

Yes, it "works". As long as you define "working" as "works fine but may corrupt your files at any time".

Yes it works. But under the hood each service handles its own file and directory locking. This can lead to file and directories being lost and/or corrupted because the two services are trying to do conflicting things at the same time.

Yes, I can do it. Is it a good idea? Probably not.

Here's the warning from the manual:

Note: while the GUI will let you do it, it is a bad idea to share the same volume or dataset using multiple types of access methods. Different types of shares and services use different file locking methods. For example, if the same volume is configured to use both NFS and FTP, NFS will lock a file for editing by an NFS user, but a FTP user can simultaneously edit or delete that file. This will result in lost edits and confused users. Another example: if a volume is configured for both AFP and CIFS, Windows users may be confused by the extra filenames used by Mac files and delete the ones they don’t understand; this will corrupt the files on the AFP share. Pick the one type of share or service that makes the most sense for the types of clients that will access that volume, and configure that volume for that one type of share or service. If you need to support multiple types of shares, divide the volume into datasets and use one dataset per share.

There are other problems (such as the lag). I don't even try to discuss the other problems in any depth because if you are choosing to ignore something as serious as potential corruption, performance and other inadvertent problems shouldn't be given priority. I don't want to be the guy that gets a nasty PM later telling me "I took your advice and now my data is gone! It's your fault and you are GONNA help me or I'm gonna find you and beat the crap out of you." You are basically running an unsupported and untested configuration if you want to continue using both CIFS and NFS with the same files.

So while you might see that its not helpful, he's giving you far better advice than you expected to get. It would behoove you to not ignore the advice because its not the answer to your question. He's doing you a big favor, and you'd be smart to thank him and take his advice. ;)

He didn't rack up 1000 posts on accident. ;)

FreeNAS lets you do things that many would consider stupid, maybe even idiotic. For some people, mistakes like this have given them some great... shall we call it "opportunities". They had lots of time to read our fine manual, update their resume, take a little (or a lot) unpaid vacation, things like that. Nobody here likes to see people take unpaid vacations, so we try to provide the best advice we can. Sometimes that advice isn't what you want, but instead of responding like you did you should ask yourself why someone with 1000 posts is telling someone with less than 20 posts not to do something.

I talked to someone last week (and he's not the first and I'm sure there will be another before the end of the month) where they contact me begging for help. Why? Because if they can't get the data back the company will be out of business by the time litigation is done.

Good luck.
 
Last edited by a moderator:

Chris Tobey

Contributor
Joined
Feb 11, 2014
Messages
114
Thank you to both of you. Saying something along the lines of "no, this is not a solution to what is a very complex issue in a configuration that is not recommended" is much better than not even addressing the question and just saying "don't do that", which was the point I was trying to express.

At any rate, I am very happy with my setup, even with the occasional lag :) Luckily catastrophic failure and loss / corruption of my all data would be upsetting, but not damaging to my career, an acceptable risk.
 

StephenFry

Contributor
Joined
Apr 9, 2012
Messages
171
ea support = no
store dos attributes = no

O, my word... this is it! This is what I've been struggling to find for YEARS. Thank you thank you thank you, this sped my browsing to the point where there are no perceptible delays. LOVE

Please note that this should be the default setting as it simply doesn't work "out of the box"....

No kidding, I've e-mailed these setting to all the people I know use FreeNAS and all expect one are now buying me a beer.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Status
Not open for further replies.
Top