This has become a larger and larger problem that has affected more and more users. If your CIFS directory browsing is slow there are multiple ways to resolve the issue.
#1: More hardware-
You can always have more ARC and/or add an L2ARC (you can never have too much ARC). This caches data in RAM (or the L2ARC as appropriate) and should improve performance... after its been read the first time. The catch is the last part of the sentence. If it isn't cached already it will still be slow. If it isn't accessed frequently, then it may be expired from the ARC or L2ARC so it will be slow (again). So clearly not the best short or long-term option, but may be necessary in situations where your ARC and/or L2ARC are stressed and there is a need to relieve some of that stress.
#2: Increase the amount of ARC that stores metadata-
This is controlled by setting the tunable vfs.zfs.arc_meta_limit. Note that this does not force ZFS to use more ARC for metadata, but it makes ZFS more favorable to using more ARC for metadata. Also note that this value is not strictly enforced. If ZFS feels that it is more favorable to use more ARC for metadata than the vfs.zfs.arc_meta_limit then it will exceed the limit, but the higher it goes above the limit the more pressure ZFS has placed on it to not add more metadata to the ARC. ZFS typically uses about 25% of ARC for metadata. Be careful about setting this value too high. If ZFS caches too much metadata and not enough file data, server performance should suffer. There's generally better options if you have this problem.
You can check how many bytes of ARC are currently storing metadata with the command:
# sysctl vfs.zfs.arc_meta_used
vfs.zfs.arc_meta_used: 13967879256
You can check (and adjust) the ARC metadata limit with the following commands:
# sysctl vfs.zfs.arc_meta_limit
vfs.zfs.arc_meta_limit: 12000000000
# sysctl vfs.zfs.arc_meta_limit=16000000000
vfs.zfs.arc_meta_limit: 1200000000 -> 16000000000
#3: Make your L2ARC store metadata exclusively-
This is controlled by the ZFS property secondarycache.
You can set the property for your pool (or dataset) with the command:
# zfs set secondarycache=metadata <poolname or poolname/dataset>
To change it back to default use the command:
# zfs set secondarycache=all tank <poolname or poolname/dataset>
Note that this can have serious consequences for your system if the pool or dataset relies on your L2ARC to store file data. As this is often not desired this isn't generally the best option.
#4 Disable DOS attributes-
Samba imitates DOS attributes (read-only, hidden, system, archive) and ZFS stores them as extended attributes. These are really a throwback to the 80s and not too many people use them anymore. ZFS stores them as extended attributes (EAs) and there is a pretty serious performance penalty with doing so. If you don't need them the best thing you can do is disable them outright. Here's how:
Under the CIFS Settings page you can add the following settings as shown below:
Simply add the following to the Auxiliary Parameters of CIFS settings and save the changes. They will immediately apply themselves.
ea support = no
store dos attributes = no
map archive = no
map hidden = no
map readonly = no
map system = no
This does several things:
1. Disables Samba from using EAs.
2. Tells Samba not to store the DOS attributes (any existing bits that are set are simply abandoned in place in the EAs).
3. Forces all 4 parameters to off/no/unchecked/disabled.
Note: The read-only bit I am mentioning, disabling, then forcing to off is not the standard read-only permissions users may be given to files or folders.
If this is your problem, simply applying the above settings should result in an instantaneous increase in performance. If you have directories with 100k+ files it will still take a few seconds to populate, but you'll find it is MUCH more expedient than it was previously.
If the changes do not help, or you wish to undo the changes simply remove the settings. The old bits will immediately reassert themselves as they were (the abandoned bits are never overwritten with these changes).
Problems with this approach:
If you, the software you use, hardware that uses the CIFS shares directly, or your end-users rely on these settings this will break things for them.
By "software you use" I generally mean backup software. Some old-fashioned backup software uses the old "archive" bit to determine what does or doesn't need backed up. This shouldn't be used by anything that is even remotely recent, so unless you've explicitly enabled the backup software to use the archive bit or you are using backup software that is pre-2005 or so, you shouldn't have a problem.
By "hardware that uses the CIFS shares directly" I'm refering to things like all-in-one scan/copy/fax/print machines that let you scan documents to a CIFS share. Those will sometimes want to force one or more bits to on or off, and those operations will fail.
From my own personal experience, anyone that has had slow Samba directory listings have seen amazing performance increases by using these parameters. Nobody I've worked with directly has required additional changes above and beyond the above 6 parameters.
Generally its safe to try these settings and if there isn't any problems for a week its safe to assume that nothing required those bits.
Edit (4/24/2017): Directory listing performance should be much improved as of 9.10.2-U3 (released 4/21/2017).
#1: More hardware-
You can always have more ARC and/or add an L2ARC (you can never have too much ARC). This caches data in RAM (or the L2ARC as appropriate) and should improve performance... after its been read the first time. The catch is the last part of the sentence. If it isn't cached already it will still be slow. If it isn't accessed frequently, then it may be expired from the ARC or L2ARC so it will be slow (again). So clearly not the best short or long-term option, but may be necessary in situations where your ARC and/or L2ARC are stressed and there is a need to relieve some of that stress.
#2: Increase the amount of ARC that stores metadata-
This is controlled by setting the tunable vfs.zfs.arc_meta_limit. Note that this does not force ZFS to use more ARC for metadata, but it makes ZFS more favorable to using more ARC for metadata. Also note that this value is not strictly enforced. If ZFS feels that it is more favorable to use more ARC for metadata than the vfs.zfs.arc_meta_limit then it will exceed the limit, but the higher it goes above the limit the more pressure ZFS has placed on it to not add more metadata to the ARC. ZFS typically uses about 25% of ARC for metadata. Be careful about setting this value too high. If ZFS caches too much metadata and not enough file data, server performance should suffer. There's generally better options if you have this problem.
You can check how many bytes of ARC are currently storing metadata with the command:
# sysctl vfs.zfs.arc_meta_used
vfs.zfs.arc_meta_used: 13967879256
You can check (and adjust) the ARC metadata limit with the following commands:
# sysctl vfs.zfs.arc_meta_limit
vfs.zfs.arc_meta_limit: 12000000000
# sysctl vfs.zfs.arc_meta_limit=16000000000
vfs.zfs.arc_meta_limit: 1200000000 -> 16000000000
#3: Make your L2ARC store metadata exclusively-
This is controlled by the ZFS property secondarycache.
You can set the property for your pool (or dataset) with the command:
# zfs set secondarycache=metadata <poolname or poolname/dataset>
To change it back to default use the command:
# zfs set secondarycache=all tank <poolname or poolname/dataset>
Note that this can have serious consequences for your system if the pool or dataset relies on your L2ARC to store file data. As this is often not desired this isn't generally the best option.
#4 Disable DOS attributes-
Samba imitates DOS attributes (read-only, hidden, system, archive) and ZFS stores them as extended attributes. These are really a throwback to the 80s and not too many people use them anymore. ZFS stores them as extended attributes (EAs) and there is a pretty serious performance penalty with doing so. If you don't need them the best thing you can do is disable them outright. Here's how:
Under the CIFS Settings page you can add the following settings as shown below:
Simply add the following to the Auxiliary Parameters of CIFS settings and save the changes. They will immediately apply themselves.
ea support = no
store dos attributes = no
map archive = no
map hidden = no
map readonly = no
map system = no
This does several things:
1. Disables Samba from using EAs.
2. Tells Samba not to store the DOS attributes (any existing bits that are set are simply abandoned in place in the EAs).
3. Forces all 4 parameters to off/no/unchecked/disabled.
Note: The read-only bit I am mentioning, disabling, then forcing to off is not the standard read-only permissions users may be given to files or folders.
If this is your problem, simply applying the above settings should result in an instantaneous increase in performance. If you have directories with 100k+ files it will still take a few seconds to populate, but you'll find it is MUCH more expedient than it was previously.
If the changes do not help, or you wish to undo the changes simply remove the settings. The old bits will immediately reassert themselves as they were (the abandoned bits are never overwritten with these changes).
Problems with this approach:
If you, the software you use, hardware that uses the CIFS shares directly, or your end-users rely on these settings this will break things for them.
By "software you use" I generally mean backup software. Some old-fashioned backup software uses the old "archive" bit to determine what does or doesn't need backed up. This shouldn't be used by anything that is even remotely recent, so unless you've explicitly enabled the backup software to use the archive bit or you are using backup software that is pre-2005 or so, you shouldn't have a problem.
By "hardware that uses the CIFS shares directly" I'm refering to things like all-in-one scan/copy/fax/print machines that let you scan documents to a CIFS share. Those will sometimes want to force one or more bits to on or off, and those operations will fail.
From my own personal experience, anyone that has had slow Samba directory listings have seen amazing performance increases by using these parameters. Nobody I've worked with directly has required additional changes above and beyond the above 6 parameters.
Generally its safe to try these settings and if there isn't any problems for a week its safe to assume that nothing required those bits.
Edit (4/24/2017): Directory listing performance should be much improved as of 9.10.2-U3 (released 4/21/2017).
Last edited:
