Lots of files, SMB folder list slow

[anodos]

Those suggestion was cut and pasted from the samba wiki.

I don't follow you in the word "tanking" (=taking performance of ?) But indeed freenas will fill in "acl_xattr" in the global section. I seem there is no way I can revert it either.

Cutting and pasting from a wiki is not a great idea. zfsacl fulfills the role that acl_xattr would play on a Linux / non-zfs server (i.e. filesystems that don't support nfsv4 ACLs). Tank
You yourself noted that there are performance problems with storing data as filesystem extended attributes, yet you're storing more information as extended attributes. It's simply more efficient to use native ZFS ACLs rather than engaging in such nasty hackery.

(I haven't tried what happens when I dont have Freenas acting like a Domain Controller).

See bolded and underlined sentence in wiki here: https://wiki.samba.org/index.php/Setup_a_Samba_Active_Directory_Domain_Controller#Introduction

Maybe this is something that should be changed in the Freenas code base ?
Perhaps that's the best way to go. But I liked the web gui.

That'd be a major change in how FreeNAS works. Wouldn't happen in 9.3. FreeNAS 10 will add bhyve support. Bhyve VM would probably be even better candidate for running an AD DC.

 

[pernils]

(Trying to summarise the thread..)

Another thing that is odd.

Freenas as Domain Controller. You have specify in Service -> CIFS -> Auxiliary parameters: store dos attribute = no and ea support = no.
The /usr/local/smb4.conf

Code:

[global]
    server max protocol = SMB2
.....
    server string = FreeNAS Server
    ea support = yes
    store dos attributes = yes
    lm announce = yes
....
    store dos attributes = no
    ea support = no
....
[data]
    path = /mnt/Vol_1/data
    printable = no
    veto files = /.snapshot/.windows/.mac/.zfs/
....

will "compile" with testparm to both of them = Yes. A bit controversial ... I thought the last cfg should be the valid one. And it will also add vfs objects = dfs_samba4, acl_xattr in the global section even it's not in smb4.conf.

Nerveless I have decided to go with the guide line to not have Domain Controller and AD member on the same server. I will deal with jail and DC in the future.

For those who insist with having DC and AD member on the same server instance you can set the parameter = no on every share you wish to have faster folder browsing (Sharing -> Windows (CIFS) -> yourshare -> Advanced Mode -> Auxiliary parameters)
But is not recommended.

If you only make freenas to be a AD member. You can add in Service -> CIFS -> Auxiliary parameters: store dos attribute = no and ea support = no.
Your /usr/local/etc/smb4.conf will then look like

Code:

[global]
....
    server string = FreeNAS Server
    ea support = yes
    store dos attributes = yes
    lm announce = yes
    hostname lookups = yes
....
    server role = member server
    netbios name = THOR
....
    log level = 1
    store dos attributes = no
    ea support = no

[data]
    path = /mnt/Vol_1/data
    printable = no
    veto files = /.snapshot/.windows/.mac/.zfs/
    writeable = yes
    browseable = yes
    vfs objects = zfs_space zfsacl aio_pthread streams_xattr
...

And will compile with testparm to ...

Code:

[global]
 ....
    server string = FreeNAS Server
    server role = member server
    security = ADS
    allow trusted domains = No
....
[data]
    path = /mnt/Vol_1/data
    read only = No
    guest ok = Yes
    veto files = /.snapshot/.windows/.mac/.zfs/
    vfs objects = zfs_space, zfsacl, aio_pthread, streams_xattr
    zfsacl:acesort = dontcare
    nfs4:chown = true
    nfs4:acedup = merge
    nfs4:mode = special

Testparm will not add the vfs object = acl_xattr and also make the last cfg valid.
As the default value for store dos attribute and ae support = no they will not be shown.

You will then have fast folder browsing but will not be able to set those DOS bits (read only/hidden/archive)

more in depth is explained in this thread https://forums.freenas.org/index.ph...-to-improve-samba-browsing-performance.24906/

 

[pernils]

So finally I have moved all the data from win2008 to freenas mini, but the users has mention that the network it's slower now that it used to be.

I have done some monitoring in the gui and the cpu is far from hitting the roof so to speak. My zfs arc hit ratio dropped to around 50% thought.

After manually zfs send a snapshot to a home build freenas box with around 65 GB I was surprised that it was done in about 40 min. The same amount (65gb) from win2008 to the nas took about 3 hours to complete.

Is samba this slow ?

Did some more research and there are some setting that have been moved away from the gui (AIO) started from version 9.1.1.
http://www.freenas.org/whats-new/2013/08/freenas-9-1-1-release.html

The reason for removing this option seems to be instability. http://hardforum.com/showthread.php?t=1779597

After reading this https://www.reddit.com/r/homelab/comments/37npow/slow_freenas_write_performance/

Will that option find it's way back again after version 10 (built on freebsd 10) comes out ?

(hmm I could see for this by self by installing freenas under virtualbox and change train...)

Did some testing by copy a large file 3 gb to the nas and then copy it back locally.

write (copy to nas) around 85 mb/s
read (copy from nas) around 95 mb/s
(this is what win 7 tells you)

Did the same on win2008 (raid 1)
write (copy to win2008) around 90 mb/s
read (copy from win2008) around 70 mb/s

The numbers are almost the same. More accurate would be to make 5 or more runs but the fact still exists, the users "feels" that the share is a bit slower now.

* edit *
added a link that could be of interest https://forums.freenas.org/index.php?threads/cifs-directory-browsing-slow-try-this.27751/
* edit *

Does anyone have some insight what we can expect on this matter in the upcoming 10 release ?...

 

[pernils]

So after finally I got it working of serving files I discovered a new obstetrical.

It seems that when you do the above you more or less disable the functionality of then AD integration.

Basically you are left with the permission bits on the unix side. What you do in AD doesn't matter.

For example :

Owner: my_domain/administrator Permission RWX
Group: my_domain/group_office Permission RWX
Other: RX

This will lead to that user in the usergroup group_office (in AD) will be able to write to the share.
Other or (Everyone in AD) will only be able to read from the share.

The Domain administrator can add as many usergroups he like (from the AD side of the share) but it will not make any difference on the outcome.

My conclusion is that SAMBA is not suited for a production environment where you want some more fine grained permission on the shares.

To engage the dos attribute etc will just make then share useless.

Does anyone have the same experience or is my AD integration broken ?
wbinfo -u will list my AD users thought.

I'm a bit crippled at the moment for testing this due the Freenas Mini is serving production files now. And my other Freenas box is re forcing to be member of my AD.
The whole thing is unstable at the best.
I decided to move to ZFS from hardware raid due to be more pleased with the integrity of my data. The experience is the opposite. Thinking if I should also keep a third backup on non ZFS filesystem.

I have some snapshot that I can't remove due something good now's what.
The virtual console constant pounding something about No jason object... related to some treemenu. And the list goes on.
I looked at some youtube video about Linux sucks due everything is more or less broken in some way. At the time I thought he is all bull... but this constant forum search and terminal hacking makes his point valid.

Bah... enough with the rant.

 

[SweetAndLow]

This isn't Linux. It sounds like you have broken your install if you are getting errors in the console. There should be zero error messages happening. It also sounds like your ad environment is broken or you don't understand how it should work, I suspect a little of both.

Your post is long but you don't describe anything, don't ask questions and don't provide log messages. Please try explaining what you think it should do, what it is doing, how you are doing it and what errors and logs you are seeing.

 

[anodos]

So after finally I got it working of serving files I discovered a new obstetrical.

It seems that when you do the above you more or less disable the functionality of then AD integration.

Basically you are left with the permission bits on the unix side. What you do in AD doesn't matter.

For example :

Owner: my_domain/administrator Permission RWX
Group: my_domain/group_office Permission RWX
Other: RX

This will lead to that user in the usergroup group_office (in AD) will be able to write to the share.
Other or (Everyone in AD) will only be able to read from the share.

The Domain administrator can add as many usergroups he like (from the AD side of the share) but it will not make any difference on the outcome.

My conclusion is that SAMBA is not suited for a production environment where you want some more fine grained permission on the shares.

To engage the dos attribute etc will just make then share useless.

Does anyone have the same experience or is my AD integration broken ?
wbinfo -u will list my AD users thought.

I'm a bit crippled at the moment for testing this due the Freenas Mini is serving production files now. And my other Freenas box is re forcing to be member of my AD.
The whole thing is unstable at the best.
I decided to move to ZFS from hardware raid due to be more pleased with the integrity of my data. The experience is the opposite. Thinking if I should also keep a third backup on non ZFS filesystem.

I have some snapshot that I can't remove due something good now's what.
The virtual console constant pounding something about No jason object... related to some treemenu. And the list goes on.
I looked at some youtube video about Linux sucks due everything is more or less broken in some way. At the time I thought he is all bull... but this constant forum search and terminal hacking makes his point valid.

Bah... enough with the rant.

Click 'system' -> advanced -> save debug and post here.

I have multiple freenas samba servers in production. No problems, but I always test and configure before putting into production.

 

[pernils]

Been fiddling with *nix system now and then for more then 10 years. Amount of google hours on *nix is way above those for win. Just a fact... but I have ditched win back home for testing various os like Ubuntu Mint PC-bsd and current Arch.
Sorry about the rant.

It pro belly is just me but I always end up with having to collect the pieces several sources before I can solve just my typical problem. In this case I trying to collect all relevant (or irrelevant) info about the subject for other poor google hunters. (Okay I have moved away from the topic now)

For what is worth.

For the Freenas box that I can list the domain users -> wbinfo -u I have also yellow background in Active Directory -> Domain Account Name: and Domain Account Password:.
But under Active Directory -> Kerberos Realm tab I have no info in the boxes KDC , Admin Server, Password Server.

But when I check the /etc/krb5.conf I have kdc = comp.mydomain.local:88 and admin_server = kdc = comp.mydomain.local:88 ect ...

On the other freenasbox that will not become Server Role: ROLE_DOMAIN_MEMBER I have all the server.domain: port listen both in the /etc/krb5.conf and also in the GUI under Active directory -> Kerberos Realm.

Both boxes have exact the same /ect/krb5.conf files but the box that must be a role_domain_member (who i can list the domain user with wbinfo-u) will not show this in the GUI. The other box is the opposite.

Both boxes is on the same subnet
Both boxes have the same ntp servers
Both boxes can ping the mydomain.local
Both boxes have the same /etc/krb5.conf
Both boxes have the same setup in hostname : main1.mydomain.local main2.mydomain.local
Both boxes have the same dns servers
and so on ....
When those boxes don't behave the same you get a bit annoying. If I sort it out I will post what I have missed.

DS
Attached file is from box suppose to be a role_domain_member that will not answer on permission settings done in AD.
PS

 

[pernils]

For my backup box it (the one who refused to be domain member) ended up in reinstall. Mostly because the usb had run out of space (only 2Gb).

My recipe on this.

My network :
firewall/dhcp/dns : 192.168.100.1
mydomain controller (win2008) : 192.168.100.6

System -> Advanced -> Show console messages in the footer : check it
(It nice to see something what is going on)


System -> Information -> Hostname : mybox.mydomain.local

Network -> Global Configuration -> Hostname : mybox

Network -> Global Configuration -> Domain : mydomain.local

Network -> Global Configuration -> IPv4 Default Gateway : 192.168.100.1

Network -> Global Configuration -> Nameserver 1 : 192.168.100.1

Network -> Global Configuration -> Nameserver 2 : 192.168.100.6

Now it's time to open a shell on the freenas box and see if you can ping your domain (ping mydomain.local). If not read forum and troubleshoot.

Directory -> Active Directory -> Domain Name (DNS/Realm-Name) : mydomain.local

Directory -> Active Directory -> Domain Account Name : administrator (or some power user that is allowed to add clients to the domain)

Directory -> Active Directory -> Domain Account Password : <your domain admin password>

Check the Enable box.

Press Save and check the footer console message.


You should now see that the freenas box (mybox) is listened on your windows server under Active Directory Users and Computers -> Computers
You should also be able to open up a shell and verify the domain users

wbinfo -u (for domain users)
wbinfo -g (for domain groups

Shares :

Storage -> your share -> Change Permission (key symbol on bottom left)

Apply Owner (user): checked

Owner (users): mydomain.local\administrator (just select from the list)

Apply Owner (group): checked

Owner (group): mydomain.local\domain users (just select from the list)

Apply Mode : checked

Permission Type: Windows

Set permission recursively: checked (you want to make the change to all of your file structur)

Press Change .....

Sharing -> Widnows (CIFS) -> Add Windows (CIFS) Share -> Path : Browse to your share

Sharing -> Widnows (CIFS) -> Add Windows (CIFS) Share -> Name : name it as you wish

Sharing -> Widnows (CIFS) -> Add Windows (CIFS) Share -> Apply Default Permission : checked

Under Advanced Mode add this to Auxiliary Parameters:

store dos attributes = No
ea support = no

If you have folders with a lots of files. This could also be added globaly under Services -> CIFS

Press OK.

You should now be good to go.

From windows under mydomain.local\administrator account ...

Left and right click on the share and properties -> Security -> edit.

You should now have ..

Everyone
Adminstrator
Domain Users (mydomain.local\Domain Users)

(this I think is for the add in Auxiliary Parameters)

Lets say that you want to add a domain group that you will give permission to write to the share and all other only read.
Go back to Storage and permission and change there Owner Group to your special group.

Else your user will only be able to add files but not to delete or modify.


******************************************

As you can see my recipe is not complete. I can't do as i wish from the windows side.

Under Directory -> Kerberos Realms -> I only have mydomain.local under Realm .. nothing in KDC or Admin Server etc ...

Could someone help to make my "recipe" complete.