sam_rids_to_names: possible deadlock - trying to lookup SID

[Kuro Houou]

I was running 9.2.1.5 fine, no issues. Went to upgrade to 9.2.1.6 and I am getting this error below all the time in the console,

Jul 4 21:30:30 freenas winbindd[8098]: sam_rids_to_names: possible deadlock - trying to lookup SID S-1-5-21-1901708109-570851903-2486881474

When I look at the Owner of the folders/files it shows this SID. If I try to change it to root it won't let me, it's like it doesn't exit.. If you look at the actual CIFS shared folder it says, Unix User\root as the owner. Anyone have any idea how to fix this? I tried making a new 9.2.1.5 usb boot disk and tried re-creating the CIFS share from scratch but the permissions seem to have stuck and I can't reset them..

 

[Kuro Houou]

Just an update, looks like I can see my user I added in the FreeNAS gui as the owner of the folders now.. but still getting this error message :(

 

[Kuro Houou]

So playing around with this some more. It looks like this SID is tied to a user account I created to give one user RW to a specific folder, usually they would just have read/ex from the everyone group. I can get the error to occur when I add that specific user to the folder permissions in windows. As soon as I click OK on the permissions window I get a bunch of the possible deadlock errors. I tried deleting the user and recreating it.. same error. I tried recreating it with a different user id as well.. hoping it would get a new SID, but apparently it gives the same exact one.. So I created a new account user2 and gave it full control to the folder.. no error reported in the log. Next I tried creating the same user but instead of putting it in my normal group I usually do, I had it create a new group tied to the username. When I did this and added the user to the folder with full control I didn't get an error back. Not sure why the group mattered, but apparently it did. I will continue to monitor the logs in case the error comes back, but fingers crossed its finally gone!!!

 

[Kuro Houou]

One new update, so I was using a different usb stick then I really wanted to use. So I backed up my config, installed 9.2.1.6 on the new/right usb stick, booted that up and restored my config. After the reboot I started getting the possible deadlock error again on another SID... Not sure what happened, maybe when you backup the config it doesn't backup the SIDs for your users??? that doesn't make much sense. Any ideas anyone?

 

[JoelN]

I started getting the "deadlock" error messages under similar circumstances - apparently caused by the move from 9.2.1.5 to 9.2.1.6RC and the Samba update that occurred at the same time. I get the error whenever I try to browse Freenas from my Windows 7 client. Moving back to 9.2.1.5 ended the error messages. The moved to 9.2.1.6 brought them back. Same hardware. Bug report #4432 and #5054 indicate others have experienced what may be a similar circumstance. (Disclosure - I am a new Freenas user & not currently using recommended hardware. I will be moving to recommended hardware this week).

Edit: Moving to "maximum protocol" SMB2 didn't solve the problem for me.

 

[diedrichg]

I too am getting the deadlock. I'll see if I can replicate your fix. Hopefully that will work. Thanks.

 

[Kuro Houou]

I went back to 9.2.1.5 and I am getting these errors still :( Seems like maybe 9.2.1.6 screwed up permissions thoroughly. I go into my windows permissions, add the user, click ok... then I get the error. If I go back to the permissions I just see a SID that matches the error.. its like windows won't re-resolve the SID. I created a brand new account and same thing. Can add the user in windows permissions, close the permissions, re-open, then can't resolve the SID.. and lots of the same deadlock errors in the logs.

 

[JoelN]

Kuro, when I returned to 9.2.1.5 I did a fresh install of both OS and data. My Freenas server is new and it was not too burdensome to start from scratch. I hope you find a solution.

 

[Kuro Houou]

Kuro, when I returned to 9.2.1.5 I did a fresh install of both OS and data. My Freenas server is new and it was not too burdensome to start from scratch. I hope you find a solution.

I just did the same thing, restoring from backup now.. will probably take a couple days :( Not sure how this huge bug made it out of Beta. As far as I am concerned 9.2.1.6 should be taken down as it seems a lot of people are having this issue.

 

[cyberjock]

You may or may not be aware of these two things:

1. Every time a new version comes out a bunch of people have problems. Quite often it's user-error and not a problem with FreeNAS. Every time a new version comes out a bunch of people have problems. Most are related to doing things we tell them not to do and they do it anyway. So saying "it should be taken down as it seems a lot of people are having this issue" is just silly because nothing would ever be released.
2. Many people have screwed up permissions badly. I'm actually writing a guide for handling permissions "properly" in 9.2.1.6+ and took a break just to see what was going on in the forums. The problem is many people just tinkered until it appeared to work and then never touched it when the reality is that the permissions worked but were improper and they now don't work since they weren't proper to begin with.

 

[Kuro Houou]

I guess I am just confused as it seems this error was reported multiple times in the beta releases, yet somehow still got released in 9.2.1.6.. If their were issues going on then that couldn't be resolved wouldn't it have made sense to hold off on updates to samba in this release? Maybe keep them as they were until those issues could be resolved and then try again for 9.2.1.7 or a later release after some more testing could be done? Not trying to start anything here, maybe I am just new and upgrading to point releases at this level might not be a good idea, maybe wait for 9.x.x vs 9.x.x.x? But that's why I have backups I guess too.

I really didn't do anything out of the norm I don't think with my config, its very basic.. I only had two user accounts and windows permissions enabled on my volume and a couple cifs shares, everything was running just fine pre-update. Then I upgraded to 9.2.1.6 and things just went downhill quickly it seemed, like it did for others on the site it seems.

 

[cyberjock]

Ok, you need to understand some things about permissions. No I won't go into extreme detail because literally *nobody* "in the know" even bothers answering questions on permissions. So I'll say this and this as far as I'm going to go:

1. Many people did improper permissions but they appeared to work so they left them broken.
2. Many people did proper permissions with Samba3 but Samba4 is different so some changes may be necessary if they aren't quite the same.
3. Many people just don't understand permissions at all and fubared them to hell.

So no, there currently is no need for a 9.2.1.7 and things are working properly per the Samba developers and per our model for how they *should* work on FreeNAS/FreeBSD. If you disagree more than likely you just don't understand them fully.

Like I said, wait for my guide. It should explain much of this and you may see what you aren't understanding. But unless someone finds some horribly bad bug in 9.2.1.6 (which I think is *extremely* unlikely considering the testing that has gone into 9.2.1.x's as well as TrueNAS customers) then I'd say it is MUCH MUCH MUCH more likely you are the one in error and not FreeNAS.

 

[Kuro Houou]

Ok, you need to understand some things about permissions. No I won't go into extreme detail because literally *nobody* "in the know" even bothers answering questions on permissions. So I'll say this and this as far as I'm going to go:

1. Many people did improper permissions but they appeared to work so they left them broken.
2. Many people did proper permissions with Samba3 but Samba4 is different so some changes may be necessary if they aren't quite the same.
3. Many people just don't understand permissions at all and fubared them to hell.

So no, there currently is no need for a 9.2.1.7 and things are working properly per the Samba developers and per our model for how they *should* work on FreeNAS/FreeBSD. If you disagree more than likely you just don't understand them fully.

Like I said, wait for my guide. It should explain much of this and you may see what you aren't understanding. But unless someone finds some horribly bad bug in 9.2.1.6 (which I think is *extremely* unlikely considering the testing that has gone into 9.2.1.x's as well as TrueNAS customers) then I'd say it is MUCH MUCH MUCH more likely you are the one in error and not FreeNAS.

Well ill wait to see what your guide has to say and ill give you the benefit of the doubt as you have been using the product a lot longer then me. But in all honesty I don't know how I could have screwed up my permissions so much that it would cause errors like this... I only used windows acls on the volume and had two user accounts setup in Freenas, one was a member of wheel as it was an alternative admin account, the other was part of another group I created in freenas and which I gave full control to one specific folder, this was the account that ended up having the broken SID issue and deadlock errors as soon as I upgraded. But again, ill look forward to seeing your guide to see how maybe I did things wrong. I am a bit new to the whole Unix side of permissions, hence using the Windows ACL's on my volume, I'm quite familiar with those after being an AD Admin for 7+ years.

 

[emptyBox]

Is anyone anywhere with this? I'd like to know if there might be an easy solution, like deleting users and setting them back up. I'm coming from 9.2.1.3. I just don't want to lose data, and I don't particularly want to back everything up and start from scratch.

 

[Kuro Houou]

I don't think there is a fix, I tried everything I could think of, tried resetting permissions, changing to Unix permissions, deleting shares, reading them. deleting users, recreating them..I tried to just revert back to 9.2.1.5, the last working version I tried. That didn't work either.. I am not sure if when upgrading to 9.2.1.6 it did something to the volume or what, but I couldn't fix it by just downgrading. At the end of my troubleshooting the last thing I saw was windows was unable to resolve sid's from my server. I could add permissions for a user and it showed fine, then as soon as I closed it out and re-opened it the name was gone and just a sid was there. That's when the error message would appear for me. Its like it was trying to resolve a SID but couldn't...

 

[emptyBox]

Well, for my own documentation I found this link,
https://www.samba.org/samba/docs/man/Samba-Guide/upgrades.html
Which explains the SID issues, well its giving me a starting point at least.

Im pretty new to this also. when i set up my permissions, i scoured the fourums for eery bit of info i could find. i never had a problem until I tried this upgrade. But browsing files on my server i noticed on several of my files, from right clicking my windows box, they were associated with three users, my account, the administrator account, and there was a 3rd account, which had no name associated with it. Only an SID, s-1-5-.....

It's been hard to find solutions to this problem, like you said. I found a few other users who had filed bugs with previous freenas versions, but there wasn't a resolution to my liking. Expanding my search outside of the freenas garden, and there is a lot more info, and it seems like this is a little more common. as the above samba ref states, some users have problems and some don't. I've switched up so many other variables about the same time I upgraded I'm not sure where to start. I also know the upgrade was to samba4 which is a dif animal.

I may have to buy another USB hdd to back stuff up to while i figure this out, i can't afford to lose these datas.

And you say downgrading didn't work?

 

[Kuro Houou]

downgrading alone did not work, I just made a fresh usb drive with 9.2.1.5 after upgrading, re-imported the volume, but the error came back. I had to delete the volume and restore from backup :(

 

[JoelN]

Kuro, I'm now running 9.2.1.6 with no deadlock errors. Your comment that the upgrade may have done 'something to the volume' caused me to reinstall 9.2.1.5 with some procedural changes. Assuming my USB may be involved somehow I did a full (not quick) reformat of my USB drive using FAT (not FAT32) and loaded the 9.2.1.5 image file. My primary volume consists of a 2-way mirror. Assuming my hard drives may somehow be involved I rebuilt my 2-way mirror switching one of the drives. I created my volume, user and shares as before using the same permissions as before. No deadlock errors. I then did a GUI upgrade to 9.2.1.6. I'm now running the latest build with no errors. I still can't say exactly what caused the deadlock errors to begin with or exactly how I got rid of it. I am convinced that the Samba upgrade was the likely culprit not user ineptitude. Good luck. I hope you find a solution as well.

 

[Kuro Houou]

So you deleted you broke the mirror, then deleted the volume out of freenas, then had a fresh install of 9.2.1.5, re-created the volume and mirror, then re-did the shares. Then upgraded to 9.2.1.6. Well I can probably assume if that's the case that deleting the volume was problem the biggest help, although i am reluctant to try upgrading to 9.2.1.6 again after re-creating my volume and restoring from backup. Took about 15-20 hours to restore last time. I think ill just stay on 9.2.1.5 for awhile until 9.3 comes out just to be safe :) There really wasn't any need on my end to go to 9.2.1.6 anyways besides hoping it would fix my other issue of UPS comm errors, but I am using the work around for that now anyways.

 

[DaveF81]

Jumping in here. Seems that I have the same errors showing up in my logs. Can confirm permissions are set-up as per the docs. I'm not doing anything special (i.e. no LDAP, AD, etc) with regards to authentication, only local users. Nothing special in my Samba configuration either. As far as I can tell, the errors themselves don't appear to affect the client in any way. I've not had any read/write issues with SMB following these messages. I just think Winbindd is being a little verbose.