SMB causing CPU to climb up to 100% after 9.2.1.2 update

[Abel408]

Hello everyone,

Ever since switching our file server over from a Mac AFP server to FreeNAS, we have had a horrible time with browsing speeds and reliability. Before I continue, I want to share with you the information of our 2 nas servers (Both of which are having problems).

(Yes I know I should have more memory, I just ordered 24gb for this server)

I started a thread here about the slowness we were experiencing while browsing on Mac OSX. Someone recommended upgrading from 9.2.0 or 9.2.1. I did and that sped things up a little bit I think... Although I'm not really sure because sometimes I still have issues. BUT ever since upgrading on Friday, the CPU keeps climbing to 100%. It seems like one smbd process gets stuck and causes one fo the cpu cores to get stuck at 100%. Over time, another smbd process and core will get stuck, until all of them get stuck and the server is pretty much non-responsive. It happens pretty quick too during the work day. Never happens during after hours. I then have to reboot the server and sometime have to hard reboot it by holding in the power button. After rebooting the server, everything seems to be fast a smooth, but it wont be long until the server is practically useless.

Here is a screenshot of one of the stuck smbd processes:

And here is examples of the cpu climbing to 100%

I put the server in place on the 3rd. On the 7th, I upgraded from 9.2.0 to 9.2.1.2 and I also turned on auto-tune I believe. Here is a screenshot showing you what 1 of the servers looks like before the 7th:

Restarting the cifs service does not help. Killing the stuck process also does not work. The process is still there. Although a kill -9 <PID> will work. I need help ASAP. Should I try reverting back to 9.2.0? Is that even possible?

Thanks for any help you can provide. Let me know if you need anymore information.

 

[cyberjock]

You can try going back to 9.2.0 assuming you have a 9.2.0 config file.

But, I wouldn't spend much time troubleshooting the problem until you get your RAM in. Can't tell if your problem is insufficient RAM or not, so troubleshooting may be pointless as upgrading your RAM may make the problem go away.

 

[Abel408]

Thanks for the quick response. Seeing that I did just fine with the RAM with 9.2.0, I don't think that's the issue. I'm also seeing the same issue with the server that has 8gb of ram which should be enough for what I am doing.

 

[Abel408]

I've upgraded the memory to 24gb and I'm still seeing the same issues. Does anyone else have any ideas? Should I file a bug report?

 

[dlavigne]

Should I file a bug report?

Yes.

 

[indy]

I am having the same problem.
This looks to be the related bug report: https://bugs.freenas.org/issues/4474

As a short term help 'kill -s 9 pid' seems to help.

 

[Abel408]

I am having the same problem.
This looks to be the related bug report: https://bugs.freenas.org/issues/4474

As a short term help 'kill -s 9 pid' seems to help.

Good to know I'm not the only one. I thought it was because I have a lot of mac clients and mac seems to do a lot of weird stuff when browsing through directories. Hopefully we will get a fix soon. I've been issueing a kill-9 <pid> command every couple of hours to keep the servers alive.

 

[DaveC]

Same here, found this thread after noticing smbd consuming 2 CPU's at 100%, and had been for a few days..... (v9.2.1 running on 6 core CPU w/8G RAM)

Disabled CIFS, killed the pid's, and restarted CIFS... all is happy again.

Even better willly this weekend the two Winders VM's using CIFS will be moved to NFS.

 

[Yatti420]

This will be fixed shortly in 9.2.1.3..

 

[mka]

You're sure? I'm running 9.2.1.3 and yesterday evening saw an 100% smbd process while doing nothing over Samba.

 

[Yatti420]

Maybe 9.2.1.3.1.. Let's hope..

 

[Cryptopher]

I am having a similar problem.
I currently have three processes running, all smbd, which are keeping my cpu pegged at 100%.

Running 9.2.1.3

Any ideas on ETA of a fix for this?

 

[mka]

my current top output... 17hours CPU time straight of smbd

Code:

  PID USERNAME    THR PRI NICE  SIZE    RES STATE  C  TIME  WCPU COMMAND
15672 root          2 103    0  349M  42880K CPU0    0  17.0H 100.00% smbd
77732 root          1  20    0 16556K  2508K CPU1    1  0:00  0.10% top

 

[cyberjock]

You guys know that developers don't go here, right? If you really are having a problem that's a bug, the bugs.freenas.org website awaits. There's already a ticket on this, so please read up on it.

 

[anodos]

Please see discussion here: https://bugs.freenas.org/issues/4462
Note that there is a workaround in place for 9.2.1.4 and will be fixed in a later release.
I have encountered this problem on my machine and have mitigated it by adding the following parameter to my CIFS config under auxiliary parameters:
kernel change notify = no

This change disables libinotify, and constitutes the aforementioned workaround present in 9.2.1.4.

 

[Hugin]

I was having a lot of problems with this same thing. I upgraded to the new 9.2.1.4 Beta today and the issue has disappeared! I'd definitely recommend upgrading to the Beta now if you're needing an immediate fix. The full release is estimated to be available in about 2 weeks.

9.2.1.4 Beta Announcement

 

[cyberjock]

For anyone that tires 9.2.1.4, if the problem isn't resolved PLEASE post in the ticket. If nobody tells us it is broken we'll assume it is fixed. We want 9.2.1.4 to be the last 9.2.1 release.