After upgrade from 11.3 u3 to 12 - freenas kernel: pid 4361 (smbd), jid 0, uid 0: exited on signal 6

[andrema2]

Hi

I just upgrade one of my servers to TrueNas 12 beta 1 coming from the latest 11.3 version. The upgrade went ok, no errors.

After the boot the time capsule share using SMB was not appearing in any of my Macs.

I tried to restart SMB service and after it the share appeared normally. When I tried to reconnect on the Time Machine TrueNas started to show the error below multiple times.

Jul 1 13:50:43 freenas kernel: pid 4361 (smbd), jid 0, uid 0: exited on signal 6

I upgraded the zpool, but it didn't change the result. Any idea ?

 

[anodos]

Hi

I just upgrade one of my servers to TrueNas 12 beta 1 coming from the latest 11.3 version. The upgrade went ok, no errors.

After the boot the time capsule share using SMB was not appearing in any of my Macs.

I tried to restart SMB service and after it the share appeared normally. When I tried to reconnect on the Time Machine TrueNas started to show the error below multiple times.

Jul 1 13:50:43 freenas kernel: pid 4361 (smbd), jid 0, uid 0: exited on signal 6

I upgraded the zpool, but it didn't change the result. Any idea ?

A core file should have been generated under /var/db/system/cores/ (smbd.core). Download it, zip it up, and PM it to me with a debug file "system->advanced->save debug".

 

[colmconn]

You should definitely file a bug report about that. Don't forget to include a debug.

 

[anodos]

You should definitely file a bug report about that. Don't forget to include a debug.

I'll need the core file as well. PM through the forums works as well in this case. We'll get these issues sorted out before release. Have multiple macs backing up to nightlies without seeing this issue, which is why it's not fixed yet, but I'll try to take care of it pretty quickly. If it's just a VFS module, then I can probably send a hot-patched version. Otherwise, you'll have to wait for a proper fix.

 

[anodos]

Quick update on this one. Crash is caused by possible bug related to shared-mode locks when netatalk locking is enabled in vfs_fruit. Have WIP fix, but won't be hot-patchable. Workaround is to not have mixed-protocol shares (AFP/SMB) shares or add the auxiliary parameter fruit:locking = none for the mixed-protocol share (on the SMB-side).

 

[Yorick]

I upgraded the zpool, but it didn't change the result.

Well, boldly going forward it is for you :). Upgrading the pool means changing the feature flags. It carries no risk for the pool, but it does mean it becomes readonly for ZFS installations that don’t support those feature flags, such as all versions of FreeNAS.

 

[PDM]

I upgraded from v11.3-U5 to v12.0-U1 yesterday.

Samba keeps panicing when I try to connect from my TV (which has always worked since FreeNAS v8 then with v9 onwards with Aux Setting "ntlm_auth = yes").

Do I need to add more Aux Settings now?

Code:

lanman auth = no
ntlm auth = yes
client NTLMv2 = yes
client lanman auth = no
client plaintext = no 

Panic:

Code:

===============================================================
Dec 13 02:14:53 freenas 1 2020-12-13T02:14:53.991047+01:00 freenas.local smbd 5013 - - [2020/12/13 02:14:53.990993,  0] ../../lib/util/fault.c:80(fault_report)
Dec 13 02:14:53 freenas 1 2020-12-13T02:14:53.991125+01:00 freenas.local smbd 5013 - -   INTERNAL ERROR: Signal 11 in pid 5013 (4.12.9)
Dec 13 02:14:53 freenas 1 2020-12-13T02:14:53.991194+01:00 freenas.local smbd 5013 - -   If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting
Dec 13 02:14:53 freenas 1 2020-12-13T02:14:53.991269+01:00 freenas.local smbd 5013 - - [2020/12/13 02:14:53.991219,  0] ../../lib/util/fault.c:86(fault_report)
Dec 13 02:14:53 freenas 1 2020-12-13T02:14:53.991336+01:00 freenas.local smbd 5013 - -   ===============================================================
Dec 13 02:14:53 freenas 1 2020-12-13T02:14:53.991413+01:00 freenas.local smbd 5013 - - [2020/12/13 02:14:53.991362,  0] ../../source3/lib/util.c:830(smb_panic_s3)
Dec 13 02:14:53 freenas 1 2020-12-13T02:14:53.991515+01:00 freenas.local smbd 5013 - -   PANIC (pid 5013): internal error
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.000216+01:00 freenas.local smbd 5013 - - [2020/12/13 02:14:54.000092,  0] ../../lib/util/fault.c:265(log_stack_trace)
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.000359+01:00 freenas.local smbd 5013 - -   BACKTRACE: 6 stack frames:
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.000451+01:00 freenas.local smbd 5013 - -    #0 0x801296217 <log_stack_trace+0x37> at /usr/local/lib/samba4/libsamba-util.so.0
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.000522+01:00 freenas.local smbd 5013 - -    #1 0x802f9b656 <smb_panic_s3+0x56> at /usr/local/lib/samba4/libsmbconf.so.0
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.000593+01:00 freenas.local smbd 5013 - -    #2 0x801296007 <smb_panic+0x17> at /usr/local/lib/samba4/libsamba-util.so.0
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.000662+01:00 freenas.local smbd 5013 - -    #3 0x8012963ee <log_stack_trace+0x20e> at /usr/local/lib/samba4/libsamba-util.so.0
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.000732+01:00 freenas.local smbd 5013 - -    #4 0x801295fe9 <fault_setup+0x59> at /usr/local/lib/samba4/libsamba-util.so.0
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.000828+01:00 freenas.local smbd 5013 - -    #5 0x8099dbc20 <_pthread_sigmask+0x530> at /lib/libthr.so.3
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.001217+01:00 freenas.local smbd 5013 - - [2020/12/13 02:14:54.001060,  0] ../../source3/lib/dumpcore.c:315(dump_core)
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.001386+01:00 freenas.local smbd 5013 - -   dumping core in /var/db/system/cores
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.001462+01:00 freenas.local smbd 5013 - - 
Dec 13 02:14:54 freenas kernel: pid 5013 (smbd), jid 0, uid 0: exited on signal 6
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.215830+01:00 freenas.local smbd 5014 - - [2020/12/13 02:14:54.215702,  0] ../../lib/util/fault.c:79(fault_report)
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.215962+01:00 freenas.local smbd 5014 - -   ===============================================================
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.216053+01:00 freenas.local smbd 5014 - - [2020/12/13 02:14:54.215995,  0] ../../lib/util/fault.c:80(fault_report)
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.216139+01:00 freenas.local smbd 5014 - -   INTERNAL ERROR: Signal 11 in pid 5014 (4.12.9)
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.216211+01:00 freenas.local smbd 5014 - -   If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.216290+01:00 freenas.local smbd 5014 - - [2020/12/13 02:14:54.216239,  0] ../../lib/util/fault.c:86(fault_report)
Dec 13 02:14:54 freenas 1 2020-12-13T02:14:54.216364+01:00 freenas.local smbd 5014 - -   ===============================================================

 

[Sebastian Steinmetz]

I've got a similar issue when backing up from a macOS mojave machine using an smb based TimeMachine share. Would be curious to understand what's causing it, and how to prevent it from happening.

 

[anodos]

I've got a similar issue when backing up from a macOS mojave machine using an smb based TimeMachine share. Would be curious to understand what's causing it, and how to prevent it from happening.

It's a race on a tevent timer being freed in a cleanup function for durable handles on share disconnect IIRC. I have fixed it in 12.0-stable, but you will have wait until U1.1, revert back to release, or PM me for the WIP fix update (depending on how severely it's impacting you.

 

[Sebastian Steinmetz]

Thanks for the swift reply. The obvious question: When do we expect U1.1 to be pushed ? I could wait some more I guess.

 

[PDM]

It's a race on a tevent timer being freed in a cleanup function for durable handles on share disconnect IIRC. I have fixed it in 12.0-stable, but you will have wait until U1.1, revert back to release, or PM me for the WIP fix update (depending on how severely it's impacting you.

...and my issue would also be fixed in U1.1?

 

[anodos]

...and my issue would also be fixed in U1.1?

I think that's probably the case, if you want to test the fix for U1.1, you can feel free to PM me for an install ISO for testing.

 

[PDM]

I think that's probably the case, if you want to test the fix for U1.1, you can feel free to PM me for an install ISO for testing.

The WIP U1.1 does indeed address the panics I was having! Thank you.

Is it advisable to leave the default "handle_aio_read_complete file [...] max=65536 and nread=65536" as-is? Or should I try to optimize for my environment?

 

[tkempf]

I'm on 12 U1.1 and am still seeing this problem during timemachine backups. Is there still an issue open on this, or should i create a new one ?
Does it make sense to test with the fruit:locking = none param ?

Jan 22 03:01:06 betterone kernel: pid 88874 (smbd), jid 0, uid 0: exited on signal 6
Jan 22 06:45:54 betterone kernel: pid 93097 (smbd), jid 0, uid 0: exited on signal 6
Jan 22 07:47:30 betterone kernel: pid 95932 (smbd), jid 0, uid 0: exited on signal 6
Jan 22 08:19:01 betterone kernel: pid 96110 (smbd), jid 0, uid 0: exited on signal 6

 

[anodos]

I'm on 12 U1.1 and am still seeing this problem during timemachine backups. Is there still an issue open on this, or should i create a new one ?
Does it make sense to test with the fruit:locking = none param ?

Jan 22 03:01:06 betterone kernel: pid 88874 (smbd), jid 0, uid 0: exited on signal 6
Jan 22 06:45:54 betterone kernel: pid 93097 (smbd), jid 0, uid 0: exited on signal 6
Jan 22 07:47:30 betterone kernel: pid 95932 (smbd), jid 0, uid 0: exited on signal 6
Jan 22 08:19:01 betterone kernel: pid 96110 (smbd), jid 0, uid 0: exited on signal 6

Fix was delayed due to need to address an openzfs bug (see U1.1 release notes). If for some reason rolling back to an unaffected release isn't possible, or if you wish to test the fix, you can send me a PM asking for the patch and testing instructions.