TrueNAS 13.0-U1 is Now Available

[Breit]

Has anyone opened a bug report for this potential issue?

As far as I'm concerned it is not exactly clear what the issue actually is. But it does not seem to have anything to do with client authentications.

Short recap:
It's more related to ACLs applied in a wrong or inconsistent way in previous TN versions?!
In my case the ACLs were set prior to 13.0 (some in 11.x, some in 12.x) and it only worked in 13.0 but not 13.0-U1.
But if ACLs were set in 13.0, access through SMB shares also works on 13.0-U1.

 

[cobrakiller58]

First time I've ever done a software update and hoped for a problem...

12.0-U8.1 to 13.0-U1 appears to have gone smoothly SMB shares owned by Root/Wheel were immediately accessible

 

[anodos]

As far as I'm concerned it is not exactly clear what the issue actually is. But it does not seem to have anything to do with client authentications.

Short recap:
It's more related to ACLs applied in a wrong or inconsistent way in previous TN versions?!
In my case the ACLs were set prior to 13.0 (some in 11.x, some in 12.x) and it only worked in 13.0 but not 13.0-U1.
But if ACLs were set in 13.0, access through SMB shares also works on 13.0-U1.

I'd have to see an example of an unaltered (no troubleshooting performed) ACL for an affected path. My guess is that we're having problems where ACL has been set to only contain owner@, group@, everyone@ entries and chmod(2) may have been performed on some (or all of paths). chmod(2) can strip relevant write-related permissions over SMB protocol.

 

[indy]

But if ACLs were set in 13.0, access through SMB shares also works on 13.0-U1.

Not true - I started with 13.0-RELEASE from scratch and could not access my shares on 13.0-U1.

I'd have to see an example of an unaltered (no troubleshooting performed) ACL for an affected path. My guess is that we're having problems where ACL has been set to only contain owner@, group@, everyone@ entries and chmod(2) may have been performed on some (or all of paths). chmod(2) can strip relevant write-related permissions over SMB protocol.

That can't be it either - my shares had permissions for specific users and stopped working on U1.

 

[anodos]

That can't be it either - my shares had permissions for specific users and stopped working on U1.

Can you provide example of ACL existing on files that are affected / does not work as expected?

 

[indy]

This is the state from inside a snapshot that was/is not working on 13.0-U1:

Code:

getfacl fileserver.pdf 
# file: fileserver.pdf 
# owner: root 
# group: wheel 
       user:indy:rwxpD-aARWc---:------I:allow 
     user:media-pc:rwxpD-aARWc---:------I:allow 
         everyone@:--------------:------I:allow

I can not access those older snapshots through shadow copy either.


This is the current (working) state after I changed the ownership through the web-interface:

Code:

getfacl fileserver.pdf 
# file: fileserver.pdf 
# owner: smb 
# group: smb 
       user:indy:rwxpD-aARWc--s:------I:allow 
     user:media-pc:rwxpD-aARWc--s:------I:allow 
         everyone@:--------------:------I:allow

 

[anodos]

This is the state from inside a snapshot that was/is not working on 13.0-U1:

Code:

getfacl fileserver.pdf
# file: fileserver.pdf
# owner: root
# group: wheel
       user:indy:rwxpD-aARWc---:------I:allow
     user:media-pc:rwxpD-aARWc---:------I:allow
         everyone@:--------------:------I:allow

I can not access those older snapshots through shadow copy either.


This is the current (working) state after I changed the ownership through the web-interface:

Code:

getfacl fileserver.pdf
# file: fileserver.pdf
# owner: smb
# group: smb
       user:indy:rwxpD-aARWc--s:------I:allow
     user:media-pc:rwxpD-aARWc--s:------I:allow
         everyone@:--------------:------I:allow

Hmm... your user is `smb` ? What are permissions on parent directories (going back to /mnt) on non-working case?

 

[indy]

These are the ACLs of a top-level folder inside a non-working snapshot.

root@truenas[....zfs/snapshot/auto-2022-07-04_00-00-D]# getfacl misc
# file: misc
# owner: root
# group: wheel
user:indy:rwxpD-aARWc---:fd----I:allow
user:media-pc:rwxpD-aARWc---:fd----I:allow
everyone@:--------------:fd----I:allow[/code]
I can not show the ACLs of the dataset/share itself since I do not have snapshots above that level.

 

[MikeyG]

I had the SMB share access issue. My permissions were set to "Advanced" on each dataset. I'm not sure why, but I think it used to be that way by default since I usually would select basic with full permissions when adding my group to it and then see it had switched to "advanced" later on. I noticed in the permissions list, synchronize was unchecked. Applying just the "synchronize" permission to datasets fixed it.

Code:

root@backupnas:/mnt/BackupPool # getfacl Time\ Machine/
# file: Time Machine/
# owner: root
# group: wheel
  group:smb-admins:rwxpDdaARWcCo-:fd-----:allow
         everyone@:--------------:fd-----:allow
root@backupnas:/mnt/BackupPool # getfacl Time\ Machine/
# file: Time Machine/
# owner: root
# group: wheel
  group:smb-admins:rwxpDdaARWcCos:fd-----:allow
         everyone@:--------------:fd-----:allow

After adding synchronize and going back into the permissions screen, the permissions seem to have reverted to Basic with Full Control. On experimenting with another dataset, change advanced to basic with full control permissions and applying also seems to have fixed it.

root/wheel permissions do not seem to apply to this issue for me, as I have a random character string applied as an owner (probably from when this machine was in an AD environment) and after making the changes I described, it still works:

Code:

root@backupnas:/mnt/BackupPool # getfacl Files
# file: Files
# owner: 21104
# group: smb-admins
            owner@:rwxp--aARWcCos:-------:allow
            group@:rwxp--a-R-c--s:-------:allow
         everyone@:rwxp--a-R-c--s:-------:allow

 

[anodos]

I had the SMB share access issue. My permissions were set to "Advanced" on each dataset. I'm not sure why, but I think it used to be that way by default since I usually would select basic with full permissions when adding my group to it and then see it had switched to "advanced" later on. I noticed in the permissions list, synchronize was unchecked. Applying just the "synchronize" permission to datasets fixed it.

Code:

root@backupnas:/mnt/BackupPool # getfacl Time\ Machine/
# file: Time Machine/
# owner: root
# group: wheel
  group:smb-admins:rwxpDdaARWcCo-:fd-----:allow
         everyone@:--------------:fd-----:allow
root@backupnas:/mnt/BackupPool # getfacl Time\ Machine/
# file: Time Machine/
# owner: root
# group: wheel
  group:smb-admins:rwxpDdaARWcCos:fd-----:allow
         everyone@:--------------:fd-----:allow

After adding synchronize and going back into the permissions screen, the permissions seem to have reverted to Basic with Full Control. On experimenting with another dataset, change advanced to basic with full control permissions and applying also seems to have fixed it.

root/wheel permissions do not seem to apply to this issue for me, as I have a random character string applied as an owner (probably from when this machine was in an AD environment) and after making the changes I described, it still works:

Code:

root@backupnas:/mnt/BackupPool # getfacl Files
# file: Files
# owner: 21104
# group: smb-admins
            owner@:rwxp--aARWcCos:-------:allow
            group@:rwxp--a-R-c--s:-------:allow
         everyone@:rwxp--a-R-c--s:-------:allow

Yes, I noted the missing synchronize bit in another ticket. I will probably have a fix ready for testing tomorrow or the day after (there are a few other reported issues I'm still tracking down). There was a default behavior to always map the synchronize bit on ALLOW entries when converting from native format to samba NFSv4 format before generating an NTFS ACL in get_ntacl.

In TrueNAS 13 I refactored our FreeBSD-specific ACL module so that it is common-code with the one we are using in SCALE (which is better from a long-term maintainability standpoint), but unfortunately an error crept in here. It wasn't caught by our CI because we this behavior can't be triggered via normal API calls (requires SMB client to explicitly remove the synchronize bit). It will have coverage going forward.

 

[Volts]

Nice sleuthing!

 

[indy]

I switched my shares back to root/wheel... works fine, as long as the synchronize permission is set.

 

[georgelza]

bit confused... here 13.1 is released but if you enabled the 13.x channel it posts a big banner at the bottom saying this is not production ready.

Are we ok to execute the 13.0 or 13.1 update now.

G

 

[anodos]

I have attached replacement for vfs_ixnas for 13.0-U1 to this post.

WARNING: This is only for 13.0-U1. Using on an incorrect version of TrueNAS may cause smbd service to refuse connections.

Steps to apply:
1. Stop SMB service
2. Either clone current boot environment or make a copy of /usr/local/lib/shared-modules/vfs/ixnas.so
3. Extract attached zip
4. replace /usr/local/lib/shared-modules/vfs/ixnas.so with one from zip
5. Start SMB service

Steps to revert:
1. Stop SMB service
2. Copy stored old version of ixnas.so to /usr/local/lib/shared-modules/vfs
3. Start SMB service

Hashes (generated with `sha256` command on TrueNAS 13):
SHA256 (ixnas.so) = 7f20d35bd00c983efa6ed7ceba0db75119641b6c75a800866486bfee067db20b
^^^ New version
SHA256 (ixnas.so) = 4821e9089bc23e1cbea853e215204f60f78e13810864a2610b633f3801e1c75b
^^^ Original version

How to verify it's working:
1. Find path within SMB share that is affected (no SYNCHRONIZE bit set)
2. Run smbcacls command with appropriate arguments to see ACL on path.

Example:
smbcacls //127.0.0.1/SHARE stuff -U smbuser%Cats -j | jq
This will pretty-print JSON-formatted ACL of path "stuff" within the share "SHARE" authenticating with user "smbuser" with password "Cats".

You should see `synchronize: true` on all ALLOW entries (regardless of what is set in local FS ACL).
Like any shell commands, escaping or quoting path may be required.

Commit in our samba repo: https://github.com/truenas/samba/commit/6e1bae5a4589b500eb69302ff3ea08e3d2e9dff8


NOTE: this fix has been superseded: https://www.truenas.com/community/threads/truenas-13-0-u1-is-now-available.102326/page-4#post-706487

 

[alexkgr]

I have attached replacement for vfs_ixnas for 13.0-U1 to this post.

WARNING: This is only for 13.0-U1. Using on an incorrect version of TrueNAS may cause smbd service to refuse connections.

Steps to apply:
1. Stop SMB service
2. Either clone current boot environment or make a copy of /usr/local/lib/shared-modules/vfs/ixnas.so
3. Extract attached zip
4. replace /usr/local/lib/shared-modules/vfs/ixnas.so with one from zip
5. Start SMB service

Steps to revert:
1. Stop SMB service
2. Copy stored old version of ixnas.so to /usr/local/lib/shared-modules/vfs
3. Start SMB service

Hashes (generated with `sha256` command on TrueNAS 13):
SHA256 (ixnas.so) = 7f20d35bd00c983efa6ed7ceba0db75119641b6c75a800866486bfee067db20b
^^^ New version
SHA256 (ixnas.so) = 4821e9089bc23e1cbea853e215204f60f78e13810864a2610b633f3801e1c75b
^^^ Original version

How to verify it's working:
1. Find path within SMB share that is affected (no SYNCHRONIZE bit set)
2. Run smbcacls command with appropriate arguments to see ACL on path.

Example:
smbcacls //127.0.0.1/SHARE stuff -U smbuser%Cats -j | jq
This will pretty-print JSON-formatted ACL of path "stuff" within the share "SHARE" authenticating with user "smbuser" with password "Cats".

You should see `synchronize: true` on all ALLOW entries (regardless of what is set in local FS ACL).
Like any shell commands, escaping or quoting path may be required.

Commit in our samba repo: https://github.com/truenas/samba/commit/6e1bae5a4589b500eb69302ff3ea08e3d2e9dff8

NOTE: if you run into an issue with these steps, please PM me and I will figure out what went wrong and edit this post accordingly.

i can't download attachment

 

[anodos]

i can't download attachment

Here's a download link from Google drive. If you continue to have problems getting it, please send me a PM.

ixnas.so.zip

drive.google.com

 

[j.lanham]

aaaaaaaaaaa

I have attached replacement for vfs_ixnas for 13.0-U1 to this post.

WARNING: This is only for 13.0-U1. Using on an incorrect version of TrueNAS may cause smbd service to refuse connections.

Steps to apply:
1. Stop SMB service
2. Either clone current boot environment or make a copy of /usr/local/lib/shared-modules/vfs/ixnas.so
3. Extract attached zip
4. replace /usr/local/lib/shared-modules/vfs/ixnas.so with one from zip
5. Start SMB service

Steps to revert:
1. Stop SMB service
2. Copy stored old version of ixnas.so to /usr/local/lib/shared-modules/vfs
3. Start SMB service

Hashes (generated with `sha256` command on TrueNAS 13):
SHA256 (ixnas.so) = 7f20d35bd00c983efa6ed7ceba0db75119641b6c75a800866486bfee067db20b
^^^ New version
SHA256 (ixnas.so) = 4821e9089bc23e1cbea853e215204f60f78e13810864a2610b633f3801e1c75b
^^^ Original version

How to verify it's working:
1. Find path within SMB share that is affected (no SYNCHRONIZE bit set)
2. Run smbcacls command with appropriate arguments to see ACL on path.

Example:
smbcacls //127.0.0.1/SHARE stuff -U smbuser%Cats -j | jq
This will pretty-print JSON-formatted ACL of path "stuff" within the share "SHARE" authenticating with user "smbuser" with password "Cats".

You should see `synchronize: true` on all ALLOW entries (regardless of what is set in local FS ACL).
Like any shell commands, escaping or quoting path may be required.

Commit in our samba repo: https://github.com/truenas/samba/commit/6e1bae5a4589b500eb69302ff3ea08e3d2e9dff8

NOTE: if you run into an issue with these steps, please PM me and I will figure out what went wrong and edit this post accordingly.

Is this going to be rolled into the install at some point after verification?

 

[anodos]

aaaaaaaaaaa

Is this going to be rolled into the install at some point after verification?

Yes. If anyone has an issue with this fix, please PM me and I will try to work out a teamviewer session to investigate on your server so that any additional fixes can be made if required.

 

[adrianwi]

I felt brave yesterday, so upgraded from 12.0-U8 to U8.1 and then to 13.0-U1 without any problems.

I recreated my emby jail using the 13.1-RELEASE template, and then started the job of upgrading the others using iocage upgrade. I encountered a minor issue with the process pausing, but I just needed to enter 'qqq' when the file list is displayed and everything completed fine. I've done 3, so have a few more to do.

Thanks to everyone involved at iXsystems.

 

[jkv1983]

Thanks anodos

After incurring the same issue on updating to 13.0-U1 (everything SMB related was fine for me on 13.0) I followed you steps to patch vfs_ixnas. This resolved the access issue (windows and mac clients could access the SMB shares). However, on 13.0-U1 I am not able to edit ACLs from windows (whereas I could on 13.0). Not sure if others are seeing this. Anyway I've rolled back to 13.0 for now.