"Strip ACL" option goes missing when nesting datasets with a parent NFS share

[Intel]

I'm having a hard time with NFSv4/SMB permissions when nesting multiple datasets. There seems to be a bug somewhere since the "Strip ACL" option does show up when I create a dataset right below my data_pool but doesn't return after I create children 'datasets' underneath.

I'm following this tutorial: https://youtu.be/QIdy6sR0HrI?t=412

My setup so far:
- created a local_group 'acl-media'
- created 2 users, added to 'acl-media'
- created 'pool/media' used "SMB" type and setup the ACL as the video shows. Used the group as owner of this parent dataset.
- tested by using NFS (mapalluser + mapallgroup to match one of the users and the one group) and SMB. I can write and it shows up with the right UID/GID.

Now it gets hairy, I create sub-datasets "movies" and "tv" I set to inherit expecting things to just work from the ACL settings in the 'pool/media' but this is not the case.

I was trying to 'nuke all the ACLs' to try again and the button doesn't show up at all anywhere in the TrueNAS Scale GUI.


What am I missing?

root@pgn:/gdata# ls -lah
total 20K
drwxrwx--- 8 1000 1001 8 Aug 21 03:49 .
drwxr-xr-x 19 root root 25 Aug 21 03:10 ..
drwxr-xr-x 2 root 1001 2 Aug 21 03:44 dataset
drwxr-xr-x 2 root 1001 2 Aug 21 03:47 movies
drwxrwx--- 2 1000 1001 2 Aug 21 02:48 'New folder'
drwxr-xr-x 2 root 1001 2 Aug 21 03:49 SMBtype
drwxrwx--- 2 root root 2 Aug 21 02:38 tv
drwxrwxrwx 2 1000 1001 2 Aug 21 03:31 yelp
root@pgn:/gdata# touch SMBtype/testme
touch: cannot touch 'SMBtype/testme': Permission denied
root@pgn:/gdata# touch tv/ok
touch: cannot touch 'tv/ok': Permission denied
root@pgn:/gdata# touch movies/alright
touch: cannot touch 'movies/alright': Permission denied
root@pgn:/gdata# mount | grep /gdata
100.64.0.1:/mnt/g2/media on /gdata type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=100.64.0.254,local_lock=none,addr=100.64.0.1)

 

[Intel]

This thread is helpful somewhat: https://www.truenas.com/community/threads/acl-permissions-on-smb-share-new-items-issue.98821/page-2

The problem seems to be rooted in NFS - or at least that's my current focus. SMB seems to be working fine from windows. I am trying to mount a NFS share and force all activity and access to be UID 'proxmox' and GUI 'acl-media' -

From the NFS client:

root@pgn:/# nfs4_getfacl /gdata/
A:fd:OWNER@:rwaDdxtTnNcCoy
A:fdg:GROUP@:rwaDdxtTnNcy
A:fdg:545:rwaDdxtTnNcy
A:fdg:1001:xtnc
A:fd:1001:rwaDdxtTnNcy

root@pgn:/# ls -lah /gdata/
total 20K
drwxrwx--- 8 1000 1001 8 Aug 21 04:34 .
drwxr-xr-x 19 root root 25 Aug 21 04:24 ..
drwxr-xr-x 2 root 1001 2 Aug 21 03:44 dataset
drwxr-xr-x 2 root 1001 2 Aug 21 03:47 movies
drwxrwx--- 2 1000 1001 2 Aug 21 02:48 'New folder'
drwxr-xr-x 2 root 1001 2 Aug 21 03:49 SMBtype
drwxrwx--- 2 root root 2 Aug 21 02:38 tv
drwxrwx--- 2 1000 1001 2 Aug 21 03:31 yelp
root@pgn:/# touch /gdata/movies/sample

root@pgn:/# touch /gdata/movies/sample
touch: cannot touch '/gdata/movies/sample': Permission denied
root@pgn:/# touch /gdata/tv/sample
touch: cannot touch '/gdata/tv/sample': Permission denied
root@pgn:/# touch /gdata/yelp/sample

UID 1000 is TrueNAS first user account I created... so it looks like I can't do anything where 'root' is the primary user that created the folder... but GID 1001 is the owner of 'movies' and 'tv'. It doesn't make sense that 'proxmox' NFS client user (1001) would be denied access since 'proxmox' primary group is 1001 in TrueNAS and forced configured ... I am still scratching my head why 'proxmox' can't work. NFS seems to work if I use the @owner user though....

 

[Intel]

Ok after several hours going at it I think I can conclude that TrueNAS Scale has somewhat of a broken NFS/CIFS implementation. Files are being created by different user-id and group-ids.

I was able to get somewhat of a working implementation by:
NFSv4 settings
- maproot_user = 'proxmox' (bug: because it shows 'root' on NAS filesystem, so this is being ignored)
- maproot_group = 'acl-media' (this seems to work, for new file creation anyway)

CIFS works fine, when a user creates it the NAS filesystem shows it as originating from that user. The one showing 'root' was created via NFS client on linux. That is a bug.

Also about permissions; I couldn't get it to work using local_group ACL. I was forced to explicitly grant each user 'Full control' and this is the only way I was able to have linux-NFS using one username and the windows boxes a different user.

root@nas[/mnt/g2/media/movies]# ls -lah
total 52K
drwxrwx--- 6 gfm acl-media 8 Aug 21 07:39 .
drwxrwx--- 5 gfm acl-media 5 Aug 21 07:20 ..
drwxrwx--- 2 gfm acl-media 2 Aug 21 05:50 'New folder'
drwxrwx--- 2 gfm acl-media 2 Aug 21 07:33 'New folder (2)'
drwxrwx--- 2 gfm acl-media 2 Aug 21 07:38 'New folder (3)'
-rwxrwx--- 1 gfm acl-media 0 Aug 21 05:50 kjjh.txt
-rwxrwx--- 1 root acl-media 0 Aug 21 07:39 mydick
drwxrwx--- 2 gfm acl-media 2 Aug 21 07:33 sdf
root@nas[/mnt/g2/media/movies]#

*Edit: guess it isn't fixed. Linux clients claim a file is on disk on the NFS share but its not really in the filesystem on the NAS or visible on CIFS.

 

[Intel]

When using nested datasets and NFS server at parent dataset there is a bug on TrueNAS Scale that I just filed with more details: https://ixsystems.atlassian.net/browse/NAS-117785

TL;DR: Using a combination of CIFS + NFS shares at the top level 'dataset' then writing to a children dataset folder using NFS client will create a 'split brain' scenario where your NFS clients see files on disk but the NAS and the TrueNAS OS isn't able to see the data at all (new files created from NFS are not shown in NAS OS therefore CIFS clients on Windows don't see anything either).

Only 'zfs list' displays data usage on disk increasing from NFS transfers onto those datasets but CIFS clients and the NAS OS itself have zero knowledge and can't even find the new files created by the NFS service. I really don't know how this was possible but it seems pretty bad.

 

[anodos]

When using nested datasets and NFS server at parent dataset there is a bug on TrueNAS Scale that I just filed with more details: https://ixsystems.atlassian.net/browse/NAS-117785

TL;DR: Using a combination of CIFS + NFS shares at the top level 'dataset' then writing to a children dataset folder using NFS client will create a 'split brain' scenario where your NFS clients see files on disk but the NAS and the TrueNAS OS isn't able to see the data at all (new files created from NFS are not shown in NAS OS therefore CIFS clients on Windows don't see anything either).

Only 'zfs list' displays data usage on disk increasing from NFS transfers onto those datasets but CIFS clients and the NAS OS itself have zero knowledge and can't even find the new files created by the NFS service. I really don't know how this was possible but it seems pretty bad.

knfsd by default does not allow traversal of filesystems unless it is mounted through the v4 pseudoroot. This is a security / design decision in the kernel NFS server in Linux (and I believe most NFS servers).

 

[anodos]

When you create a new dataset, it does not inherit the ACL from the parent datasets. This is OS / FS behavior on new dataset creation. This is because the new dataset is a de-facto separate filesystem. If you were to create an EXT4 volume and mount it under /mnt/g2/media you would also not expect it to inherit permissions from the path it is mounted under.

Since new datasets are created without an ACL, no "strip ACL" option will be visible.

 

[Intel]

When you create a new dataset, it does not inherit the ACL from the parent datasets. This is OS / FS behavior on new dataset creation. This is because the new dataset is a de-facto separate filesystem. If you were to create an EXT4 volume and mount it under /mnt/g2/media you would also not expect it to inherit permissions from the path it is mounted under.

Since new datasets are created without an ACL, no "strip ACL" option will be visible.

Thank you this was insightful. I guess that makes sense.

CIFS seems to behave much different to NFS if the share is at the parent filesystem 'media' in my case, from what I was seeing uploads to 'media/tv' from CIFS/Windows was being stored in the nested filesystem 'tv' as one would expect... NFS was sending the data somewhere (still don't know but 'find' command on the NAS couldn't even find the files)