Dataset content doesn't show up in NFS Share

[user0241233]

Hello,

Explaining the issue here may be a bit tricky but I'd do my best to keep it as clear as possible. My system has AD accounts that are used to login to Linux and Windows systems. These AD accounts are also visible in TrueNAS Scale. This is what I have as the current network storage system:

Above diagram shows that I have dataset/filesystem hierarchy going until 'users' and then each user folder is created with mkdir command. SMB and NFS share is created for 'users' and mounted to Windows and Linux systems respectively. ACLs are applied in a way that each user can view files in all folders but can only do any file/folder modifications/creations/deletions in their own folder. Now, except for the slight issues here and there (one described here), this works fine for the most part.

The issue is I need to create snapshots and ultimately replication of all the data, and snapshots and replications of 'users' dataset are going to be several hundred TBs. Also, in the event of a user losing a certain important file or folder, I'd need to create a clone of 'users', which will take up all the available space, be time taking and overkill.

What I plan to do now is create a dataset for each user so that I can create snapshots and replications on each user level. I have created essentially the same as above diagram, except user[1-3] are now datasets. For some reasons, I now have to create NFS and SMB shares of 'data' instead of 'users' but I suspect that shouldn't change any thing. I have applied the ACLs the same as the previous system.

Now, testing this system, SMB shares work just fine. I can see the files, folders and ACL permissions working nicely. Users can create files/folders in their respective datasets. But, NFS shares don't show any content inside any user dataset. Meaning any file/folder created inside user[1-3] datasets doesn't show. Any user also cannot create any file/folders in their datasets as well.

TrueNAS Scale Version: TrueNAS-SCALE-22.02.4
If this issue can be fixed with an update, I'd only consider updating if there's a stable version available.
I'm not sure what extra info I can provide here. I guess let me know what info you'd need regarding this. Really appreciate any help regarding this issue.
Thank you,

 

[Whattteva]

Now, testing this system, SMB shares work just fine. I can see the files, folders and ACL permissions working nicely. Users can create files/folders in their respective datasets. But, NFS shares don't show any content inside any user dataset. Meaning any file/folder created inside user[1-3] datasets doesn't show. Any user also cannot create any file/folders in their datasets as well.

NFS shares do not cross filesystem boundaries. Each dataset is a distinct filesystem. Think of it as a separate partition. You need separate exports for each dataset.

TrueNAS Scale Version: TrueNAS-SCALE-22.02.4
If this issue can be fixed with an update, I'd only consider updating if there's a stable version available.
I'm not sure what extra info I can provide here. I guess let me know what info you'd need regarding this. Really appreciate any help regarding this issue.
Thank you,

No update will fix this. This is simply how NFS works. If you want the share to cross filesystem boundaries, you're stuck with SMB.

 

[user0241233]

NFS shares do not cross filesystem boundaries. Each dataset is a distinct filesystem. Think of it as a separate partition. You need separate exports for each dataset.

@Whattteva Sorry I'm too much of a newbie into this to understand what you're saying. Do you mean I need to create NFS share for each user dataset (meaning an NFS share for user1, another for user2 and another for user3; instead of just creating NFS share for 'users' dataset) for this to work?

No update will fix this. This is simply how NFS works. If you want the share to cross filesystem boundaries, you're stuck with SMB.

Can you please explain what you mean by cross filesystem boundaries?

 

[Whattteva]

@Whattteva Sorry I'm too much of a newbie into this to understand what you're saying. Do you mean I need to create NFS share for each user dataset (meaning an NFS share for user1, another for user2 and another for user3; instead of just creating NFS share for 'users' dataset) for this to work?

Yes.

Can you please explain what you mean by cross filesystem boundaries?

Think of it as separate distinct entities like 2 separate disks or 2 separate partitions on the same disk.

 

[Daisuke]

Yes.

You can create an usergroup with all 3 users and assign that usergroup the proper dataset permissions, then all 3 users will be able to access the dataset through the mounted NFS share. You can check easy the correct dataset permissions in TrueNAS:

Code:

# stat /mnt/default/opt/backup
# getent group myusergroup

 

[Whattteva]

You can create an usergroup with all 3 users and assign that usergroup the proper dataset permissions, then all 3 users will be able to access the dataset through the mounted NFS share. You can check easy the correct dataset permissions in TrueNAS:

Code:

# stat /mnt/default/opt/backup
# getent group myusergroup

That doesn't work if you'd read the OP. He wants to separate the datasets so replication tasks won't take too long because it's 3 users' data worth of files. He changed to separate layout to keep the snapshots separate.

 

[user0241233]

NFS shares do not cross filesystem boundaries. Each dataset is a distinct filesystem. Think of it as a separate partition. You need separate exports for each dataset.

@Whattteva that being the case, can you please explain why when I create the NFS share of 'data', I can see 'data/users' and 'data/users/user[1-3]' all of which are individual datasets? Each dataset being a separate filesystem, I shouldn't be able to see 'data/users' or 'data/users/user[1-3]' in the first place. The issue only comes at the contents inside each user folder (issue being I cannot see the contents inside user folders), which are individual files/folders.

 

[user0241233]

Also, creating all these NFS shares is not ideal as there is a considerable number of users and that would be a lot to manage. What could be a workaround to making NFS work without having to create individual NFS shares.

 

[Whattteva]

@Whattteva that being the case, can you please explain why when I create the NFS share of 'data', I can see 'data/users' and 'data/users/user[1-3]' all of which are individual datasets? Each dataset being a separate filesystem, I shouldn't be able to see 'data/users' or 'data/users/user[1-3]' in the first place. The issue only comes at the contents inside each user folder (issue being I cannot see the contents inside user folders), which are individual files/folders.

Im assuming what you are seeing isn't the actual data in it. It's just the mount points (empty folders). If you put files in data/users, you'd probably not see it reflected in the data share.

 

[Whattteva]

Also, creating all these NFS shares is not ideal as there is a considerable number of users and that would be a lot to manage. What could be a workaround to making NFS work without having to create individual NFS shares.

I've already told you the workaround. SMB share.