Apps Host Path access problem with SMB

[mervincm]

So let me restate to make sure that I understand what IX and TrueCharts are suggesting

-SMB share at the root of the pool is a bad practice
-SMB share and NFS share of the same dataset is a bad practise
-host paths in general are a bad practice, stick with PVC as much as possible, if not PVC then NFS
-multiple apps that map to the same host paths are a bad practice


unless I made an error, I don't see a path forward with TrueNAS.
most people will want an SMB share to their media files. How will they give plex/tdarr/ access to it?

 

[danb35]

most people will want an SMB share to their media files.

I kind of doubt that. Many people, sure, but I kind of doubt that "most" TrueNAS users even use it for media storage. But with that said, I agree that at least one of those "bad practices" is going to have to give for a lot of people.

 

[mervincm]

Fair enough. "most" to "many" is subjective.

 

[danb35]

But AIUI, the issue is ACLs. TrueNAS wants to apply ACLs to SMB shares, and ACLs mean that none of the charts can chmod or chown anything within those host paths, which causes no end of problems. If you know that your SMB shares aren't using ACLs, again AIUI, you're fine to override the validation check and use those as host paths for the apps.

The workaround that's been proposed (i.e., use one or more subdirectories of the shared directory as the host path) relies, IMO, on a bug in the validation--if the dataset uses ACLs, any of its subdirectories will as well, so the same risk is present. IOW, the validation should kick this back the same as it does when you try to mount the share itself as a host path. But it doesn't right now, so at least for now, the workaround works.

 

[mervincm]

I am trying to decide how to move forward. I fit in an area where I understand where teams make suggestions (maximize stability, and minimize issues/tickets) but I am not satisfied by the functionality that is offered within the "rules" I am technical, but unfortunately not quite comfortable enough with TrueNAS to not need a hand every once in a while :)

I wanted app-dedicated service accounts. I like the security, the transparency of it. I like being able to look at a folder, see who the owner is, and see what app last touched that fine. I like being able to open up Netdata and see the resource utilization by app (since user=app) I also like to use host path for some apps. sometimes it is just really convenient to access an app's config folder via an SMB share, tweak a config, or grab a few files as part of a backup. sometimes I have multiple apps that need access to the same data. I understand that this permissions/ACL area is still a bit of a mess, and frankly, it led me to several re-installs before I landed on something that works for me.

I share your perspective 100% on this workaround. I see no less risk.

 

[fenichelar]

I am struggling to understand the logic behind why multiple apps mapping to the same host path is a bad practice. In the default setup, the apps are both being run by the same user, apps, so there are no permissions issues. Why add the overhead of NFS or SMB when the apps can directly access the files? If anything, requiring NFS or SMB makes the permissions more complicated thus increasing the likelihood of a permissions issue.

Example: Plex for media distribution and Bazarr for subtitle management. Instead of mapping the media folder to both apps, the expectation is now that we will use SMB or NFS to share the media folder with both apps. Why?

 

[Daisuke]

SMB share at the root of the pool is a bad practice

I think simply changing the ix-applications dataset permissions to 0640 will address this concern. Currently this is prohibited, I created this thread to find a fix.

 

[artstar]

I am struggling to understand the logic behind why multiple apps mapping to the same host path is a bad practice. In the default setup, the apps are both being run by the same user, apps, so there are no permissions issues. Why add the overhead of NFS or SMB when the apps can directly access the files? If anything, requiring NFS or SMB makes the permissions more complicated thus increasing the likelihood of a permissions issue.

Example: Plex for media distribution and Bazarr for subtitle management. Instead of mapping the media folder to both apps, the expectation is now that we will use SMB or NFS to share the media folder with both apps. Why?

Agreed. Call me naive but in a non-enterprise environment, doing away with the host verification suits me just fine.

Of course, had I been running these applications in an ESXi host, for example, then the only way to connect would be via NFS. Thankfully, for those of us home users who are willing to forego that extra layer of protection, it was kind of iX to allow us to disable/ignore the verification. I can live without the support arising from any potential complications (i.e. none in my use case).

 

[fenichelar]

Agreed. Call me naive but in a non-enterprise environment, doing away with the host verification suits me just fine.

I understand how using the same host path for SMB and applications creates permissions/ACL complexities and therefore is considered a bad practice. But even in an enterprise environment, I don't understand the logic behind why mapping multiple applications to the same host path is considered a bad practice.

 

[Murphy1138]

I understand how using the same host path for SMB and applications creates permissions/ACL complexities and therefore is considered a bad practice. But even in an enterprise environment, I don't understand the logic behind why mapping multiple applications to the same host path is considered a bad practice.

Bluefin just broke my Plex app, had to set the apps user to have modify rights to my media data set, then in the environment variables on the app add PLEX_UID = 568
PLEX_GID = 568

Then reload the app and now can access my media again.
I also access this via smb

 

[NickF]

-SMB share at the root of the pool is a bad practice

Doing anything at the root of a pool is bad practice. You should store all your different data classes inside of separate datasets. Movies and TV shows can be one, Software repos can be another, stuff like that. There are a host of reasons why this is the recommend design.

SMB share and NFS share of the same dataset is a bad practise

Yes, because things like case sensitivity and differing ACLs between *nix and Windows are a thing... If in an environment where Windows hosts are involved, you should just use SMB.

host paths in general are a bad practice, stick with PVC as much as possible, if not PVC then NFS

If you are going to go through the trouble of separating your application form your host system through containerization, and then you immediately give that application direct access to files on the host system, then why not run the application directly on the host OS in the first place?

-multiple apps that map to the same host paths are a bad practice

Exclusive Files Locks are a thing that can happen. https://en.wikipedia.org/wiki/File_locking

So yeah, it's going to cause problems sometimes.

Containerization is great. But it's not magic. The same rules and principles that apply to normal file access and sharing apply, and the layers of abstraction that exist in containerization may even make file sharing between containers and the host more complicated than without in the first place. That's why PVCs exist in the fist place.

If you find it too troublesome, it may behoove you to just run full fat virtual machines instead. No one is forcing containerization on folks, and infact, VMs are still a perfectly valid deployment model for everything you are containerizing in a homelab anyway.

 

[Daisuke]

Doing anything at the root of a pool is bad practice. You should store all your different data classes inside of separate datasets. Movies and TV shows can be one, Software repos can be another, stuff like that. There are a host of reasons why this is the recommend design.

I agree but until @anodos releases the SMB patch which fixes the current issues with specific datasets, what is your proposed fix? If you take the time to set the correct dataset permissions, this is a good temporary solution.

Containerization is great. But it's not magic. The same rules and principles that apply to normal file access and sharing apply, and the layers of abstraction that exist in containerization may even make file sharing between containers and the host more complicated than without in the first place. That's why PVCs exist in the fist place.

From official Kubernetes documentation, you can safely use a hostPath volume between multiple containers/apps, if you set the permissions to readOnly mounts when dealing with multiple users. However, if you assign read/write permissions to a unique apps user, you will not encounter any issues. Is up to the end-user how he/she secures the data accessed by multiple containers. This feature is required by many apps, they need a way to read/write data from a common volume.

The concern is having ix-applications dataset isolated, it should not accessed by anyone except root user. Right now, that dataset is allowed to be read by anyone and permissions cannot be changed. See thread.

I'm not saying what you stated is not accurate, I agree with your post. IX released all these SMB related changes, leaving users into hot water. cc @morganL @Kris Moore

The apps user should:

• not be able to access ix-applications dataset in any shape or form, see thread
• be able to read/write data using a hostPath volume set as SMB share, with volume accessible by multiple containers

 

[Bl00dWolf]

I have found a work around. if you delete the SMB share then start the app you want you can then remount the share once it is running. this is kind of a pain to do every time you need to restart a app but this way you wont have to uncheck the "Validate host path" setting.

Thx, that helped me.

But it's very stupid in general, this situation.
For example, I have qbittorrent, it worked fine, what should I do now? I need an SMB share and that qbit has access to it.

 

[NickF]

I agree but until @anodos releases the SMB patch which fixes the current issues with specific datasets, what is your proposed fix? If you take the time to set the correct dataset permissions, this is a good temporary solution.

I totally understand your frustration. I share similar frustrations with different issues. I think in a lot of ways, they are trying to build the plane while it's in the air, and we are seeing the results of that here.


The way I see it, there are a host of ways to "solve" the problem but it really depends on the quantity and size of the datasets we are talking about. The parameters of the solution need to be consistent with the workflow in question.

As an example, lets imagine you have a dataset which needs to be accessed by several programs, but is only ever written to by a single application. If the dataset is small enough, you could use ZFS send/receive. If you have enough RAM, you could even do deduplication to minimize the loss in storage.

This type of deployment would be fine for something like radarr and Plex, as an example of one of the things @mervincm in this thread mentioned. Plex is capable of looking inside of nested folders for things, and from Plex's perspective over SMB, the nested dataset is just a nested folder.

So if "dataset_a_read_only" is nested inside of dataset _b, your Public Domain torrented, or otherwise legally obtained videos inside of that nested dataset will show up. Once you've seeded to the ratio you want, you can move the files from dataset_a_write to dataset_b. Snapshots will replicate, and they will be removed from dataset_a_read_only, so Plex won't have duplicates.

You can have SMB enabled on dataset_b by doing it this way. No more ACL problems, since it is literally a different dataset.

This also gives you a bit more power in terms of keeping an eye on how long you are allowing folks from the internet to connect back to you through whatever torrent swarms you are in.

Also, for the record, I think that containerizing applications that need access to a common set of data makes things unnecessarily complex. If you were to simply host all of the things that need access to a certain dataset inside of a single VM, alot of these problems would go away.

 

[Daisuke]

If you were to simply host all of the things that need access to a certain dataset inside of a single VM, alot of these problems would go away.

Very nice post and logic. However, I refuse to use VMs, but I hear you, been a Linux sysadmin for last 20 years and SRE for last 10. I stopped using VMs 10 years ago, since Docker and Kubernetes took over.

 

[NickF]

Very nice post and logic. However, I refuse to use VMs, but I hear you, been a Linux sysadmin for last 20 years and SRE for last 10. I stopped using VMs 10 years ago, since Docker and Kubernetes took over.

Not from my standpoint...

I still use VMs for far more things than Kubernetes. Although, I am using Kubernetes more and more. One tool does not replace the other in my mind, Kubernetes is like having a multibit screw driver, whereas VMs are a trusty Phillips #2. ;)

 

[Daisuke]

I need an SMB share and that qbit has access to it.

See the temporary solution I posted into thread, SMB Shares section.

 

[Mr.Dan]

Thx, that helped me.

But it's very stupid in general, this situation.
For example, I have qbittorrent, it worked fine, what should I do now? I need an SMB share and that qbit has access to it.

I found another work around. I have a dataset called downloads with 3 files in it Completed, incomplete and watch, i gave qbit access to the 3 files and then created a smb share for the main dataset this fixed my problem

 

[Murphy1138]

This all sounds like change for changes sake.
Been running torrent and Plex apps on the same dataset with separate folders and a single smb share for years and it has been working fine no issues.
I’m not creating different shares or datasets for apps, it’s borking the user experience.

 

[stepheng67]

So what is the solution or general censes of what to do? I was going to role back to TRUENAS SCALE 22.02.4 until there's an official fix/solution. But none of my application are running because I get an error that the Kubernetes service is not running. So I know it's a BIG no no, but I unchecked Enable Host Path Safety Checks, get get running for the short term.