Recommended Plex workflow with sharing when accounting for HostPath validation?

[oblivioncth]

Yes, another Bluefin thread related to HostPath validation...

I'm trying to move my setup, that I thought was pretty simple, from ESXi that had an Unraid VM with my storage and Ubuntu VM with some services like Plex, but now I'm not sure that was a good idea because of this unfortunate HostPath validation issue.

To be clear, I want to do things "the right way", and keeps things as clean as possible as long a nothing conflicts with my absolute requirements. So, I understand why this check was implemented, and would like to abide by the restriction, but I'm honestly not sure I even can while achieving close what I had before.

There are many specific sub-questions I have in regards to trying to setup my datasets correctly to work with Plex, but let me just keep this simple:

How do people that use Plex on Bluefin regularly add media to their library?

In my situation the person that needs to add media is not that tech savvy, and so the solution needs to be something straightforward that they can do without me.

The way I see it there are several approaches, but because of restrictions most are not actually possible.

1) SMB share the dataset to both users and the Plex container. Containers can't mount SMB shares
2) SMB share the dataset to users, NFS to the Plex container. Mixing protocols leads to potential permissions and file locking issues. In this particular instance I could probably get away with a read only NFS share to Plex, but that might not fly for other containers and still has the overhead of NFS.
3) Set the SMB share as the HostPath and disable validation. As I said, I'd like to not do this. It's not supported and can lead to issues.

So really in the end, I only see three actually options, two of which seem quite flawed to me and the third is very unoptimal:

1) Use the SMB share as the HostPath but keep SMB disabled. Then, when adding files, kill Plex, enable the share and add files, then kill the share and restart Plex. This is awful for obvious reasons as is a nonstarter for me because it's too complicated for the end user. But regardless of that, AFAIK this violates the purpose of HostPath validation anyway because you're ultimately still having the container and share interact with the dataset, with the potential of stomping on each others permissions, just not simultaneously.

2) Set the SMB share as the dataset root (note not the root dataset) as I was planning anyway, and then set Plex's HostPath to a subfolder within that path. According to here and here this is possible to do while leaving HostPath validation on, but this really seems like an unintentional hack to me that still violates the same principle and will likely be patched out in a future release.

3) Share the dataset via NFS only to both the container and users. I believe this is only possible with TrueCharts containers, has the overhead of NFS with large files (which of course movies for plex all are), and requires mounting in windows via the CLI, which means I always have to set up new shares for the users on their systems. This one technically isn't a deal breaking but its quite unsatisfying.

Or finally, say YOLO and disable the validation anyway.

While I get that all of this was done with enterprise in mind, Plex is enormously popular and there's lots of people saying they use it on SCALE, but at this point I'm just not seeing how it's feasible in a clean way because of this. There's plenty of tutorials you can find for setting it up, but most of it is on Angelfin or just shows people adding their media dataset via HostPath with all of their files already magically in the dataset and they make no comment on when the time comes to modify their library.

So again, how do people that use Plex on Bluefin handle managing media in their library while also abiding by these restrictions?

 

[browntiger]

You know this has been discussed about 1000 times. @Daisuke posted a full detail how to install this. If I repeat this you probably still not going to listen... and write two page long stories.

You can do whatever you want. But
1) It is NOT a good idea to declare the root of SMB share as a Kubernetes volume. 2) It is NOT a good idea to turn off the safety 3) There is always a chance of locking. Probably slim to none for a home server.

Most of us got it working and moved on. [@Daisuke made put a good effort to explain this to everyone in details and pictures, how to set this up, and sorry, very little point of repeating]. My setup predates, but nearly identical to his.

 

[oblivioncth]

You know this has been discussed about 1000 times. @Daisuke posted a full detail how to install this. If I repeat this you probably still not going to listen... and write two page long stories.

You can do whatever you want. But
1) It is NOT a good idea to declare the root of SMB share as a Kubernetes volume. 2) It is NOT a good idea to turn off the safety 3) There is always a chance of locking. Probably slim to none for a home server.

Most of us got it working and moved on. [@Daisuke made put a good effort to explain this to everyone in details and pictures, how to set this up, and sorry, very little point of repeating]. My setup predates, but nearly identical to his.

Oh I've read all the other discussions, but from my perspective none of them had satisfactory conclusions. And oh geeze sorry that I was detailed with my concerns, would you rather I have said "plz help... cant share hostpath datasets. app no work. fix?".

Clearly you're the one that didn't read here. By Daisuke's solution you're referring to:

2) Set the SMB share as the dataset root (note not the root dataset) as I was planning anyway, and then set Plex's HostPath to a subfolder within that path. According to here and here this is possible to do while leaving HostPath validation on, but this really seems like an unintentional hack to me that still violates the same principle and will likely be patched out in a future release.

I literally linked to his post in that statement, and in that statement I note the issue is that as far as I can tell this workaround seems to be an unintended bug that I'd assume will be patched out of future versions. Just because the HostPath is a subfolder doesn't mean it isn't mucking around with the same permissions in the same filesystem, it's just one level down. I wouldn't want my setup to rely on something like that and "just move on". While software is ever changing, I'd like to implement a solution that is not likely to quickly need changes.

If you can convince me this isn't that case and isn't likely to go way or cause issues, I'll happily use this approach. Based on your comments in the second thread I linked I see that this is the setup you are using.

I get your frustration with a topic being hammered to death, but please don't act like I'm not trying to delve deeper and touch on nuances of the topic that aren't completely addressed in the existing conversations.

In that second thread, the following post shared my exact concern with this workaround and the only responses to it were people (you included) just saying that "it works" with nothing but crickets from iXSystems on whether or not it's a bug.

SOLVED - Plex won't allow its data path to be SMB shared

Hello! I'm building a new Truenas Scale server with 22.12-RC1. It's almost done except that I ran into an issue where Plex/Docker won't let its data path or the subdirectories to SMB shared. Here are the errors: Nov 26 16:15:22 truenas dockerd[7797]: time="2022-11-26T16:15:22.172278717-08:00"...

www.truenas.com

To me that doesn't bode well.

 

[bcat]

My hot take on this: If you're a home user, you understand your hostpath volumes and their permissions settings, then don't be afraid to turn off hostpath validation. As you noted, it's enabled by default because TrueNAS caters towards enterprise use cases (and, I suspect, iXsystems doesn't want folks reporting bugs against TrueNAS due to container misconfigurations... which is totally reasonable). But for a non-enterprise use case, I see nothing wrong with disabling the validation other than a lack of support (which I don't generally expect anyway, as a non-paying customer of an open source project).

For me, I have exactly one container, namely Plex, with exactly one hostpath volume, which contains my family's media library. It's shared over SMB and permissions are handled via NFSv4 ACLs. I run Plex as its own user that I've added to the relevant dataset ACLs (with read-only access). It all works perfectly, and I don't see any reason to complicate matters by introducing NFS into the mix.

Is this the right solution for everyone? Definitely not. But the whole point of configuration options is that they are configurable. IMO you've explored the solution space thoroughly, and if the solution that works best for you is "disable validation", then do that and don't stress over it. :)

I literally linked to his post in that statement, and in that statement I note the issue is that as far as I can tell this workaround seems to be an unintended bug that I'd assume will be patched out of future versions. Just because the HostPath is a subfolder doesn't mean it isn't mucking around with the same permissions in the same filesystem, it's just one level down. I wouldn't want my setup to rely on something like that and "just move on". While software is ever changing, I'd like to implement a solution that is not likely to quickly need changes.

FWIW, this matches my understanding as well. The fact that a hostpath volume under a shared dataset happens to pass validation today honestly feels like a bug in the validation (which I wouldn't be surprised if iXsystems fixed in a future version of TrueNAS). For me personally, restructuring shares to pass validation today doesn't seem beneficial since this seems more like an unintended gap in the hostpath validation than an intended path.

 

[oblivioncth]

Thanks for the sensible reply.

I'm glad someone else agrees with the sketchiness of the aforementioned workaround.

I also particularly appreciate the first hand account of someone confirming they used Plex with a SMB shared dataset and NFSv4 ACLs without issue (read only of course).

After your response I said screw it and implemented an acceptable (to me) layout that happened to allow me to use the "workaround", just for the slight edge it gives over outright disabling HostPath validation (e.g. the rare case of actually getting support); however, I've done this knowing that it essentially causes the same conflict as you and I agree on.

So far so good. Plex is working really well, GPU passthrough and RAM disk included.

In the end you're right. Since this software is much more so directed at enterprise and more "hard core" users, in the end I might have to cut slight corners for the sake of convenience in a home setting. As long as I make note of the lines I'm crossing so that I know what may be at fault if something does go wrong, then I should be ok in the end :).

 

[Daisuke]

How do people that use Plex on Bluefin regularly add media to their library?

See also this thread.

 

[oblivioncth]

See also this thread.

Lol...

I understand people get tired of noobs asking the same questions over and over and assume they're just rehashing the same topic again without anything different to say, but that thread of yours was already referenced several times in the previous posts.

I made this one because in my opinion the workaround you shared, while good in the sense that it avoids the conflict warnings, ultimately still suffers from violating the same principle (app and share touching the same data) and likely only works due to an oversight from iXsystems.

See above.

 

[Daisuke]

that thread of yours was already referenced several times in the previous posts

I apologize, I was reading your OP.

 

[mistermanko]

Well it speaks for itself when the head of engineering of iX himself disables host path validation for his Plex...

Reddit - Dive into anything

www.reddit.com

 

[oblivioncth]

Well it speaks for itself when the head of engineering of iX himself disables host path validation for his Plex...

Reddit - Dive into anything

www.reddit.com

Ah, really seems like they just double-downed on this direction in an official capacity for meeting the standards of their enterprise segment (which was basically already mentioned here), but that for home use as long as you pay attention to what containers you're running and how they interact for permissions, turning it off is ultimately not that bad of an option assuming you're willing to accept responsibility for any issues.

 

[ozoo]

Glad I found this thread.
I recently upgraded to bluefin, and have had several issues since then, one being sharing the plex library folder using SMB
It seems to be a basic use case for everyone using a windows workstation, and it is 'out of the box' broken.
From a UI point of view, I'm sure truenas can do better: The plex app stays indefinitely deploying, without any useful information presented to the user.
I had to dig inside containerd.log to find the culprit was the SMB share.
In order to fix this issue, official documentation said to 'better plan your dataset', but I don't see how this helps in the case of plex.
The current workaround of disabling the 'safety check' works, but it's not app-specific, so I think it defeats the purpose of adding this validation.
All the users using plex and windows will probably end up with this 'safety check' disabled globally.

Is there a better way to access the Plex library files from windows without using a SMB share ?

Thanks

 

[Daisuke]

All the users using plex and windows will probably end up with this 'safety check' disabled globally.

Until things break really ugly. Is a very bad idea to disable the check. Do you understand why iXsystems asks you not to disable that option? What can't you organize your datasets the right way, I explained into my guide why is important not to disable hostPath validation. It literally takes 5 minutes to move all your data to different datasets, as explained into my guide.

As a side note, @truecharts team do not offer any kind of support, if you disable that check. Really happy to see they took this decision.

 

[bcat]

Until things break really ugly. Is a very bad idea to disable the check. Do you understand why iXsystems asks you not to disable that option? What can't you organize your datasets the right way, I explained into my guide why is important not to disable hostPath validation. It literally takes 5 minutes to move all your data to different datasets, as explained into my guide.

If this were truly an option nobody should use, then it wouldn't be configurable at all. But given that 1) the configuration option exists in the first place, 2) what it does and why is well documented at this point, and 3) iXsystems themselves has provided various materials explaining how it can be disabled, I honestly think "very bad idea" is a bit extreme.

Also, for what it's worth, I personally suspect the organization outlined in your thread only passes hostpath validation because they didn't yet cover the case of a hostpath volume pointing to a child dataset of a shared dataset. The potential issues of accessing the same files both over SMB/NFS and from a container's volume (as well as of NFSv4 ACLs not necessarily playing nice with container expecting POSIX permissions) still exist in your recommended configuration, and hence I wouldn't be at all surprised if the hostpath validation were made stricter to catch this case.

In any case, for me personally, disabling hostpath validation is a more straightforward solution than reorganizing my datasets to pass the validation today (especially as they would still violate the "spirit" of the intended separation of shares from container volumes even after the reorganization.

To be clear, I'm not saying everyone should disable hostpath validation. I'd even be fine saying that most folks shouldn't disable it. But saying that nobody should use a configuration option that was intentionally added by the TrueNAS devs for this exact purpose seems like an unnecessarily strict stance, and I honestly think it causes unnecessary confusion.

Rather than provide a blanket "do not disable this setting", I think a scary warning (as the TrueNAS UI currently shows when you toggle the hostpath validation setting) that discourages folks from flipping the bit without understanding it is a good balance.

As a side note, @truecharts team do not offer any kind of support, if you disable that check. Really happy to see they took this decision.

Per their FAQ, they "will not provide support on things that involve storage" [emphasis mine] in that case. I agree this is totally reasonable. However, nothing prevents one from setting up an app without shared hostpath volumes to repro a bug for them and then disabling hostpath validation again afterwards.

 

[Daisuke]

If this were truly an option nobody should use, then it wouldn't be configurable at all.

Exactly, IXsystems should remove that option. Ideally, hostPath validation should be removed from UI and be left available only through command line. I don't know IXsystems posted that info into release notes. It allowed users to take the easy route and untick a button.

 

[NugentS]

Exactly, IXsystems should remove that option.

No - I would have to completely re-organise the dataset structure. I like that option there.

 

[Daisuke]

I would have to completely re-organise the dataset structure.

It takes 5min to do it, what's the block? I literally moved 14Tb of data in 30 seconds.

 

[bcat]

Exactly, IXsystems should remove that option.

But... why? I don't mean to be stubborn; I just honestly haven't understood from your various posts on the subject:

1. Why we should second guess the product and eng folks who specified, documented, and implemented the UI toggle.
2. How your suggested workaround (reorganizing datasets so the share points at a parent dataset of the hostpath volume) addresses the failure modes that the hostpath validation guards against (since after applying the workaround, the same underlying files are still accessed both from inside containers and over network shares).

 

[NugentS]

It takes 5min to do it, what's the block? I literally moved 14Tb of data in 30 seconds.

Its not the moving of data, its the somewhat complex mesh of shares both NFS and SMB that I use for the various backup apps, containers etc that fill me with a certain amount of concern when the thought of changing it comes up.

I may change it eventually - I just don't want to be forced into doing so

 

[Daisuke]

same underlying files are still accessed both from inside containers and over network shares

They are not, please read the entire SMB Shares section from my guide.

complex mesh of shares both NFS and SMB that I use for the various backup apps, containers etc

I hear you, the cli -c 'app kubernetes update validate_host_path=false' will always be there. You will still end-up creating new datasets.

 

[Daisuke]

Per their FAQ, they "will not provide support on things that involve storage"

Thanks for the link, I will add that into guide. That's what I was referring to.