Recommended Plex workflow with sharing when accounting for HostPath validation?

[bcat]

They are not, please read the entire SMB Shares section from my guide.

I have, and I've just reread it now. Let's take your example there---and please do let me know if I'm not parsing it correctly.

As I understand it, you propose a pool at /mnt/default containing two datasets:

• /mnt/default/media: shared via SMB
• /mnt/default/media/center: used as a hostpath volume for containers

This setup passes hostpath validation in the current version of Bluefin, and I'm not disputing that.

What I am saying is that in this configuration, the risks that hostpath validation attempts to mitigate still exist. To give a couple examples (not an inclusive list):

1. If a container maintains a database inside the hostpath volume (e.g., /mnt/default/media/center/foo.db), that database can still be accessed over SMB (e.g., from Windows via \\TRUENAS\media\center\foo.db), corrupting the application's state.
2. If a container expects POSIX permission semantics for /mnt/default/media/center, but /mnt/default/media and its descendent datasets use NFSv4 ACLs (as is default in TrueNAS SCALE for datasets created with "Share Type" set to SMB, since NFSv4 ACLs mirror the Windows permission model), then the container may still behave unexpectedly when reading and writing files within the hostpath volume.

These are the exact same kinds of issue that can arise if a single dataset (e.g., /mnt/default/media) is used as both an SMB share and a hostpath volume. As far as I can tell, factoring out a child dataset of the share to use as the hostpath volume has no practical effect other than allowing the current implementation of hostpath validation to pass. It doesn't actually address the risks that hostpath validation "solves" (as I understand per the release notes and various iXsystems/TrueCharts posts on the subject).

To be clear, I'm not saying that structuring datasets the way you propose is bad. What I'm saying instead is that (at least as I read your post) telling ~everyone to structure their datasets in that way can be a bit dangerous, as it makes folks think they've mitigated the risks of files accessible from both shares and hostpath volumes, when in fact the risks still exist. That's why I think iXsystems took the right route by implementing a UI toggle (with a scary warning about unsupported configurations) that folks can choose to exercise after they've thought through these implications.

 

[chri5]

Glad I found this thread.
I recently upgraded to bluefin, and have had several issues since then, one being sharing the plex library folder using SMB
It seems to be a basic use case for everyone using a windows workstation, and it is 'out of the box' broken.
From a UI point of view, I'm sure truenas can do better: The plex app stays indefinitely deploying, without any useful information presented to the user.
I had to dig inside containerd.log to find the culprit was the SMB share.
In order to fix this issue, official documentation said to 'better plan your dataset', but I don't see how this helps in the case of plex.
The current workaround of disabling the 'safety check' works, but it's not app-specific, so I think it defeats the purpose of adding this validation.
All the users using plex and windows will probably end up with this 'safety check' disabled globally.

Is there a better way to access the Plex library files from windows without using a SMB share ?

Thanks

This post was forgotten in the last page so I'll bump it.
I need access to my plex library via smb. Sharing music on my server is its primary function. Can anyone suggest how I can use an smb share as my plex library?

 

[oblivioncth]

@bca

That is basically exactly how I was going to describe it.

Just because the share can acess the containers HostPath indirectly doesn't mean it isn't still accessing it and potentially allowing users to change the permissions of files that the container uses. The flip side is also true, because the share is likely using NFSv4 ACLs, the container gets forced into this as well and may try to modify permissions with chmod in a way that wont work out.

Essentially, this setup only works around the way that iXSystems implemented the HostPath check, and does not actually work around the situation that caused them to add the check in the first place. A container and share are ultimately still touching the same files, just with the extra step of going through a child dataset.

I would imagine though that many containers actually work fine in this setup though as long as you make sure aclmode is set to Passthrough. Or you could even try disabling ACLs on the share entirely, though this might cause issues for users of the share. aclmode isn't touched on in the guide at all however.

To draw a parallel for potential better clarity, to me what this is like is if you had code like this:

Code:

#include <iostream>

int main()
{
    bool deadCode;
    std::cout << "Hello World";
}

and someone saw there was a warning that 'deadCode' was unused. Then they said "oh I'll just disable the warning in my compiler (i.e. disabling HostPath validation), but then you said, "No that isn't the proper way! Instead do this:"

Code:

#include <iostream>

void dummy(bool b) { (void)b; }

int main()
{
    bool deadCode;
    dummy(deadCode);
    std::cout << "Hello World";
}

Sure, this will get rid of the "unused variable" warning, but it doesn't change the fact that fundamentally the variable still isn't actually used. Now this example is slightly flawed in that the obvious answer here is just to remove the unused variable, but ignore that possibility the sake of my argument.

My point is that the setup suggested in Daisuke's thread (which I am actually using) is basically just "tricking the compiler" as above. It works around the detection only, but not the actual circumstance.

 

[Daisuke]

If a container maintains a database

Database containers and any other container have primary data stored on PVC, inaccessible to SMB shares. Sure, if you change that standard, things will break guaranteed. Also is clearly stated into Kubernetes documentation that hostPaths must be mounted read only, therefore you cannot write any data,

At home, I have a Scale server for media and a cluster of 8 Raspberry Pi running a Kubernetes cluster with MetalLB and Longhorn, for development and my own Kubernetes learning. Two control planes and six workers.

If a container expects POSIX permission

We are talking about SMB shares only, which default to POSIX.

I agree with your logic but for the purpose of this discussion, we are talking about Plex and SMB shares. If an user like @NugentS has complex configurations, he will already understand all this and address it properly, without need from community.

 

[oblivioncth]

This post was forgotten in the last page so I'll bump it.
I need access to my plex library via smb. Sharing music on my server is its primary function. Can anyone suggest how I can use an smb share as my plex library?

It is a bit complicated, but that's what's being debates here.

You more-or-less have three options:
1) Don't use SMB or HostPath and instead NFS share to Plex and Windows.
2) Use the same dataset as the SMB share and Plex HostPath and disable HostPath validation. This brings all the caveats discussed previously.
3) Use the workaround shown in Daisuke's thread where you make a child dataset, SMB share the parent dataset and use the child dataset for the HostPath of Plex. But know that likely this is nearly the same as (2) and could stop working in a future update, requiring you to resort to 2 anyway.

While not necessarily OK for all containers, if you do 2 or 3 and just make sure that the HostPath for Plex is mounted as Read Only, you're pretty safe as the container won't be able to touch permissions anyway. Plex doesn't need write permissions to its library directories unless you care to delete media through the Plex UI.

I will also note that it's probably a good idea (and Daisuke does cover this) to set the runAs user of your Plex container to the same user that owns the SMB shared dataset. This would help if you decide to have the HostPath mounted as R/W.

 

[oblivioncth]

We are talking about SMB shares only, which default to POSIX.

Maybe this used to be true, but for me at least when you set the "Share Type" parameter from Generic to SMB it changes the ACL type to NFSv4. What's more interesting is that if you then within the same dialog set it back to POSIX before hitting save, when you go to edit the dataset again you'll find that it ignored you and set the type to NFSv4 anyway.

 

[bcat]

Maybe this used to be true, but for me at least when you set the "Share Type" parameter from Generic to SMB it changes the ACL type to NFSv4. What's more interesting is that if you then within the same dialog set it back to POSIX before hitting save, when you go to edit the dataset again you'll find that it ignored you and set the type to NFSv4 anyway.

Indeed, and the TrueNAS SCALE docs recommend setting "Share Type" to "SMB" (NFSv4 ACLs) for SMB shares:

There is more discussion in the docs' ACL primer; however, both the official TrueNAS/SMB tutorials and the naming of the relevant bits in the UI (e.g., the "ACL Type" dataset property being displayed as "SMB/NFSv4") suggest to me that NFSv4 ACLs should be preferred for SMB shares in the absence of a particular reason otherwise.

(Ancedata warning) NFSv4 ACLs work nicely with Plex media in my personal setup, because I can grant the builtin_users group read/write access to files in my media datasets while restricting the plex user to read-only access. (Similar things could be done with POSIX ACLs, but the inheritance behavior of NFSv4 ACLs makes them preferable for my use case.)

 

[Daisuke]

Maybe this used to be true

Generic is POSIX, as shown into guide. Works for Windows, Linux and Mac. Yes, they show you a warning popup prior the change you mention.

I’m going to stop hijacking this thread, we are talking everything but the thread subject.

 

[mistermanko]

1) Don't use SMB or HostPath and instead NFS share to Plex and Windows.

Common misunderstanding. NFS shares that link to the same path as the host path you set up for your k3s container - Plex i.e. - will result in the container not able to start. It's basically the same with NFS and SMB, sadly.

 

[oblivioncth]

Common misunderstanding. NFS shares that link to the same path as the host path you set up for your k3s container - Plex i.e. - will result in the container not able to start. It's basically the same with NFS and SMB, sadly.

You misunderstand what I mean actually. I meant not using host path at all and giving the container access to your data over NFS. I believe this is only possible with truecharts containers.

Generic is POSIX, as shown into guide. Works for Windows, Linux and Mac. Yes, they show you a warning popup prior the change you mention.

I’m going to stop hijacking this thread, we are talking everything but the thread subject.

I mean it is relevant. I started this thread because I saw no 100% "catchless" approach to sharing plex media with the container and users, so the setup of dataset permissions is pertinent to that. The more fleshed out the thread is the more info they have to figure out what they want to do for their setup anyway.

I do see you said to use POSIX ACLs in that thread, which is probably a good idea when mixing with containers. Just a shame the official TrueNAS docs don't mention this and simply say "use the SMB option in the dataset when making an SMB share", since they are trying to avoid talking about mixing shares with containers period.

 

[mistermanko]

You misunderstand what I mean actually. I meant not using host path at all and giving the container access to your data over NFS. I believe this is only possible with truecharts containers.

Yes, after re-reading your comment, you're right.
Although NFS-everything can work, when dealing with databases for example, you're gonna have bad time.

 

[chri5]

set the runAs user of your Plex container to the same user that owns the SMB shared dataset. This would help if you decide to have the HostPath mounted as R/W.

Need some help with this.

Who is, or how do I find the runAs user of the Plex container? (Sorry I'm a noob although I've had all this working fine for a year before Bluefin)

Fresh install...

I have 2 pools.

1. fast (2 SSDs mirror) for apps and VMs
2. slow (2 spinners mirror) for media

The only dataset I have so far is fast/ix-applications

Reading @Daisuke's recommended Pools and Datasets I should first create a 'media' SMB dataset...

so in my: slow/media (not done this yet)

Then create another dataset (not SMB) inside slow/media called 'center'...

and use 'slow/media/center' for the Plex library.

I've tried the above but windows users can't see 'center' and Plex can't find 'center' for its library.

 

[oblivioncth]

Yes, after re-reading your comment, you're right.
Although NFS-everything can work, when dealing with databases for example, you're gonna have bad time.

This is an important caveat I should have mentioned. For something like Plex library files its not and issue at least.

Need some help with this.

It's under the Security and Permissions section of the pod configuration. Though I'm using truecharts so I'm not sure if it's in a different location with the official pods, if you're trying to use those.

Assuming your datasets were configured as shown in the guide, it may then be a permissions issue. It can be done differently if you mess with ACLs but the easiest way, as shown in the guide, is to have the runAs user, owner of the dataset, and user you log in with to access the share all be the same.

Despite the caveats/concerns with the method in the guide that have been covered here, I do use it and it works fine for me.

 

[chri5]

This is an important caveat I should have mentioned. For something like Plex library files its not and issue at least.


It's under the Security and Permissions section of the pod configuration. Though I'm using truecharts so I'm not sure if it's in a different location with the official pods, if you're trying to use those.

Assuming your datasets were configured as shown in the guide, it may then be a permissions issue. It can be done differently if you mess with ACLs but the easiest way, as shown in the guide, is to have the runAs user, owner of the dataset, and user you log in with to access the share all be the same.

Despite the caveats/concerns with the method in the guide that have been covered here, I do use it and it works fine for me.

Finally got it working, Thanks for your help (and a few others in this thread). Certainly wasn't easy but I now have a windows share that Plex can use as its library with no complaints from truenas.

 

[oblivioncth]

A mod in the Truecharts Discord stated that the workaround of using a child dataset as a host path who's parent is set as a share has already been patched out in the nightlies. I asked for a reference to the change, but haven't (and may never) received a reply. Tried searching through Jira and the commits in the GitHub repo but couldn't find anything that proves this is the case; however, I did find the addition of a known issue that supposedly can occur when using this method, though I haven't seen anyone mention that they've run into this:

| 22.12.0 | <a href="https://ixsystems.atlassian.net/browse/NAS-119335" target="_blank">NAS-119335</a> | Host path validation issue | There is a potential for the system to enter an endless validation loop resulting from an issue with the Kubernetes Setting option to Validate host path if you have an SMB or NFS share using a host path similar to what an application uses, for example the share uses /mnt/tank/media and the app uses /mnt/tank/media/movies.This issue can exist if the share is one directory below the application host path. Work around is to either not use the Validate host path option or to not use a host path too similar, for example use /mnt/tank/media for the share and /mnt/tank/movies for the application. | Targeted 22.12.2 |

Docs-3966 Update Known Issues with Host Path Validation Issue · truenas/documentation@2ae0b8f

This commit updates the Known Issues table to include information on the Kubernetes Setting Validate host path option possibly causing an endless validation loop if a share (SMB or NFS) has a host ...

github.com

Hopefully the person on Discord gets back to me as it would be nice to confirm ahead of time that my/our suspicions were indeed true.

 

[chri5]

Really don't like how this is turning out. The use case is clear, user needs to put music in plex library. How do they do that if they can't see the plex library folder? I don't want to wrestle with the command line, I want to copy and paste and be done with it.

 

[oblivioncth]

Really don't like how this is turning out. The use case is clear, user needs to put music in plex library. How do they do that if they can't see the plex library folder? I don't want to wrestle with the command line, I want to copy and paste and be done with it.

Basically at the end of the day you (and I and everyone else) is going to have to disable host path validation for Plex specifically, which really shouldn't have problems since plex doesn't try to chmod things, more so if you make it read only for Plex (still r/w on the share). As mentioned early in the thread the head engineer of iXsystems does this themselves.

 

[bcat]

Basically at the end of the day you (and I and everyone else) is going to have to disable host path validation for Plex specifically, which really shouldn't have problems since plex doesn't try to chmod things, more so if you make it read only for Plex (still r/w on the share). As mentioned early in the thread the head engineer of iXsystems does this themselves.

FWIW, I think the other reasonable alternative for many cases (not databases, to be clear) is "mount from within the container using NFS". It looks like TrueCharts makes this straightforward (they've added options in their chart config to mount NFS shares without the underlying container needing to do anything special to support this), and for something like streaming media, I would imagine performance would be just fine.

I don't personally do that, because enabling NFS on the host just to share one thing with one container seems overkill (vs. simply disabling hostpath validation); however, it's worth emphasizing the NFS route is an option if disabling hostpath validation bothers someone.

In any case, neither of these alternatives involve potentially invasive changes to dataset or SMB share organization, which is what I primarily care about. :)

 

[oblivioncth]

FWIW, I think the other reasonable alternative for many cases (not databases, to be clear) is "mount from within the container using NFS". It looks like TrueCharts makes this straightforward (they've added options in their chart config to mount NFS shares without the underlying container needing to do anything special to support this), and for something like streaming media, I would imagine performance would be just fine.

I don't personally do that, because enabling NFS on the host just to share one thing with one container seems overkill (vs. simply disabling hostpath validation); however, it's worth emphasizing the NFS route is an option if disabling hostpath validation bothers someone.

In any case, neither of these alternatives involve potentially invasive changes to dataset or SMB share organization, which is what I primarily care about. :)

Just came here to edit my post to note this as well, since I had way back in the OP.

I agree though that if it would just be for the one container I'd personally not bother, but it is an option.

Also worth nothing is that apparently NFS has issues with databases so this solution shouldn't be used for any container that needs to operate on one.

 

[truecharts]

A short response might be warranted:

Our advised solution is to combine SMB+NFS storage and use our "NFS" storage option inside the App storage configuration.
This is not (likely) to be disabled in the future and does not have the same issues has hostPath does.

Fears for databases over NFS are, mostly, unwarranted with TrueCharts:
- We always heavily advice to never change main App config storage, often aptly named "config", away from the "PVC" default.
- Storage for other databases is not user accessible.

When in doubt our support staff is always available to help out on discord :)