Accessing container's advanced settings

Oskari Rauta

Cadet
Joined
Mar 10, 2016
Messages
8
I am looking for advanced settings of container which aren't available in GUI.

Previously in TrueNAS CORE, I did set up a jail which provided AirPrint support for my non-airprint printer. I could had just made it available with avahi, but I instead made my own printer server and disabled network functionality of printer and connected it to my TrueNAS server with USB cable. Then I made a hotplug script that started/stopped jail if printer came online or went offline.

Now I'd like to do this with a container. Problem is that to access devices, containers need special permissions (mounting devfs for example) which aren't available in GUI. I would love to see these permissions and other advanced settings in GUI, but most likely that ain't going to happen any time soon.

So, what about CLI? I tried to access settings of my container through CLI with failure. Is there a way to setup containers with more advanced setup that seems to be available visible for eye in GUI?

Or any other suggestions to archieve my goal?
 

waqarahmed

iXsystems
iXsystems
Joined
Aug 28, 2019
Messages
136
@Oskari Rauta can you please be specific as to what type of advanced settings are you referring to ? I understand the devfs example but that is FreeBSD and in Linux we have udev and we can have udev inside the container as well but would need some more context as to what exactly we want to do and what knobs would we have to configure to do that.

Looking forward to hearing from you. Thank you
 

Oskari Rauta

Cadet
Joined
Mar 10, 2016
Messages
8
I need to access usb peripherals through /dev

With devfs I mean devtmpfs mounting that usually mounts similar device tree that is available in container's host.
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
I am looking for advanced settings of container which aren't available in GUI.
SCALE works with Helm and k8s, which doesn't really work with container settings. It's all a bit abstracted away, even ignoring the abstraction of the GUI.


Previously in TrueNAS CORE, I did set up a jail which provided AirPrint support for my non-airprint printer.
Jails vs k8s, is a world of difference... I think it's best not to talk too much about how things from a jail can be done in k8s. Because it's like explaining totally different solutions to a problem.

I could had just made it available with avahi, but I instead made my own printer server and disabled network functionality of printer and connected it to my TrueNAS server with USB cable. Then I made a hotplug script that started/stopped jail if printer came online or went offline.
Spinning up containers when printers come and go, seems like a bad idea... prone to failure and totally not needed, as USB already has hotplug which just about every decent print server should be able to deal with afaik.

Now I'd like to do this with a container.
Forget the start-stop script to begin with.

Problem is that to access devices, containers need special permissions (mounting devfs for example) which aren't available in GUI.
You need to set the "securityContext.privileged", to "True" and create a hostPath mount to the "/dev/" usb device references. There are some examples on how to do it in the TrueCharts repository. (if you have a good reference for a print server, i'm open for suggestions there too)
See: https://github.com/truecharts/truecharts

I would love to see these permissions and other advanced settings in GUI, but most likely that ain't going to happen any time soon.
Besides securityContext.Privilaged, everything else (hostPath mounts) is already available in the GUI.

So, what about CLI? I tried to access settings of my container through CLI with failure.
Containers? can you please use correct k8s terminology...

Is there a way to setup containers with more advanced setup that seems to be available visible for eye in GUI?
Use Helm directly, like everyone else

Or any other suggestions to archieve my goal?
It looks like you think the new apps/containers are somewhat similair to standard docker or jails. It's not. Its Helm-Kubernetes based. On linux, so forget about all the jail steps needed to connect devices to jails. It's so much different that I doubt it's going to help you to know how to do that.


@waqarahmed
You might want to have securityContext.privileged available in the GUI next version. See our charts which utilise USB devices. It's not pretty, but it works solid to put in a hostPath to the device and setting privilaged to "true"
 

Oskari Rauta

Cadet
Joined
Mar 10, 2016
Messages
8
SCALE works with Helm and k8s, which doesn't really work with container settings. It's all a bit abstracted away, even ignoring the abstraction of the GUI.

Jails vs k8s, is a world of difference... I think it's best not to talk too much about how things from a jail can be done in k8s. Because it's like explaining totally different solutions to a problem.

True. But when it comes to service as I described, it's as simple that it shouldn't be problem to either. Solution with jail CAN be solution with image, but I'd rather make it different this time.

Spinning up containers when printers come and go, seems like a bad idea... prone to failure and totally not needed, as USB already has hotplug which just about every decent print server should be able to deal with afaik.

Forget the start-stop script to begin with.

You need to set the "securityContext.privileged", to "True" and create a hostPath mount to the "/dev/" usb device references. There are some examples on how to do it in the TrueCharts repository. (if you have a good reference for a print server, i'm open for suggestions there too)
See: https://github.com/truecharts/truecharts

Besides securityContext.Privilaged, everything else (hostPath mounts) is already available in the GUI.

Containers? can you please use correct k8s terminology...

Use Helm directly, like everyone else

It looks like you think the new apps/containers are somewhat similair to standard docker or jails. It's not. Its Helm-Kubernetes based. On linux, so forget about all the jail steps needed to connect devices to jails. It's so much different that I doubt it's going to help you to know how to do that.

Missed those examples. Sorry. I am using cups with Samsung driver and avahi to advertise AirPrint.
This means that cups and avahi services need to stop when printer is off-line and come back up when printer is up.

Sorry for incorrect terminology, it's just that I've built my own with Dockerfile, and have used to call it container. Even though it uses kubernetes or whatever available, result is same, it runs my custom combination of applications and scripts.

Well there is some similarity, but I wouldn't say jails are the same thing. Actually I would had loved to go with TrueNAS CORE, but I decided to give scale a try just for.. Well, accordingly to GUI, docker images :)
Why? Because I might be wrong here, but due to that they take a lot less space, I think they are more light-weight than jails in BSD (not forgetting difference..) and for that, I'd like to think that they consume less resources. I am a fan of alpine based... Images. And minimal images when ever possible.

Explanation about jail version of this was just how I archieved this with CORE. Maybe I should had gone the simpler way and just tell that I need to provide direct access to device nodes from my container and receive notifications from udev, without explaining further why I need this.

For the moment, I haven't yet created my own repository with helm charts, will be looking to that when I have a image that does what I want, since it's not necessary until base image has necessary functionality. Sure, with charts I can customise installation of image, for example, set admin password for printer's server or something like that if I want, but I am not really attempting to create a public repository competing with existing ones, I just have my own requirements and rather roll my own here; I'll check later, but probably with this, I can create my image as planned.

You seem eager to participation. You are welcome :)
My plans actually consist of running 2 apps in my setup, GitLab and this printer server. Hopefully GitLab will eventually find it's way to truecharts, but more likely I still want to go with minimalism and go with Alpine based setup with that too.

Since you seem to have a lot of expertise with images, could you explain to me why most images are based on bigger base images like for example, ubuntu? I see the point when you want to provide a service that needs more components that are not available in the smaller distribution, but for example, exim4 server, or caching server, wouldn't it be better to go with minimal setup instead of larger distribution that comes with more than necessary? I like images for their fast deployment, small size, simplicity (when image is just running one service for example) and independence from other components in system, but some of this goes away when using bigger distribution. So far all my requirements can be done with smaller distributions, even my web server (not running locally and base os is CentOS) runs all it's services visible from internet with Podman containers.
 

peter.m

Dabbler
Joined
Jan 1, 2021
Messages
41
You're going to run into problems with this. First of all running 2 apps in the same container is doable, but bad practice. In your case, you need 3 - avahi, cups, and dbus for them to communicate with each other. Problem is, dbus looks for a session bus... it gets messy. There was actually a container that i used for a while, let me look it up - https://github.com/DrPsychick/docker-cups-airprint. I remember needing to tweak the dockerfile for this, but can't remember why. You should note that because of dbus this container will take more than 30s to even start up. Now that you have that working, You'll need to kubernetize it, which will be an ordeal in of itself...

I just realised this may not work at all, because there's no way under k8s that I know of to make that container have a dedicated ip hidden under macvlan like you could under straight docker. @ornias correct me on this if I'm wrong. You'll need it, because you can't have 2 avahi-daemons running from the same IP (so network: host doesn't help here at all)

My advice is to spin up a lightweight VM just for this (alpine comes to mind). However, the current scale implementation doesn't let you forward usb ports - yet.
 

Echaleon

Cadet
Joined
Feb 17, 2021
Messages
1
@waqarahmed
You might want to have securityContext.privileged available in the GUI next version. See our charts which utilise USB devices. It's not pretty, but it works solid to put in a hostPath to the device and setting privilaged to "true"

I'd like to second this sentiment. I'm trying to deploy Zerotier to a container to remotely access the server, but to do so properly requires being able to set the container as privileged and to add NET_ADMIN and SYS_ADMIN capabilities, at least with the few helm charts and containers I have seen: https://github.com/leunamnauj/kubernetes-zerotier-bridge

As far as I understand it, I'll have to deploy as a straight helm chart for now with the CLI, no?
 

waqarahmed

iXsystems
iXsystems
Joined
Aug 28, 2019
Messages
136
@ornias we can certainly expose that in ix-chart in the next version which would enable users to configure privileged security context while launching docker images.
 

waqarahmed

iXsystems
iXsystems
Joined
Aug 28, 2019
Messages
136
@Oskari Rauta talking about having some device in your container, you can use host path volumes ( Launch Docker Image and then Host Path Volumes section ). Please let me know if that does not full fill your use case or you run into issues :)
 

Oskari Rauta

Cadet
Joined
Mar 10, 2016
Messages
8
You're going to run into problems with this. First of all running 2 apps in the same container is doable, but bad practice. In your case, you need 3 - avahi, cups, and dbus for them to communicate with each other. Problem is, dbus looks for a session bus... it gets messy. There was actually a container that i used for a while, let me look it up - https://github.com/DrPsychick/docker-cups-airprint. I remember needing to tweak the dockerfile for this, but can't remember why. You should note that because of dbus this container will take more than 30s to even start up. Now that you have that working, You'll need to kubernetize it, which will be an ordeal in of itself...

I just realised this may not work at all, because there's no way under k8s that I know of to make that container have a dedicated ip hidden under macvlan like you could under straight docker. @ornias correct me on this if I'm wrong. You'll need it, because you can't have 2 avahi-daemons running from the same IP (so network: host doesn't help here at all)

My advice is to spin up a lightweight VM just for this (alpine comes to mind). However, the current scale implementation doesn't let you forward usb ports - yet.

You are most likely assuming correctly, I didn't think of this issue either.
Running avahi, cups and dbus in different containers might be tricky, but it should be unified pod for them (or cluster in this case?)
But if this pod, or container if all are running inside same container, cannot get LAN ip like vm's, there's definetly going to be a big issue :/

@waqarahmed thanks. I need to think if there's a solution out of pointed issue- sure there is, but most likely it's a bit too hacky solution (another ip for ethernet device and point k8s to use that ip instead, but this doesn't really sound like something that I am willing to do...)
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
You are most likely assuming correctly, I didn't think of this issue either.
Running avahi, cups and dbus in different containers might be tricky, but it should be unified pod for them (or cluster in this case?)
But if this pod, or container if all are running inside same container, cannot get LAN ip like vm's, there's definetly going to be a big issue :/

@waqarahmed thanks. I need to think if there's a solution out of pointed issue- sure there is, but most likely it's a bit too hacky solution (another ip for ethernet device and point k8s to use that ip instead, but this doesn't really sound like something that I am willing to do...)
AVAHI is a whole lot of trouble running on k8s.

You're in over your head, get used to the system before trying to build complex things yourself.
 

Oskari Rauta

Cadet
Joined
Mar 10, 2016
Messages
8
AVAHI is a whole lot of trouble running on k8s.

You're in over your head, get used to the system before trying to build complex things yourself.

It really seems that way. I am seriously considering going back to Core instead.
Containers are great, but they seem to not offer me (atleast very easily) what I require them to do for me.
I do realize that SCALE is still alpha, but it's propably better to review it when first stable release comes out.
Well, haven't decided anything yet, but using jail to provide what I want, is more likely going to work out better in my case.
 
Top