Question about using VLANs with containers

[Vertigo 7]

I dinked around for a little bit in 21.04 and I didn't see a way to pick an interface to use with Plex, for example. Setting up the VLAN and bridge interfaces was a fairly similar and straight forward process similar to Core, but I could not find anything that allowed me to specify which interface a container would use like you could for a jail in Core.

Did I miss something? I know a lot of documentation is still pending but some basic config stuff like this would be immensely helpful.

 

[ornias]

With k8s (the backend of the scale containers) you don't usually directly connect pods/containers to interfaces. You select one interface globally and deal with routing internally.

There are ways of dealing with this, but thats going byond stock k8s and this prone to issues and breaking.

For example:
The "Big Blue Button" allows users to do precisely what you want. But thats still not a good idea. It adds multiple layers of abstraction and things that can break to the native k8s stack.

 

[Vertigo 7]

Wouldn't that create a problem for people and organizations that isolate VLAN traffic? I'd hate to have to spin up VMs just to run something that should run perfectly fine in a container

 

[ornias]

Wouldn't that create a problem for people and organizations that isolate VLAN traffic? I'd hate to have to spin up VMs just to run something that should run perfectly fine in a container

You can still select the LAN/VLAN that all containers connect to.
For home use (the target of SCALE Apps) a single demilitarised VLAN for containers (which is currently possible) is perfectly fine.
For more advanced features including container clustering, you would need to wait for the second release of SCALE ETA somewhere 2022.

The current default deployment type is NodePort, which is just not build usecases and isn't even clustering capable. There are ways of setting this up in a k8s native fassion, they just aren't included in SCALE Apps yet.

---
Note:
The only option currently is to use "External Interface" with the SCALE "Bug-Blue-Button". This bypasses k8s somewhat and hooks a pod directly onto a specific external network interface. I highly advice against using it, but it can be done.

The reason this cannot be done with k8s-native setups like services and ingress, is a known limitation of Multus CNI it looks like:

Kubernetes service for multus interface · Issue #466 · k8snetworkplumbingwg/multus-cni

What would you like to be added: This issue is used for update of multus service abstraction and its implementation will be under Kubernetes network plumbing working group repo. Currenlty multus ne...

github.com

That being said:
To suit your usecase I pushed an update yesterday and published a GUI today, that should also enable you to use the "External Interface" selection, for all TrueCharts Apps :)

 

[Vertigo 7]

I think I'm following you. My particular use case in my homelab has jails in 3 different and isolated VLANs, for example Homebridge in my IoT VLAN, Plex in a media VLAN, and a jail for running Handbrake in a general use VLAN. I'm dropping traffic from each subnet to other local subnets at my firewall and I don't want endpoints on those subnets to be able to reach any other local subnet, especially the IoT subnet. But if I can't do something similar in the current iteration of Scale, then it is what it is and I'll continue to use Core for the time being. Taking full advantage of VLAN trunking is of keen interest to me.

 

[ornias]

Well, the way to do that with containers is just using k8s isolation features, instead of vlans (just using one demilitarised VLAN).
Though those are also not implemented as of yet.

Like I said: What you wanted is possible with ExternalInterfaces and I don't expect any other solution for your usecase in like a LONG LONG time if ever.

 

[PackElend]

To suit your usecase I pushed an update yesterday and published a GUI today, that should also enable you to use the "External Interface" selection, for all TrueCharts Apps :)

is this still possible?
I can only find this option available in https://truecharts.org/charts/stable/docker-compose/

Like I said: What you wanted is possible with ExternalInterfaces and I don't expect any other solution for your usecase in like a LONG LONG time if ever.

is there any hope that it will be possible in Scale once?
https://github.com/k8snetworkplumbingwg/multus-cni/issues/466 is finished but https://github.com/k8snetworkplumbingwg/multus-service is still in development but https://github.com/k8snetworkplumbingwg/multus-cni has been around for awhile.