How to connect Apps to different IP(s) and vlan's !?

[Louis2]

Hello,


I am trying "scale" as alternative for "core". Especially in area of jails/vm's/apps. The problem I am facing is, that vm's, jails and applications, only work
as long as you do not use vlans ...... and do not want to use "localy defined static IP-addresses (IPV4 & IPV6)".

So today a played around a bit with apps.
- I defined a trunk
- with a couple of vlans
- each connected to their own bridge

My intention is to connect such a vlan related bridge with one or more apps
and other apps with another bridge.

I also defined some storage datasets for the apps and did start to define two test apps (nextcloud and plex), expecting that I could connect an app to a bridge and to assign IP-addresses.

Not so ...... and if so ....... it does not work .....

I noticed that there is settings => advance settings .... in the app screen where you can in a limited way define some settings which I had expected with each app!!

However, the settings are generic , so at best I can choose ..... if I change the settings to what I like for app-A of those for app-B .....

That is absolutely not what I had in mind!

Is there any way to link apps to the IP-addresses you have in mind for each particular app.
and if so how (preferable via a GUI!) !!??

 

[aussiejuggalo]

Wouldn't mind knowing this to. I spent 50 odd hours screwing around with Scale before giving up and going back to Core. Deploying containers, adding storage etc. Isn't to bad but the networking is an utter confusing mess and stupidly restricted because they want it all done through the GUI not CLI.

I had all kinds of problems with Plex networking, could never get it accessible via LAN (even with host path), always had to be remote.

 

[truecharts]

On RC1 and before this was only possible on the "BigBlue Button" by the methods provided by iX to linkan App to an IP directly.

On RC2 you could (also) do this on our catalog as well using Kubernetes native methods: by disabling the integrated loadbalancer under the settings and using metalLB instead. Which allows giving out IP's on a per-service basis.

Our guide and development for that is not 100% done yet though, so we would only advice it for users with past kubernetes experience.

 

[truecharts]

Wouldn't mind knowing this to. I spent 50 odd hours screwing around with Scale before giving up and going back to Core. Deploying containers, adding storage etc. Isn't to bad but the networking is an utter confusing mess and stupidly restricted because they want it all done through the GUI not CLI.

That's a bit of an odd statemment actually:
You want it easy, but you don't want to use the GUI that actually hides most of the complications for you?

When deploying an App, you almost never actually touch the internal network at all, is just exposed to the host by default, just like docker. That goes for both our Apps, As the Apps by iX-Systems, as well as the "big blue button".

I had all kinds of problems with Plex networking, could never get it accessible via LAN (even with host path), always had to be remote.

The problems you are having here are 99% of the time because Plex uses a complex clusterfuck itself. We've had multiple reports about this and all of them got resolved. It's not related to the Apps system by iX Systems really, more the way Plex at times has trouble dealing with NAT.

 

[Louis2]

Tja what to say,

I am not at all an expert in regard kubernetics or docker. What I hoped was, that I could use "the app part" like "jails" on TrueNas core. However as far as I know now, that is not possible.

More in general, kubernetics is intended to maintain server applications across multiple (physical) hosts. And that is not the type of usage I had in mind More over I am very surprised to see that a lot of apps in truecharts are related to applications you normally run as part of your windows or linux desktop. So IMHO not at all applications which benefit from a "docker like" structure, the opposite seems true, far more efficient just to install them on a simple windows, unix or linux system .... (I am missing something !!??)

I noticed that there is an option in the apps menu to manage docker images ..... perhaps that has offers more opportunities than the very limited options there are in the GUI to control the apps. I did not try or investigate that. I do not intent that as well, since I prefer a GUI to manage thinks.

So .... the only option to run server-like-applications I see at this moment is to stall a couple of VM's .....
Note however that IMHO, the VM part of RC2 are by far not so mature as I would like to see as well, especially not if you are using vlans.
Could be that there are hardly or no issues if you are not using vlans and not using ipv6.

 

[truecharts]

Kubernetes runs single-node just fine, actually multi-node isn't even supported on the first release of SCALE ;-)

 

[Louis2]

I did not intend to say that the apps are not running, I just intended to say that apart from test purposes, I do not see any real use for kunbernetics in "single-node". Especially not in relation with "gui applications".

 

[truecharts]

I did not intend to say that the apps are not running, I just intended to say that apart from test purposes, I do not see any real use for kunbernetics in "single-node". Especially not in relation with "gui applications".

As you don't want any help getting your vlans setup, we'll leave it at that...
If you need help with the metalLB method, feel free to throw us a bone! :)

 

[Louis2]

I will have a look at metalLB, however be aware about what I want to accomplish.

I intent to use the TrueNAS (scale or core to be decided), to use as NAS and as host for a couple of applications in different security zones. E.g. a web server is in the redzone, where the media server I only use internally is in the "PC-zone" and the NAS itself, with my private data is in the GreenZone.

So for that reason I did separate my network in multiple zones using vlan's and those zones are arriving via a Trunk on the TrueNAS computer. On the computer "the webserver" should be implemented in "jail-1/vm-1", media server in "jail-2/vm-2" etc.

And that does not really match with
- multiple kubernetes applications running in one VM.
- And it also does not match at the network / access side with one access application connected with one IP
- not to speak about the fact that I need to support IPV6, something not really implemented as far as I can see.

So these points together and also some other mixed feelings in relation with the maturity of scale at this moment, I tend to go back to TrueNas Core as soon as version 13 becomes available. Despite the fact the SCALE is perhaps more future proof...

 

[truecharts]

In that case you would also need to wait for us to implement the k8s network policies as well, to prevent Apps from communitating with eachother and thus bypassing the vlan.
Those are on our to-do list but will take some time.

So:
- Works fine, as soon as we also added the network policies GUI, Apps can be ran just as isolated as containers. Not really an issue of SCALEitself
- Works fine with MetalLB already
- Not 100% sure how the portbinding works in relation to IPv6, it might work just fine or might not work at all.

K.S.

 

[sybreeder]

For me much easier is to not use builtin catalogs and just pull image from docker website and configure everything manually. For me it just works. I did this with pihole plex and others. Builtin plex for me was much less intuitive to configure than directly from website.
And i can easily connect docker either to specific vlan or specific nic.
I've removed truechart completely and docker start much much faster. Seems that i've had some issues with that catalog. Sometimes it doesn't want to update. Now docker start instantly.

 

[truecharts]

For me much easier is to not use builtin catalogs and just pull image from docker website and configure everything manually. For me it just works. I did this with pihole plex and others. Builtin plex for me was much less intuitive to configure than directly from website.
And i can easily connect docker either to specific vlan or specific nic.
I've removed truechart completely and docker start much much faster. Seems that i've had some issues with that catalog. Sometimes it doesn't want to update. Now docker start instantly.

This only works for simple Apps/Containers though, but yes: the docker option in SCALE is generally solid.
The solution iX designed to put it on a specific NIC/VLAN, however, is very hacky and not compatible with native kubernetes. This leads to all kinds of caveats, so that's why we generally don't advice on it to be used. It's not "bad" really, but just... "a different kind of troublesome"

We've had no reports of our catalog not updating or things stalling, it's also a bit weird of an issue you're describing, because catalog updates (which take a while) do not delay the start of the containers/apps (even it crashing does not delay container startup).

So we're pretty certain that you might have just had some bad luck there :)

 

[sybreeder]

This only works for simple Apps/Containers though, but yes: the docker option in SCALE is generally solid.
The solution iX designed to put it on a specific NIC/VLAN, however, is very hacky and not compatible with native kubernetes. This leads to all kinds of caveats, so that's why we generally don't advice on it to be used. It's not "bad" really, but just... "a different kind of troublesome"

We've had no reports of our catalog not updating or things stalling, it's also a bit weird of an issue you're describing, because catalog updates (which take a while) do not delay the start of the containers/apps (even it crashing does not delay container startup).

So we're pretty certain that you might have just had some bad luck there :)

Generally no matter what version of truenas scale i had everytime it boots docker starts after it will scan all catalogs. And indeed i has moments that I'd didn't want to update and there was no apps.
I removed all apps completely and I've stared basically from scratch. And after remove Your catalog it works smooth. Apps are on nvme.
And yes i use some basic apps like plex wifi controller, pihole and other. Whenever i wanted to use truecharts gui wasn't easier than just plain docker launch from hub
The most problematic for me was network configuration.

 

[Louis2]

My actual feeling it that it is better to use just docker. But apart from that, despite that scale is probably a good direction, for now I am back to core sincerely hoping that core gets an upgrade to FreeBSD 13 very soon.

 

[aussiejuggalo]

That's a bit of an odd statemment actually:
You want it easy, but you don't want to use the GUI that actually hides most of the complications for you?

Yes easy which sometimes means it's easier to do things through CLI instead of GUI. Locking people into Scale GUI causes a lot more problems than it solves. For example, I use Portainer for most of my stuff but sometimes I've had to bypass Portainer and go to Docker directly for stuff because Portainer was just being stupid.

I've removed truechart completely and docker start much much faster. Seems that i've had some issues with that catalog. Sometimes it doesn't want to update. Now docker start instantly.

Glad I wasn't the only one. Had to wait 20 - 30 mins for the catalogs to update every single restart or catalog update. Stopped using TrueCharts because of it. Containers would still work but it was much slower while updating.

My actual feeling it that it is better to use just docker. But apart from that, despite that scale is probably a good direction, for now I am back to core sincerely hoping that core gets an upgrade to FreeBSD 13 very soon.

A lot of people feel the same and would prefer straight Docker access without Kubernetes. There are threads around showing how to do it but it's super hacky and sometimes doesn't survive updates.

I think they should've given users the choice of straight Docker with or without Portainer or Kubernetes. Even system admins with Kubernetes experience are confused who this OS is aimed at.

 

[Ericloewe]

Yeah, well. Opinions on Docker or Kubernetes are obviously respected, but it's not a productive discussion here.

Suggestions on how to develop TrueNAS are best made in the bug tracker.