Ingress or reverse proxy?

[Patrick M. Hausen]

Hi, @ornias,

just a push in the right direction, please.

If I want to run multiple TrueCharts applications on my host, all on port 443 with SNI, should I look into the "ingress" section of the settings or this part of the manual?

Reverse Proxy - TrueCharts

Project Documentation for TrueCharts

truecharts.org

Curiously the manual does not mention "ingress". At least not in the navigation.

Or are these the same and it's only a matter of terminology and organisation of the docs?

If have a wildcard certificate for my internal domain, and I already found the part about using the TN SCALE certificate store. All understood. Now how to get foo.ettlingen.hausen.com and bar.ettlingen.hausen.com to different node ports?

Thanks!
Patrick

 

[ornias]

Please be aware that TrueCharts support is not using this forum.
You're basically lucky I even checked today.

The manual is basically ages outdated by now.

 

[morganL]

Please be aware that TrueCharts support is not using this forum.
You're basically lucky I even checked today.

The manual is basically ages outdated by now.

Can you point people to where the support is located?

 

[stavros-k]

You can find TrueCharts support here -> https://truecharts.org/discord

 

[ornias]

You can find TrueCharts support here -> https://truecharts.org/discord

We've have multiple support channels:

Get in Contact - TrueCharts

Project Documentation for TrueCharts

truecharts.org

;-)

 

[Ixian]

Patrick you can hit me up here or on the Truecharts discord if you still need help, I just finished setting up the same thing (using Traefik as the Ingress controller for my apps). I also moved K8s and by extension Traefik to a different ip on my test SCALE server so Traefik can listen on standard ports 80 and 443, that way all I had to do was forward my local dns app names like https://sabnzbd.mydomain.tld to the ip k3s/Traefik is listening on and Traefik does it's thing forwarding from there.

 

[Patrick M. Hausen]

@Ixian Thanks. Only question - how do I get from here: https://www.truenas.com/docs/scale/apps/tconscale/ and having TrueCommand available on e.g. ports 9004 and 9005 on my SCALE system's main address to having it available on an alias IP address and ports 80 and 443?

Currently I solve this by running each Docker application in its own VM and simply bind the Docker ports to the host ports since there is nothing with a web UI on these VMs but the single Docker based application. But I intend to move these from VMs on CORE to apps on SCALE if possible.

I really want one IP address per app and standard ports. I don't see :9004 as acceptable for a production system.

What's the "SCALE way" to get there? Of course I could run a reverse proxy somewhere, e.g. in yet another VM, but there must be something official, right?

 

[Heracles]

Hey @Patrick M. Hausen,

how do I get from here: https://www.truenas.com/docs/scale/apps/tconscale/ and having TrueCommand available on e.g. ports 9004 and 9005 on my SCALE system's main address to having it available on an alias IP address and ports 80 and 443?

There is only way to have multiple services listening on port 443 at the same IP : SNI. For that, you need a proxy that can do at least SNI routing. Here, I do it with HAProxy which is part of my pfSense firewall infrastructure. Without the SNI routing, there is no way for a technology to distinguish between services hosted behind it. Once SNI routing is possible, you can often do SSL acceleration (decrypt completely and forward clear text behind the proxy).

Have fun designing and building your own solution,

 

[Patrick M. Hausen]

There is only way to have multiple services listening on port 443 at the same IP : SNI.

I know I want each service on standard ports on its own IP address. Preferrably without a proxy by just mapping the ports like I do with docker-compose, currently. I can then add a proxy definition for all the applications I really want to expose to the Internet. And I am already doing that. But most just stay internal.

Examples:

Onlyoffice

Code:

version: '3.3'
services:
  documentserver:
    container_name: onlyoffice
    restart: always
    image: 'onlyoffice/documentserver'
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - '/app/onlyoffice/DocumentServer/logs:/var/log/onlyoffice'
[...]

TrueCommand

Code:

version: '3.3'
services:
  truecommand:
    container_name: truecommand
    restart: always
    image: 'ixsystems/truecommand:2.0'
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - '/app/ixsystems/truecommand:/data'

Each in its own VM, so own IP address.

If I move these to SCALE I lose the VMs around the applications but I also get rid of a lot of CPU and memory overhead.
So how can I run the SCALE UI, TrueCommand, OnlyOffice, ... each on their own alias address in my network?

 

[Ixian]

I know I want each service on standard ports on its own IP address. Preferrably without a proxy by just mapping the ports like I do with docker-compose, currently. I can then add a proxy definition for all the applications I really want to expose to the Internet. And I am already doing that. But most just stay internal.

You won't get that with SCALE apps due both to how Kubernetes works and more specifically how Ix systems is deploying it through the GUI. There's no Macvlan-like equivalent (even the standalone docker install option they added recently just allows you to use a docker container inside a k3s pod - it doesn't leverage the docker network stack or other parts of the docker ecosystem). I know this is going to take folks coming from both BSD-style jails and Docker/Docker-compose environments a while to wrap their heads around - how SCALE apps handle networking, etc. is different. It is not a simple docker-compose like setup.

SNI routing, as Heracles points out, is your best bet. There are a few approaches - like Heracles when I move this to production I'm going to use my existing HAProxy service running on my OPNSense box; I'm already using it for services like Nextcloud and creating internal-only SNI routing is trivial.

The Truecharts folks are building what is essentially a curated app list with some extras, notably integrated Traefik support (more specifically their apps can use Traefik as a Kubernetes Ingress Controller) - with that setup, you can install the Traefik app in SCALE and pretty easily configure the other apps in their catalog to use that for Ingress (reverse proxy in this case).

If you go that route you could add an ip to SCALE (on your same interface or a different one) and under Apps>Advanced Settings configure Kubernetes to listen on it, that way the Traefik app can listen on 80/443 so you don't need to rememeber odd ports for it. If you configure your apps to use it for Ingress all you need to do then is configure your DNS to point to the Traefik (technically, K8s) IP via https://your.domain.tld and Traefik will handle the rest.

Edit: Adding that the Truecharts folks specifically configured their Traefik deployment to use a special method involving the Kubernetes load balancer service that allows Traefik to to run on lower ports, you'd have to ask them how they configured it, it's not simple to replicate.

 

[Patrick M. Hausen]

If you go that route you could add an ip to SCALE (on your same interface or a different one) and under Apps>Advanced Settings configure Kubernetes to listen on it, that way the Traefik app can listen on 80/443 so you don't need to rememeber odd ports for it. If you configure your apps to use it for Ingress all you need to do then is configure your DNS to point to the Traefik (technically, K8s) IP via https://your.domain.tld and Traefik will handle the rest.

That was the missing bit I needed. Thanks. One IP address and SNI is perfectly OK as long as I can at least get one different from the SCALE UI.
And the fact that I need to explicitly install the Traefik app wasn't that obvious, either.

Edit: what about egress? My current TrueCommand VM has got two network interfaces. One connected via VPN to my office LAN where my backup TrueNAS is standing. And one in my management VLAN at home for the other TN systems. I guess it is just NATed for outbound connections to the SCALE address? So if I configure both VLANs for the SCALE host and appropriate routes, the TC container will be able to reach everything SCALE can reach?

Thanks again.
Patrick

 

[Ixian]

The Traefik app, in this context, is specifically the one the Truecharts folks provide in their catalog, and is meant to work with the other apps in their catalog. It's not a general-purpose Traefik app. They added it so "home" users would have an easier time doing what you're trying to do. I expect this to cause a tremendous amount of confusion when release hits.

I haven't messed much with my VLANs yet and this setup. I have an IoT vlan and another for my security cameras (and yet another for guest WiFi but that isn't allowed any LAN access) but I haven't yet tested how, for example, the Home Assistant app will work since it needs access to them. Since I am doing layer-3 routing for my VLANs through my Brocade switch it technically shouldn't be an issue but I am not familiar enough with K8s networking to know for sure. Going to find out :)

 

[ornias]

Actually: It's the official Traefik Helm chart with only a few modifications to get it to play nicely with the GUI.
The "the general-purpose" bit, is also in the works. It's called 'external services' because they donnot reside in k8s. But will take some more time, because some higher priority issues came up.

To be clear: We offer K8S native Apps with as many k8s features exposed as reasonable. This means we donnot expect everyone to be able to setup everything, a lot of features are advanced features.