Different IP Address for Each k3s Pod

[wootonius]

I apologize if this is somewhere in the docs or if it is a dumb question but I'm totally stumped. The gist of my question is this: How do I set up my k3s pods so that each one has its own IP address?

I've used LXC/LXD containers before and if I remember correctly that involved network bridge(s) for each container to get different IP addresses. Is it the same for k3s? I'm happy to read any documentation that walks through how to do this but I haven't come across anything that covers my use case yet. Any hints or direction would be much appreciated.

Thank you in advance and thank you to everyone working on Truenas! I'm newly moved over from Proxmox and I really like Truenas Scale.

 

[morganL]

Creste network interfaces with the IP addresses you want to use and hten assign them to each pod...

The developer notes are here: https://www.truenas.com/docs/hub/scale/dev-notes/

Better documentation coming in April.

 

[ornias]

assign them to each pod...

Uhmm... Morgan... you do know you guys never provided any documentation for that right? :P

 

[morganL]

Better documentation (compared with source code, Developer and Release Notes).. coming in April. (have to improve TrueNAS 12.0 docs 1st)

 

[wootonius]

Cool, I can wait. Thank you for the quick response!

 

[bubski]

@wootonius did you manage to figure this out? I tried following what @morganL suggested, but no matter what I do, and how many times I read the dev notes, I fail. I don't know how to create a network interface right…

 

[wootonius]

@wootonius did you manage to figure this out? I tried following what @morganL suggested, but no matter what I do, and how many times I read the dev notes, I fail. I don't know how to create a network interface right…

I got it sorted but not in the way I was expecting. I used the Quick Start Guides from the Truecharts site and set up all my apps using domains with a Traefik ingress (apologies if my terminology wasn't correct). Here's a link to the instructions I used:

Support Policy - TrueCharts

Project Documentation for TrueCharts

truecharts.org

I'm not sure if this is applicable to your setup but it made mine a lot easier than I was expecting.

 

[truecharts]

RC2 will also allow users to disable the integrated kubernetes loadbalancer and use MetalLB (from our App catalog for example) instead.
Which allows giving each App their Own IP address(es) :)

 

[bubski]

Thanks @wootonius! That looks like a good direction for me to research. I fiddled with it a bit in a VM, but I don't think I know how to make it work with non-truecharts apps. (Perhaps I'm not the only one ). Which leads me to believe that what RC2 brings, as @truecharts said, may somewhat clear things up for me. I'm very inexperienced with all this, and learning as I go.

 

[truecharts]

Thanks @wootonius! That looks like a good direction for me to research. I fiddled with it a bit in a VM, but I don't think I know how to make it work with non-truecharts apps. (Perhaps I'm not the only one ). Which leads me to believe that what RC2 brings, as @truecharts said, may somewhat clear things up for me. I'm very inexperienced with all this, and learning as I go.

The correct way within the TrueCharts ecosystem is still, as described by @wootonius, using ingress and not exposing apps directly on an IP.
While we technically support it, we don't consider it something we advice.

The seperate IP per service (not pod!) option is there mostly for advanced users that know what they are doing and the possible caveats of doing so.

That being said: What we said before only works on TrueCharts Apps, not on the docker button or ix-official apps, those do not support servicetype "LoadBalancer" at all.

TLDR:
If you want to use MetalLB because you don't know ingress yet, you better learn Ingress instead of MetalLB ;-)

 

[bubski]

The correct way within the TrueCharts ecosystem is still, as described by @wootonius, using ingress and not exposing apps directly on an IP.
While we technically support it, we don't consider it something we advice.

The seperate IP per service (not pod!) option is there mostly for advanced users that know what they are doing and the possible caveats of doing so.

That being said: What we said before only works on TrueCharts Apps, not on the docker button or ix-official apps, those do not support servicetype "LoadBalancer" at all.

TLDR:
If you want to use MetalLB because you don't know ingress yet, you better learn Ingress instead of MetalLB ;-)

Looks like I should do some learning, and familiarize myself with ingress then. (Coming from TrueNAS core and jails, this is all very different).

However, please help me understand what you said @truecharts:
"That being said: What we said before only works on TrueCharts Apps, not on the docker button or ix-official apps"

Would I then still be able to use ingress and all that for my non-TrueCharts applications? And if not, is there a way to create custom applications in TrueCharts ecosystem (I couldn't find anything hinting that on the website).

 

[truecharts]

Looks like I should do some learning, and familiarize myself with ingress then. (Coming from TrueNAS core and jails, this is all very different).

However, please help me understand what you said @truecharts:
"That being said: What we said before only works on TrueCharts Apps, not on the docker button or ix-official apps"

Would I then still be able to use ingress and all that for my non-TrueCharts applications? And if not, is there a way to create custom applications in TrueCharts ecosystem (I couldn't find anything hinting that on the website).

If you want more in depth information on TrueCharts we have our own support channels.

Byond that: Nothing we said here applies to TrueNAS official Apps in any way shape or form. Nor do they support ingress at all.