Routing app traffic via external interface

[TempleHasFallen]

After going through tons of troubleshooting steps, this is what I have gathered that apps with external interfaces (ex. IP on LAN), by default route the traffic through the internal kubernetes NIC.

I've tried setting routes via the GUI, but it does not affect the default route (unless you know exactly the IPs/subnets you're going to be connecting to).

Assuming you don't want all apps to use your NAS IP, there are various ways this can be achieved, I'm looking for better solutions at the moment.

Script Inside Container, post init

Basically an entrypoint script which is executed and modifies the default route.
This requires the "NET_ADMIN" capability which needs to be added either via GUI or via k3s kubectl.
This is described in this thread.
I wasn't able to replicate it due to my pod erroring out with

Code:

ip: rtnetlink answers: operation not permitted

regardless of if my app was configured with NET_ADMIN, SYS_ADMIN, NET_RAW, privileged mode (or any combination of those)




Script on Host, on demand

Best I've come up with is a script which can be run on demand and ideally post system init (with delay) to modify the default route of apps to match their external interface.
This also works for containers that don't have

Code:

ip

installed.

Code:

#!/bin/sh
for container in app1 app2 app3 app4
do
        conid=''
        conpid=''
        conid=$(docker ps | grep k8s_$container | awk 'NR==1{print $1}')
        conpid=$(docker inspect --format '{{.State.Pid}}' "$conid")
        if [ ! -z "$conpid" ]
        then
                nsenter -n -t "$conpid" ip route del default
                nsenter -n -t "$conpid" ip route add default via 192.168.1.1 dev net1
                echo "Added route for $container"
        fi
done

If anyone has any more efficient/automated/better ideas, please share your ideas.

 

[KristijanL]

i have the same issue, thanks for the script!
do you know, is this being worked on, or should we open a request somewhere?