Douche_Baguette
Cadet
- Joined
- Jul 8, 2015
- Messages
- 8
Hi there!
I am new to TrueNAS SCALE but I have a lot of experience with old versions of FreeNAS (core). I also have a decent understanding of Docker and networking. Anyway, I have a handful of "custom" apps running on my TrueNAS SCALE setup, which are obviously running as kubernetes pods. They WORK as expected. For example homebridge, scrypted, qbittorrent - I have each one using host networking on its own static IP, and although I wish I could manually specify a static mac to use so that I could handle the static IP in my Unifi controller instead, I understand that this isn't possible right now so that's fine.
As an example, my normal LAN devices are on 10.0.1.0/24, so say my macbook is 10.0.1.180, TrueNAS SCALE is on 10.0.1.3 (static), homebridge is on 10.0.1.8 (static), scrypted on 10.0.1.9 (static), and qbittorrent on 10.0.1.11 (static).
However, there's obviously something I'm missing about the networking configuration.
My understanding is that kubernetes, by default, gives the instances a eth0 network interface which is basically an internal kubernetes network for inter-pod connectivity and DNS. 172.16.x.x subnet. I've had to manually add the host network interface and static IP, and in some instances I've had to configure the app to "use" net1 interface instead of eth0 in order to use the 10.0.1.0/24 IP that I've specified. I can access their respective web UIs on their IP/ports. All good.
But the weird thing is that, seemingly, the instances can ONLY access the internet on the kubernetes eth0 interface. The net1 interface only seems to work for LAN traffic. For example, in my qbittorrent instance:
Those are my interfaces. If I just do a "ping google.com", the pings come back, no problem. But those are going though eth0. If I "ping -I net1 google.com", the DNS resolves but all pings time out. If I "ping -I net1 10.0.1.129" (my desktop PC on the LAN), it pings no problem. As such, if I configure qbittorrent to use net1 as its ethernet interface, it can't connect.
I am trying to add some networking rules in my network management software but if the actual internet traffic isn't originating from the IP I've assigned it (10.0.1.11), that's not gonna work.
What am I missing here? I understand why some people may want that kubernetes internal interface but I'm not using it in this case and I'd really just prefer the one interface be used for all communication.
Apps -> Advanced Settings has the correct ipv4 gateway IP and interface selected.
Thanks for the help.
I am new to TrueNAS SCALE but I have a lot of experience with old versions of FreeNAS (core). I also have a decent understanding of Docker and networking. Anyway, I have a handful of "custom" apps running on my TrueNAS SCALE setup, which are obviously running as kubernetes pods. They WORK as expected. For example homebridge, scrypted, qbittorrent - I have each one using host networking on its own static IP, and although I wish I could manually specify a static mac to use so that I could handle the static IP in my Unifi controller instead, I understand that this isn't possible right now so that's fine.
As an example, my normal LAN devices are on 10.0.1.0/24, so say my macbook is 10.0.1.180, TrueNAS SCALE is on 10.0.1.3 (static), homebridge is on 10.0.1.8 (static), scrypted on 10.0.1.9 (static), and qbittorrent on 10.0.1.11 (static).
However, there's obviously something I'm missing about the networking configuration.
My understanding is that kubernetes, by default, gives the instances a eth0 network interface which is basically an internal kubernetes network for inter-pod connectivity and DNS. 172.16.x.x subnet. I've had to manually add the host network interface and static IP, and in some instances I've had to configure the app to "use" net1 interface instead of eth0 in order to use the 10.0.1.0/24 IP that I've specified. I can access their respective web UIs on their IP/ports. All good.
But the weird thing is that, seemingly, the instances can ONLY access the internet on the kubernetes eth0 interface. The net1 interface only seems to work for LAN traffic. For example, in my qbittorrent instance:
Code:
root@qbittorrent-ix-chart-7b6dcd96c8-9w5w7:/# ifconfig
eth0 Link encap:Ethernet HWaddr 1E:FC:BB:94:2D:0C
inet addr:172.16.0.86 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::1cfc:bbff:fe94:2d0c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:46542 errors:0 dropped:0 overruns:0 frame:0
TX packets:30054 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:34788172 (33.1 MiB) TX bytes:2248372 (2.1 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:66 errors:0 dropped:0 overruns:0 frame:0
TX packets:66 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6604 (6.4 KiB) TX bytes:6604 (6.4 KiB)
net1 Link encap:Ethernet HWaddr CE:5A:1A:11:E0:FE
inet addr:10.0.1.11 Bcast:10.0.1.255 Mask:255.255.255.0
inet6 addr: fd15:46de:5fa6:457a:cc5a:1aff:fe11:e0fe/64 Scope:Global
inet6 addr: fe80::cc5a:1aff:fe11:e0fe/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:23141 errors:0 dropped:0 overruns:0 frame:0
TX packets:693 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1753109 (1.6 MiB) TX bytes:322547 (314.9 KiB)Those are my interfaces. If I just do a "ping google.com", the pings come back, no problem. But those are going though eth0. If I "ping -I net1 google.com", the DNS resolves but all pings time out. If I "ping -I net1 10.0.1.129" (my desktop PC on the LAN), it pings no problem. As such, if I configure qbittorrent to use net1 as its ethernet interface, it can't connect.
I am trying to add some networking rules in my network management software but if the actual internet traffic isn't originating from the IP I've assigned it (10.0.1.11), that's not gonna work.
What am I missing here? I understand why some people may want that kubernetes internal interface but I'm not using it in this case and I'd really just prefer the one interface be used for all communication.
Apps -> Advanced Settings has the correct ipv4 gateway IP and interface selected.
Thanks for the help.
