Docker App Inaccessible When Using External Interface / Different IP Than Host

[Dopamin3]

So I'm testing Scale RC 2. I had a FreeBSD VM that couldn't reach the Scale host, and I was able to fix that by adding a network bridge br0 with the same IP address as the main interface. Now the interface itself of my NIC shows no IP address. Everything seemed to be working fine.

However now I got to testing docker. I decided to spin up Heimdall with port forwarding 9000 -> 80. It works and I can access it if I don't add anything under the Networking - Add external Interfaces section. This means I can access it from http://TRUENASSCALEIP:9000. When I try to assign it a different IP address it only lets me set the Host Interface to br0, so I do that and give it a Static IP in my LAN subnet. After doing that it is not accessible from the http://ASSIGNEDIP:9000 However if I shell into the container it can ping out to the LAN/Internet and shows the correct IP address when doing ifconfig.

I think this is something with the bridge, so any help would be appreciated.

Maybe related to this thread? https://www.truenas.com/community/threads/network-setup-with-vm-and-docker-containers.98227/

 

[Dopamin3]

48 hour bump, any answers appreciated (or if you need more information on the environment)

 

[Migsi]

Bumping as I have the same issue right now.

 

[NugentS]

A docker runs on a private LAN inside the host on a 172.16.x.y address. Its NAT'd behind the host. From my reading (and I would love to be wrong) this is how it works - you cannot assign it a non NAT'd address

 

[Migsi]

A docker runs on a private LAN inside the host on a 172.16.x.y address. Its NAT'd behind the host. From my reading (and I would love to be wrong) this is how it works - you cannot assign it a non NAT'd address

As intuitive this sounds, why is there the possibility to assign "external" (local, non cluster) interfaces to a container? I'd assume there was found a way to define a dedicated interface, reachable from the local network, where all traffic to it gets forwarded to the specified container. Appearently this is not the case.

 

[NugentS]

I managed to assign a main LAN address via DHCP or static. BUT it only gets the IP - you can't pass any options like gateway, dns etc. Given that I wanted to change the gateway I stopped there

 

[Migsi]

I managed to assign a main LAN address via DHCP or static. BUT it only gets the IP - you can't pass any options like gateway, dns etc. Given that I wanted to change the gateway I stopped there

Have you been able to communicate to the container via that dedicated address? Thats the main issue the OP and me are facing. The gateway can be set when setting up static addresses. Appearently there was a bug related to that (https://jira.ixsystems.com/browse/NAS-112334) but that should be fixed.

To me the issue seems that the container is unable to response via that defined interface. I was unable to determine why, but a request to the dedicated address leads to a timeout not a refuse, so the container is reachable.

 

[NugentS]

Honestly I didn't try. The gateway was wrong - so I dropped the idea

 

[NugentS]

I think on a similar subject. My test ScaleNAS has two IP addresses on two network cards
192.168.38.36 & 192.168.138.36 (for example)
I have a bunch of dockers installed. The Containers are available on 192.168.38.36 but not on 192.168.138.36.

Is this behaviour as expected?

Wasn't sure wether to post this seperately or not

 

[truecharts]

Adding an external interface removes/overrides/replaces/adds-ontop-of the forwarding configuration.
So the port would be 80 not 9000 on the directly connected external interfaces.

With some of the patches comming with 22.02.01 and BlueFin, it should become more-viable to use MetalLB which allows exposing on specific IP adresses including forwarding setup.

 

[NugentS]

I was trying to do this without adding an external interface.
TN Scale at the moment has two interfaces on the NAS. 192.168.38.36 & 192.168.138.36. GW is 192.168.38.15 (on NIC1)
Given that containers are basically NAT'd by Scale I would have thought they would be avaiable on both TN Interfaces. But apparently not

I am not adding interfaces to the containers - but to Scale itself

 

[NugentS]

I shall wait till the next scale release to continue testing.
As storage its fine (once the hidden NFSv4 ACL was pointed out to me)
But containers are very restrictive - and I feel they shouldn't be