Rancher internet access - not

[mmeasel]

Running 11.2-Release and followed the instructions ( several times ) and no matter what, I can't get rancher to see past the bridge. This causes it to fail multiple things like trying to start the interface. Does anyone have any special sauce or advice ?

I've even tried assigning an IP to the bridge and rancher can ping that, but not the interface that it's bridged to (vmx1 - but I've tried other combinations ).
TCPDump running on my FN host can see packets flowing on the tap.

I found an old post where it seems to indicate that the interface IP in rancher is the same as that of the external interface. Doesn't work.
I saw where someone said "don't assign an IP to the external interface". Doesn't work.
Tried not using a 10.x address and tried using a different subnet and tried using the same subnet. Nada.
Tried e1000, vmxnet2 and vmxnet3 adapters. Didn't matter.
Tried various tunables like setting gateway. Didn't help.

Does anyone have this working and can you please point out any steps that you did that are different from the documentation ?

TIA

oh, I am able to ssh into rancher from FN once the networking is configured ( vmx1 - 192.168.0.2/29 or whatever ) which made me think firewall but can't find one running.

 

[sretalla]

what's the IP of your host and the VM guest?... remember that the VM is on the bridge to the ethernet adapter, but it needs to do its own addressing and routing from there.

 

[mmeasel]

last attempt the host is 10.0.0.1 and the guest is 192.168.0.2
the host has two interfaces vmx0 and vmx1
vmx1 is attached to 192.168.0.0/29 network with a gateway at 192.168.0.6
the gateway is pingable from the host, but not from the guest

 

[mmeasel]

asking again if anyone has an example that they'd share, please

 

[sretalla]

so perhaps we need to see some outputs from ifconfig on the host and from the vm...

If I get you correctly, you have a host with 2 virtual NICs (which one is bridged to the VM TAP?... seems it should be vmx1)

Your guest is on the same subnet (assuming the bridge is good).

Why are you subnetting to /29? seems a really odd thing to do in the private address space when /24 is freely available and much simpler to keep track of.

It sounds to me like the gateway setting may be the problem on either the guest or host...

My first suggestion would be to subnet it all to /24 and go from the basics up.

 

[mmeasel]

Sure.

Code:

vmx0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=60039b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:50:56:32:f2:d8
    hwaddr 00:50:56:32:f2:d8
    inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect
    status: active
vmx1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=200099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,RXCSUM_IPV6>
    ether 00:50:56:15:5f:cf
    hwaddr 00:50:56:15:5f:cf
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect
    status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    ether 00:bd:7f:36:ff:00
    hwaddr 00:bd:7f:36:ff:00
    nd6 options=1<PERFORMNUD>
    media: Ethernet autoselect
    status: active
    groups: tap
    Opened by PID 3600
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:a5:e3:fe:80:00
    nd6 options=1<PERFORMNUD>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vmx1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 2 priority 128 path cost 2000
    member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 4 priority 128 path cost 2000000

And on rancher:

Code:

[rancher@rancher ~]$ ifconfig
docker-sys Link encap:Ethernet  HWaddr 02:42:D1:A3:B7:3B 
          inet addr:172.18.42.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:d1ff:fea3:b73b/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:426 (426.0 B)

docker0   Link encap:Ethernet  HWaddr 02:42:D8:18:13:F8 
          inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth0      Link encap:Ethernet  HWaddr 00:A0:98:13:89:74 
          inet addr:192.168.0.2  Bcast:192.168.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:123 errors:0 dropped:0 overruns:0 frame:0
          TX packets:212 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:16001 (15.6 KiB)  TX bytes:18450 (18.0 KiB)

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:93 errors:0 dropped:0 overruns:0 frame:0
          TX packets:93 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:8304 (8.1 KiB)  TX bytes:8304 (8.1 KiB)

So I changed it to /24 just for giggles. Still doesn't work.

Routing:

Code:

root@freenas[~]# netstat -4 -rn

Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire

default            10.0.0.254         UGS        vmx0
10.0.0.0/24        link#1             U          vmx0
10.0.0.1           link#1             UHS         lo0
127.0.0.1          link#3             UH          lo0
192.168.0.0/24     link#2             U          vmx1
192.168.0.1        link#2             UHS         lo0

and on rancher

Code:

[rancher@rancher ~]$ route -n

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

0.0.0.0         192.168.0.254   0.0.0.0         UG    0      0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker-sys
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

[rancher@rancher ~]$ 

 

[sretalla]

So what device is 192.168.0.254? (I guess it's your internet router)

It's interesting to note that although it's in the routing table, vmx1 doesn't think it has an IP address in ifconfig.