Plugin Jails on different networks

[thorgrim]

Hi all !

My Freenas box is connected on two different networks (routers) with two separate nics, One is on 192.168.1.x and the other one on 192.168.3.x.

Up until now, the jails I had set up for plugins where only to be accessed from my local network ( sb, cp and the like ) so I used the IP range from behind the second router to keep it more "private". But now I would like to install the Owncloud plugin to get access from the web to my documents, pictures and other things. And for this one, I would like to plug it directly to the first router which is the one connected to the internet so I can simply define one port I can redirect from WAN to Owncloud.

Any idea how to do that ? Is it possible to define which nic the jail should use to route its network traffic ? I tried simply setting up a valid IP from the first router in the jail configuration but this didn't do the trick (would have been too easy I guess :) )

Thanks !

 

[dlavigne]

Were you able to figure this out?

 

[thorgrim]

Unfortunately no. I didn't have much time to look at it either but I did some more digging on it and I think the network configuration on my Freenas box right now may be a bit strange...

So, I did a schema of my setup which is attached. As shown, the Freenas box has two nics (re0 and em0) connected to two different routers. Here is the full ifconfig while connected to the Freenas box directly :

Code:

em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=42098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
        ether 68:05:ca:19:38:31
        inet 192.168.1.216 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether 14:da:e9:68:61:74
        inet 192.168.3.15 netmask 0xffffff00 broadcast 192.168.3.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
        nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0xb
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:b8:0c:cb:8e:00
        inet 192.168.3.15 netmask 0xffffff00 broadcast 192.168.3.255
        nd6 options=1<PERFORMNUD>
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair4a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 17 priority 128 path cost 2000
        member: epair3a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 16 priority 128 path cost 2000
        member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 15 priority 128 path cost 2000
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 14 priority 128 path cost 2000
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 13 priority 128 path cost 2000
        member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:78:1f:00:0d:0a
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:91:5f:00:0e:0a
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair2a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:d0:f2:00:0f:0a
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair3a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:54:b0:00:10:0a
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair4a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:d7:73:00:11:0a
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active

What I find a bit strange here is that the bridge interface seems to be bound to the em0 nic which is connected to the "internal" router (R2) but it has the IP address of re0 which is connected to the "external" router (R1). Right there, I don't understand :)

Now here is the ifconfig output when connected to my owncloud jail :

Code:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair4b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:63:24:00:12:0b
        inet 192.168.1.244 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active

Here it has an IP in the internal network which is what was configured by default in my jails configuration in the Freenas web admin. In that state, I can access it with no issue except it is connected to the second router so not "easily" accessible from the outside.

So my guess here is that I should define a second bridge on my box using the re0 interface and bind the jail to the bridge1. I don't think creating a bridge like that is any difficult but attaching the jail with it I have no idea how to do that. I will have to look for some doc on FreeBSD and Freenas but if anyone has some pertinent info I'll take it !

In the same time, if somebody can explain why the bridge0 interface seems to be using the internal nic with the external IP... That is really doesn't make any sense to me.

Will try to find some time and keep this post updated if I make any progress.

 

[dlavigne]

I wonder if what you are seeing is related to https://bugs.freenas.org/issues/4719#note-10...

 

[stranger]

I'm having the same issue.
The epair can be easily added to the jail
ifconfig epair9b vnet 7

Where 7 would be the number of the jail from jls and 9 is the number of the epair interface that you've created. Change the values for your own setup.
Remember to make sure that all your interfaces in the bridge and the bridge itself are up.

I get that far but then I can ping the IP bound to the physical interface but nothing outside the box.