VLANs and Jails

[taylorjonl]

I have divided my network into three zones using VLANs:

10 = DMZ
50 = UNTRUSTED
100 = TRUSTED
1000 = MANAGEMENT

Configuring FreeNAS was the easy part, I setup the VLANS and here is an ifconfig from a freshly rebooted computer before starting a jail:

Code:

ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
ix1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
igb0: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 10.0.0.50 netmask 0xff000000 broadcast 10.255.255.255
        groups: lagg
        laggport: ix0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: ix1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
vlan10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 192.168.0.50 netmask 0xffffff00 broadcast 192.168.0.255
        vlan: 10 vlanpcp: 0 parent interface: lagg0
        groups: vlan
vlan50: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 192.168.1.50 netmask 0xffffff00 broadcast 192.168.1.255
        vlan: 50 vlanpcp: 0 parent interface: lagg0
        groups: vlan
vlan100: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 172.20.0.50 netmask 0xffff0000 broadcast 172.20.255.255
        vlan: 100 vlanpcp: 0 parent interface: lagg0
        groups: vlan
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        groups: bridge
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
        member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        groups: epair

I can ping these addresses on the respective networks so I know my network is configured correctly. My problem comes when I try to setup the jails...

The plan is to have many jails, but initially I am trying to setup two jails. One will be my Plex jail I want connected to the UNTRUSTED and TRUSTED VLANs, the other is a torrent jail that I want to be connected to the DMZ VLAN. I am trying to do this through the GUI.

My initial try is for the torrent jail, there I left everything default except these two properties:

interfaces=vnet0:bridge10 vnet_default_interface=vlan10

When I start the jail, the ifconfig output changes to this:

Code:

ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
ix1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
igb0: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 10.0.0.50 netmask 0xff000000 broadcast 10.255.255.255
        groups: lagg
        laggport: ix0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: ix1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
vlan10: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 192.168.0.50 netmask 0xffffff00 broadcast 192.168.0.255
        vlan: 10 vlanpcp: 0 parent interface: lagg0
        groups: vlan
vlan50: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 192.168.1.50 netmask 0xffffff00 broadcast 192.168.1.255
        vlan: 50 vlanpcp: 0 parent interface: lagg0
        groups: vlan
vlan100: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 172.20.0.50 netmask 0xffff0000 broadcast 172.20.255.255
        vlan: 100 vlanpcp: 0 parent interface: lagg0
        groups: vlan
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        groups: bridge
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 11 priority 128 path cost 2000
        member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 1000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        groups: epair
bridge10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        groups: bridge
        member: vnet0:2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 13 priority 128 path cost 2000
        member: vlan10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 7 priority 128 path cost 2000000
vnet0:2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:ff:60:56:71:77
        groups: epair

From this output I can see that a new bridge was created that contains my desired VLAN, I can also see that my DHCP server gets the request and offers a lease, but the lease is never accepted. So to me this says that traffic is making it from jail to my router, but the jail isn't receiving traffic? So it can't complete the DHCP offer request?

What am I missing? Is the issue that the traffic into the newly created bridge is tagged so the jail doesn't recognize it?

 

[taylorjonl]

It turns out the LACP bonding was causing a problem. The UDP packets were making it to the router but when the router tried to send a DHCP offer, the offer wasn't making it to the bond. I removed the bond and it all works fine now. One day I will figure out why the bond is behaving this way but at the moment I have very little interest in figuring it out since I don't need the bandwidth from a bond just yet.

 

[alexr]

I have the same symptoms in https://forums.freenas.org/index.php?threads/iocage-jails-in-vlans.72287/ but I see the offer packets hitting lagg0 but not making it through the bridge to the jail. If I do a "dhclient vlan20" from the host side, it sees the offer and puts an IP on the vlan20 interface just fine.