VLAN in Jails and VMS on TrueNAS-12.0-RELEASE

[KpuCko]

Hello, I'm trying to create some Jails/Plugins and VMs on my FreeNAS server which has to part of another network segment (vlan)

Brief explanation of the interfaces of the server:

I have igb0 and igb1, they are part of bond interface, which is using LACP aggregation and it is set to trunk (I'm coming from Cisco's world), with no vlan restrictions (open trunk)

I also have created two vlan interfaces, the first one is vlan30, which is used to host the FreeNAS management IP (as tagged vlan)
and another one, named vlan310, which should be the vlan which provide network connectivity for the guests (for example).

Regarding this link: https://www.truenas.com/community/threads/how-to-setup-vlans-within-freenas-11-3.81633/ and based on my previous experience I know I have to create bridge interface and combine it with the vlan interface, that is exactly what I've did. There is a bridge named bridge310, and vlan310 is part of this bridge.

Just to ensure all about network subletting is fine (only for the test) I've set IP address on this bridge and I was able to ping it from my lan network, later on I removed the IP address, because I don't needed it, and I know (regarding the link above) it shouldn't have IP address set.

So far so good, except that fact when I try to create new Jail/Plugin I end up with broken DNS resolution, some errors like pkgs.freebsd.org cannot be resolved.

I can confirm that, when I start the Jail appropriate tun interface is created and the bridge became its parent. So from my point of view, all should be fine ;> but it is not.

Let me show you few pictures of the current config, and we can discuss what to test further.
Any advises are welcomed.

So let's recap - If I set IP for this vlan network on the bridge interface (on the host level) I'm able to ping it, which means the 802.1q is working fine on the host OS, but when I try to pass this communication to VM or Jail I can't. I'm attaching a screenshot for the VM config, because I assume VM network config will be much easier to understand.

 

[dak180]

I cannot speak to vms but for jails I have found that it is generally better to let the bridges get auto created by the system.

Lets consider the case of vlan30 which uses the 192.168.30.0/24 block with 192.168.30.1 as the router/dns server in this example with a jail named pvr which uses dhcp to get an address:

Code:

iocage stop "pvr"
iocage set vnet="1" "pvr"
iocage set bpf="1" "pvr"
iocage set dhcp="1" "pvr"
iocage set interfaces="vnet0:bridge30" "pvr"
iocage set resolver="search local;nameserver 192.168.30.1" "pvr"
iocage set vnet_default_interface="vlan30" "pvr"
iocage start "pvr"

When this jail is started if bridge30 does not already exist it will build it and if nothing else is using it, it should be broken down when it exits. Of course some of this will need to be changed to reflect your network settings but this should give you a useful template. Also these same properties can be set in the GUI as well but I generally find this sort of ting easier to script as part of the jail creation process.

 

[KpuCko]

Thanks for the answer @dak180 but I'm wondering how I can use all of these commands, because when I create the Jail/Plugin (I play with plugins most of the time) the plugin simply get destroyed when there is problem with the network configuration.

So how to keep the configuration of the plugin in that case?

And second question, do I have to use command line? Do all of these configurations can be set via GUI?

 

[KpuCko]

By the way I've managed to get it working, by setting ip_forwarding 1, which is strange from my point of view because the bridge is used to combine vlan interface and the guest interfaces from one hand, and on another hand, vlan interface has lagg as its parent interface, so they also should have L2 connectivity...

 

[Patrick M. Hausen]

NAT implies routing, not bridging. Perfectly valid.

 

[dak180]

Thanks for the answer @dak180 but I'm wondering how I can use all of these commands, because when I create the Jail/Plugin (I play with plugins most of the time) the plugin simply get destroyed when there is problem with the network configuration.

So how to keep the configuration of the plugin in that case?

And second question, do I have to use command line? Do all of these configurations can be set via GUI?

Yes, all of these commands can be set from the gui; I just have not had the time to take and markup screenshots. To be able set these values in the gui during setup for a plugin will require the advanced settings which are hidden by default.

 

[KpuCko]

NAT implies routing, not bridging. Perfectly valid.

But I'm not using NAT

Code:

sofx-nas01# iocage get all sofxtst02 | grep nat
nat:0
nat_backend:ipfw
nat_forwards:none
nat_interface:none
nat_prefix:172.16
sofx-nas01#

 

[Patrick M. Hausen]

Ah - get it. I confused your post with a different one. Is the DNS server for the jail in a different VLAN than the jail itself? In this case again you need some router between the VLANs.

 

[KpuCko]

No, the router act as a DNS server, and it listen on all the interfaces.
Anyway, its like a black box to me ;>

 

[KpuCko]

Any ides how to resolved the DHCP problem? All guests are unable to get their IP address via DHCP, and I really don't have a clue what to debug

 

[KpuCko]

Ok, issue solved.

The confusion came from forwarding mechanism. I left with the feeling that when I set net.inet.ip.forwarding to 1 on System -> Tunable as RC config, this will activate forwarding on boot. Unfortunately this is not true. So I have to set gateway_enable = Yes, and type RC to get forwarding enabled on boot.

With forwarding enabled, my guests VMs/Jails are able to get IP address by the DHCP service.
I'm sharing this with you and hope it will help to somebody.

 

[Patrick M. Hausen]

Ah, so you don't run a DHCP server on each subnet?

 

[KpuCko]

Actually, that is exactly what I'm doing.
My DHCP server has a leg on every VLAN segment.