Trouble configuring a VLAN for a VM

[ajkelsey]

I have a linux VM running. I have configured a VLAN interface and a Bridge interface. The VLAN has the physical NIC set, the bridge has the VLAN set as member, and the VM NIC has the bridge set. However, my VM is unable to get an ip via DHCP. What am I missing here?

 

[sretalla]

It should go like this:

Physical NIC(s) -> LAGG (if you have it) -> VLAN -> Bridge (IP address of host would go here if this is your primary connection).

Seems you switched the bridge and VLAN order.

 

[ajkelsey]

Still no luck. I don't use LAGG. The VLAN can only have the physical NIC as its interface. I'm not sure what you mean by the ip address being at the bridge. Assign an alias IP?

 

[sretalla]

I'm not sure what you mean by the ip address being at the bridge. Assign an alias IP?

When you have a bridge, you need to remove the IP address from the NIC and instead add it to the bridge itself (if an IP on the host is needed for that connection... usually the case if that's your only NIC)

 

[ajkelsey]

When you have a bridge, you need to remove the IP address from the NIC and instead add it to the bridge itself (if an IP on the host is needed for that connection... usually the case if that's your only NIC)

If I do that, will truenas then be on the vlan?

 

[sretalla]

That brings up a point that reminds me of long discussions that other folks have had in many other threads (usually including @Patrick M. Hausen ) about VLANs and having both tagged and untagged packets on the same interface... I don't know if you can really do that.

Maybe you would need to create a second bridge (perhaps a first one if you're looking at it from a numbering perspective as you probably want the default br0 to be the one with the host on it) to handle the non-VLAN traffic and the one for the VM to handle the VLAN traffic.

I suspect there's something about that which doesn't really work, but search for those threads or maybe Patrick will help out by commenting here.

One such post/thread:

VLANs Bridges and LAG Interface best practice questions

I have a working Truenas 12.0U3 installation. The installation has as follows (note I used this reference documentation https://www.truenas.com/community/threads/how-to-setup-vlans-within-freenas-11-3.81633/ ) Switch => LAG (gig24 + gig 25) => Trunk vlan 1-1000 => TRUENAS Server VLAN 1 is the...

www.truenas.com

 

[ajkelsey]

Ok. I think I'm getting there. I was able to get TrueNAS to connect using a tagged vlan so I dont have that untagged/tagged situation. I did find it tricky to move the ip address from the NIC to the bridge, but got it done.

I now have a second vlan interface and bridge setup for my VM. The ip that I assign to the bridge, do I assign that on my VM as well? Or is this just an address for the bridge and the VM can use DHCP?

 

[Patrick M. Hausen]

If the bridge is only a layer 2 connection for the VM it does not need an IP address at all. Only the VM needs one.

The bridge needs an IP address if the TrueNAS host needs to communicate in that same VLAN.

 

[ajkelsey]

I'm still having trouble. I created vlan120 and br120 for my VMs. I made sure the VM NIC was configured to use br120. No connections. The VM will work on br0 which is connected to vlan2.

I tried configuring an ip on br120, and rebooting truenas. No luck there either.

 

[Patrick M. Hausen]

What - apart from your VM is connected to that VLAN 120? I mean enp3s0 is connected to a switch port carrying tagged VLANs, right? And then? What other systems share that VLAN 120 - port based on the switch or tagged with a trunk port?

A VLAN in itself does not do much that is useful. It's an isolated virtual LAN - like an isolated switch. There has to be a router around somewhere that serves as the default gateway for the VM. TrueNAS does not do that. Your firewall or a layer 3 switch must do this.

 

[ajkelsey]

It was my trunk port. I didn't configure it to carry the new vlan. I did check my router and firewall rules. Doubled checked the port on the switch, but forgot all about that trunk. Thank you for jogging my memory. Everything works now.

 

[Patrick M. Hausen]

@sretalla can we consider it confirmed that just like FreeBSD Linux also must have all layer 3 addresses on the bridge interfaces? I guess so, but it does surprise me a bit. I always considered the restriction in FreeBSD to be a peculiarity of the FreeBSD network implementation.

 

[sretalla]

@sretalla can we consider it confirmed that just like FreeBSD Linux also must have all layer 3 addresses on the bridge interfaces?

I may not be the guy to definitively answer it, but I'm prepared to agree what we just saw seems to support that theory.

Maybe with the trunking done earlier in the process, we wouldn't have got to this point where it seemed to be required.

 

[ajkelsey]

Well, my trunk port was misconfigured the whole time. This all may have worked with a slightly different configuration. I'll be willing to check if you would like.

 

[sretalla]

For the sake of the exercise, you can certainly walk it back a few steps and report which other versions work...

If you have the time and inclination...

I don't think there's anything wrong with the configuration we landed on at the end, so it could also be considered valid for both Linux and FreeBSD, so maybe a good thing.

 

[ajkelsey]

Sure. So you want to know if configuring IPs on the physical NIC and not on the bridge works?

 

[sretalla]

Sure. So you want to know if configuring IPs on the physical NIC and not on the bridge works?

I'm guessing it will, but go ahead and test.

 

[ajkelsey]

I removed the IPs from br0.
I added them to enp3s0.
I lost connection with the UI while testing, and it reverted back.