TrueNAS 12.0-STABLE - Set up VLANs

[Newfoundland.Republic]

Folks,

From my post Are JAIL VLANs broken (since upgrading from 11.3 to 12.0) I cannot (despite incantations to the computer gods, making up new swear words, etc., etc.) get VLANs to work.

I started from scratch deleting my Emby jails, deleting the bridges to the VLANs and deleting the VLANs themselves. I then followed the process that I followed for setting the VLANs, bridges and jails that I followed in 11.2 (not sure the update...) and nothing works.

Does anyone have any step-by-step instructions for setting up VLANs, bridges and jails in TrueNAS 12.0? I am using an unused interface - igb2 - (e.g., no IP assigned) that is plugged into my UniFi switch on a port set to ALL (VLANs). Any suggestions are welcome at this point...

Thanks!

 

[Patrick M. Hausen]

Jail VLANs are definitely not broken. Look at my interfaces screenshot first.

I combined two of my physical interfaces into the LAGG you are seeing, then created 3 VLANs on top of the LAGG, then created a bridge on top of one of the VLANs. I.e. bridge0, vlan100 is the only member. Could have named it bridge100 if I intended to create more than one, e.g. one for each VLAN. All the same.
If you don't want to create a LAGG, you can create the VLAN on top of the physical. Then Bridge --> VLAN as member.
IP address goes on the bridge! Documented in the FreeBSD handbook.

Then in your jail config make sure to use vnet0 as the jail's interface and assign bridge0 down in the "Network Properties". See second screenshot.

Feel free to come back with more questions but best provide screenshots, output of ifconfig -a on the host, output of iocage get all <jailname> ... upfront.

HTH,
Patrick

 

[Newfoundland.Republic]

@Patrick M. Hausen - Thanks. Adding an IP address to the bridge seems to work. My only question is why prior to upgrading from FreeNAS to TrueNAS did this work without the bridge having an IP address assigned?

 

[Patrick M. Hausen]

I would need the output of the commands above and screenshots of your configuration in the broken state to make an educated guess. It's up to you if that is worth the trouble.

 

[Newfoundland.Republic]

Thanks @Patrick M. Hausen - I'm past the point of no-return with that suggestion .

I'm just curious now as to why the bridge also needs to have an IP address. Was this always a requirement and it was working for me without the bridge having an IP because of a "feature" (now fixed)?

Thanks

 

[Patrick M. Hausen]

The bridge should not also have an IP address. The bridge should be the only interface with an IP address. In that particular physical-VLAN-bridge instance, of course.

Chapter 34. Advanced Networking

Advanced networking in FreeBSD: basics of gateways and routes, CARP, how to configure multiple VLANs on FreeBSD, etc

www.freebsd.org

 

[Newfoundland.Republic]

Thanks. I'l have to take a much deeper dive into this. Right now, I have an IP on the bridge and when I set up the jail (either with DHCP or manually setting an IP) it works fine. But, based on your info, it shouldn't be done this way. Strange.

 

[Patrick M. Hausen]

The jail of course does need its own IP address for its virtual (hence "VNET") interface. It's a completely separate host in that regard.

On the host side the one IP address of the system belongs on the bridge and not the physical. Same for IPv4 and IPv6.

 

[Newfoundland.Republic]

I think I'm getting it. Questions: The bridge for the jail and the jail itself both use IPs from the same subnet, correct? If this is correct, any other jails using the same subnet can use the original bridge?

 

[Patrick M. Hausen]

Yes. Precisely. A bridge is a layer 2 device, so all jails and the host interface are in the same broadcast domain i.e. the same subnet.
You can create as many (within reason) bridges as you fancy and map jails to different VLANs that way. With one subnet per VLAN.

 

[Newfoundland.Republic]

It's like a layer 2 device - now I get it. Thanks!

 

[Newfoundland.Republic]

I "unsolved" this as I have one last question: If my "core" network (only a home lab but my "clients" get restless when Netflix and Steam don't work ) sit on the 10.100.200.x/24 subnet, should my bridge also sit on the 10.100.200.x/24 subnet with all the VLANs (e.g., VLAN20 - 192.168.20.x/24, VLAN25 - 192.168.25.x/24, etc.) using the "core" subnet as I would do for the rest of my networking devices?

Thanks again!

 

[Patrick M. Hausen]

You have one bridge per VLAN ... if you need jails in all the VLANs. If you need jails only in 10.100.200.x/24 then you need just one bidge. And that bridge needs to contain the proper VLAN as the only member ...

Ah ... I think I smell something. You are using untagged and tagged configurations on the same interface? Don't. Well, I recommend you don't. Then reading again I am confused - why would your VLANs use the "core" subnet in any way? Each VLAN has got its own subnet.

tldr; could you provide a quick drawing?

The TrueNAS should be part of that 10.100.200/24, right? It has got an IP address from that subnet now? That IP address should go on the bridge IF. The physical interface or VLAN interface for 10.100.200/24 should have *no* IP address. That is all layer 2 ...

 

[Newfoundland.Republic]

Not sure if this will work. Here is a simplified/high level diagram (slim black lines from Switch 24 to servers are the management physical connections on 192.168.20.x):

This is the TrueNAS network interface configuration

 

[Newfoundland.Republic]

Also the "core" (10.100.200.x) is not a VLAN. It is physical.

 

[Newfoundland.Republic]

It seems, based upon this page Understanding VLAN Configuration on FreeBSD that I can have the untagged 10.100.200.x/24 network for the bridge with the tagged VLANs coming in.

 

[Patrick M. Hausen]

Which network should the jails go into?

I guess it is 10.100.200/24? If yes does your switch pass that "core network" untagged on the trunk port? If again yes, then yes, you can make igb2 a member of bridge0. But your TrueNAS currently does not have any IP address from that network. So you don't need to put one on the bridge, either. Just configure the bridge "up" in the "options" section.

In most cases people have an IP address on the physical interface of the NAS and the jails go into the same network. In that case it is advisable to move the IP address from the physical to the bridge. But the bridge does not have to have an IP address. It's all layer 2. A bridge is a switch and a switch is a bridge ...

 

[Newfoundland.Republic]

Some into VLAN 25, some into VLAN 30, some into VLAN 100. (Mostly trying to fix DLNA but I have some other use cases).

 

[Newfoundland.Republic]

One "symptom" I found when doing is the wrong way was that you could not ping from the jail until another host (not jail) started pinging the jail. It seems to be related to Bridge/epair not passing through tagged VLAN traffic between host and VNET jail

ARP issue it would seem

 

[Patrick M. Hausen]

Some into VLAN 25, some into VLAN 30, some into VLAN 100. (Mostly trying to fix DLNA but I have some other use cases).

In that case you need three separate bridges. One for each VLAN. With the VLAN as the only member.
And you should move the IP address currently assigned to the VLAN interface to the corresponding bridge. If there is one.

VLAN - separate "virtual switch" - layer 2 - get it? You are creating one switch for each VLAN, then "plug in" the jails.