Network traffic not reaching TrueNAS through VLANS

[oracle7]

I manage a TrueNAS box model TRUENAS-MINI-3.0-X, running FreeNAS-11.3-U5. This is the current network configuration:

Code:

Name    IPv4 Address
ix0        OMMITTED/24
ix2        OMMITTED/24
vlan10    OMMITTED/24
vlan150    OMMITTED/24
vlan20    OMMITTED/24

Before adding the VLANs, everything was working fine. When I added the VLANs, devices from inside the VLAN stopped reaching FreeNAS, either through the VLAN address, or the ix0 address.

I understand that devices within the VLANs that try to reach the ix0 address will get a response originated from the VLAN interface and not from ix0. But why a device on vlan10 that tries to reach FreeNAS using the vlan10 interface address returns "Host Unreachable"? I double checked all VLAN configurations for all the network switches regardless of being part of the network path or not. I double checked the network configurations for the computer and the FreeNAS, but I could find nothing.

I'm out of ideas, and if someone can give me a hint as to what should I be looking for, or how I should approach this, it would be greatly appreciated.

 

[indy]

It is not very clear what you are doing.
Could you draw a network diagram?

Generally though packets need to be routed between different VLANs.

 

[oracle7]

Generally though packets need to be routed between different VLANs.

I know, but what about packets that run within the same VLAN? They don't need to be routed, right? It's the same network, so an ARP table is enough.

This is what I'm talking about. A computer at vlan10 should reach FreeNAS on vlan10 directly, without the need for a router. Then why is it not reaching?

This is the drawing I have:

and the lines for the VLANs:

 

[oracle7]

and the lines for the VLANs:

I forgot to mention that dashed lines are untagged VLANs, and solid lines are tagged ones.

 

[jgreco]

Ah, just to be clear here...

There is something that network engineers refer to as "stupid VLAN tricks", often done by neophytes who are trying to implement reachability policy at layer 2, which is guaranteed to cause problems.

Each IP network with an interface on your FreeNAS host needs a unique IP address. These IP addresses MUST each reside on a unique IP network, or things will not work the way they should.

The use of "untagged VLAN's" is a huge red flag that suggests you are trying for "stupid VLAN tricks" to me.

Please reveal the IP addresses you have omitted. If these are /24's, it is fine to alter the first three octets to obfuscate them, as long as you show the allocation strategy. i.e. If you have 10.1.2.10/24 on the IPMI and 10.1.2.11/24 on ix0, it is fine to change those to 192.168.1.10/24 and 192.168.1.11/24 if you do not want to reveal the actual IP addresses.

 

[oracle7]

Interface	Address
ix0	192.168.1.35/24
ix2	192.168.50.5/24
vlan10	10.1.10.250/24
vlan20	10.1.20.250/24
vlan150	10.1.150.250/24

 

[oracle7]

The use of "untagged VLAN's" is a huge red flag that suggests you are trying for "stupid VLAN tricks" to me.

This happened because VLANs were not in place when FreeNAS started running. They were added later.

There is something that network engineers refer to as "stupid VLAN tricks", often done by neophytes who are trying to implement reachability policy at layer 2, which is guaranteed to cause problems.

I know I'm new here, but I hope this rules me out as just a neophyte.

If you have 10.1.2.10/24 on the IPMI

The IPMI here is provided by the motherboard manufacturer, and not by FreeNAS, so it is a completely different ethernet port, and a completely different system/network stack.

 

[oracle7]

it is fine to alter the first three octets to obfuscate them, as long as you show the allocation strategy

The strategy is the same, but the addresses will change soon, so maybe it's not a big problem to reveal them here.

 

[jgreco]

Okay, so, does the ifconfig output show rational output? For example

em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
ether 38:7a:0e:99:58:ae
inet 10.17.40.1 netmask 0xffffff00 broadcast 10.17.40.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
vlan30: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=3<RXCSUM,TXCSUM>
ether 38:7a:0e:99:58:ae
inet 206.55.66.34 netmask 0xffffffe0 broadcast 206.55.66.63
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
vlan: 30 parent interface: em0

This is a valid configuration that has both tagged and untagged vlan trafffic on em0.

 

[oracle7]

This is the output of the ifconfig command. I ommited unused interfaces ix1 and ix3 (they're not configured) and the loopback, as well as the MAC addresses. The rest is OK, I guess.

Code:

# ifconfig
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: ix0
        options=e002bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 192.168.1.35 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>)
        status: active
ix2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 192.168.50.5 netmask 0xffffff00 broadcast 192.168.50.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>)
        status: active
vlan10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: Computers
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.1.10.250 netmask 0xffffff00 broadcast 10.1.10.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>)
        status: active
        vlan: 10 vlanpcp: 0 parent interface: ix0
        groups: vlan
vlan150: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: Server network
        options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.1.150.250 netmask 0xffffff00 broadcast 10.1.150.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>)
        status: active
        vlan: 150 vlanpcp: 0 parent interface: ix2
        groups: vlan
vlan20: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: VoIP
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.1.20.250 netmask 0xffffff00 broadcast 10.1.20.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>)
        status: active
        vlan: 20 vlanpcp: 0 parent interface: ix0
        groups: vlan

Everything has been configured from the FreeNAS GUI, i.o.w., I have not used the CLI for this.

 

[oracle7]

Any new ideas? Anyone?

 

[jgreco]

You're really going to need to inspect MAC forwarding tables and/or tcpdump to see what's going on. Best guess is something amiss in the switching environment. You need to check packet forwarding in each direction to make sure that each vlan is correctly seeing the appropriate broadcast domain.

Certain switchgear may require you to manually configure the PVID for untagged frame ingress which is a common error especially when trying to mix untagged and tagged frames. You may be better off dropping the untagged and going to strict tagging. The Force10 gear, for example, creates a really messy configuration when going with what they call "general" mode access. It makes my eyes cross and I've been doing this stuff since the '80's.

 

[oracle7]

I only work remotely, so when it comes to change the primary way for me to access stuff, it is generally the last thing I try.

But since it's you who's advising that, I'll give it a careful shot and see if my odds increase. I'll let you know if otherwise.

 

[oracle7]

And BTW, any new ideas are appreciated.

 

[jgreco]

I only work remotely,

The closest data center I work in is 80 miles away. My company's two primary data centers are 804 and 2247 miles away (slightly debatable depending on mode of transit and route). I appreciate the need to work remotely.

Part of the game here is that it isn't too rough to sit in your chair and launch packets from one device towards another and see what the tcpdump/wireshark results are, and what's appearing in the ARP tables of the hosts and the MAC forwarding tables of the switchgear. I like sitting in my chair. It's easy and it allows me to post pithy messages on this forum in between real work items.

 

[oracle7]

Part of the game here is that it isn't too rough to sit in your chair and launch packets from one device towards another and see what the tcpdump/wireshark results are, and what's appearing in the ARP tables of the hosts and the MAC forwarding tables of the switchgear. I like sitting in my chair. It's easy and it allows me to post pithy messages on this forum in between real work items.

I know, right? Nothing better than being able to run through some routing tables while sipping an orange juice at the confort of your couch! And I haven't even got to the part where you can get a pizza at 10am in the morning and not be judged...

Seriously, though, I got your point on avoiding mixing tagged and untagged interfaces in the same system. I saw you talking about it in another post here before I decided to write this one. I understood that this actually makes things more complicated instead of more simple, so before I try to delve more deeply into MAC forwarding and ARP tables, I might as well get things straight as they should be.

I took the courage to do that movement because, living 4804 miles away from the network I manage, one mistake and I'm probably locked out of stuff until someone can come in person and hook a computer up, and then I'll have to instruct non-technical people on how to fix my mistake. Been there, done that, it's a RPITA. So I avoid getting near to that place again when I can.

But then again, if dealing only with tagged frames will make stuff easier to manage, then I guess I should be heading that direction.