FreeNAS, UniFi and Link Aggregation

[IronSheepdog]

I am running FreeNAS 11.3 U3.1 on a custom build with an Asus motherboard that has two physical NICs on it. I am also running a UniFi network (USG firewall, 24 port 250w switch, two other UniFi 8 port switches, AP-AC-LR, and Cloud Key Gen2). UniFi supports LACP and it is very easy to setup. Keep in mind this is a home network. According to FreeNAS, it supports active Link Aggregation (LACP). Please correct me if I'm wrong. Also according to FreeNAS, you cannot assign both of the NICs to be on the same network. Right now, I have multiple networks in UniFi:
192.168.10.0 for main network
192.168.20.0 for cameras
192.168.50.0 for internet of things devices
192.168.70.0 for guest network

All of these networks are designed to be isolated from each other with few exceptions (i.e. my NVR computer is on the main network yet has access to the cameras).

Has anyone had any success with implementing link aggregation with FreeNAS on a UniFi switch?

I ask this because FreeNAS has no need to be on any other network other than my main network. But if I assign a static IP address for my main network to each FreeNAS NIC in UniFi, it seems to crash the network. I can "uncrash" it by simply unplugging the second FreeNAS NIC. And by crashing the network, I mean all devices on my home network lose internet connectivity as well as connectivity to the network itself.

FYI, I assign IP addresses via UniFi's controller. I never assign IP addresses on the devices themselves.

I have some experience networking, but forgive me. This is new territory for me.

 

[Elliot Dierksen]

According to FreeNAS, it supports active Link Aggregation (LACP).

Yes, it does. When you build a link aggregation group (LAGG), you need to specify LACP. The UniFi would also need to be configured for LACP on the same set of ports.

FYI, I assign IP addresses via UniFi's controller. I never assign IP addresses on the devices themselves.

I would not go this way. DHCP is going to pass info that might cause issues. If you are going to do link aggregation, assign the static IP that way in the FreeNAS box. It is difficult to do this from the GUI if that is your only management access. IMHO it is easier to do this from the console.

 

[IronSheepdog]

I would not go this way. DHCP is going to pass info that might cause issues. If you are going to do link aggregation, assign the static IP that way in the FreeNAS box. It is difficult to do this from the GUI if that is your only management access. IMHO it is easier to do this from the console.

I did notice that in the GUI, it does say that you cannot have DHCP for more than one NIC at a time. Either way, my biggest concern is needing to have each NIC on a different network. I'm not sure why, but FreeNAS doesn't let you have two or more NICs on the same network.

The problem that hasn't been addressed yet is what I stated in my first post. All networks are isolated from one another (with few exceptions). And, I have no need for FreeNAS to have access to any of the other networks other than my main network.

What would happen if I created a LAGG on FreeNAS with my two NICs; one on my main network and one on another network and the two networks are completely isolated from one another? Would LACP even work? Would it crash my network again?

 

[Samuel Tai]

What would happen if I created a LAGG on FreeNAS with my two NICs; one on my main network and one on another network and the two networks are completely isolated from one another? Would LACP even work? Would it crash my network again?

No, this wouldn't work. To create a LAGG, you'll need to remove the existing config on each member of the LAGG, and they have to be on the same network. You'll also need to use static addressing.

 

[danb35]

I'm not sure why, but FreeNAS doesn't let you have two or more NICs on the same network.

One of the sticky threads in this very forum answers this question:

Multiple network interfaces on a single subnet

We've had a number of people over the years bring up the topic of multiple network interfaces on a single subnet. Often done by Windows administrators or storage admins, where it works to varying degrees, this is sadly not proper IP networking practice. Multiple interfaces on a single network...

www.ixsystems.com

 

[IronSheepdog]

No, this wouldn't work. To create a LAGG, you'll need to remove the existing config on each member of the LAGG, and they have to be on the same network. You'll also need to use static addressing.

I guess I'm confused. If you look at the FreeNAS 11.3 U3.1 user manual section on Networking, it states that "Multiple interfaces cannot be members of the same subnet." FreeNAS tells me this if I try to assign static IP addresses from the same network/subnet to each NIC in FreeNAS. In fact, it won't let me do it at all. My way around it was to go into UniFi and manually assign IP addresses to each connection in the router. However, I believe this is what crashed the network the first time.

 

[danb35]

Look more closely at the section on link aggregation. In short, the LAGG is its own interface. The LAGG has an address; the individual NICs do not.

 

[IronSheepdog]

Look more closely at the section on link aggregation. In short, the LAGG is its own interface. The LAGG has an address; the individual NICs do not.

So, let me state what I think I understand and tell me if I'm wrong. I go into the console and remove configuration for each NIC (if any). Then create a LAGG?
My concerns are: Does it matter if UniFi takes over and DHCP assigns an IP to each NIC before I create a LAGG?
And, is it better to setup Link Aggregation on the switch before or after creating a LAGG in FreeNAS?

 

[Samuel Tai]

My concerns are: Does it matter if UniFi takes over and DHCP assigns an IP to each NIC before I create a LAGG?
And, is it better to setup Link Aggregation on the switch before or after creating a LAGG in FreeNAS?

Unplug both NICs to prevent DHCP assignments. Set up the LAGG on both sides, and then reconnect.

 

[IronSheepdog]

Unplug both NICs to prevent DHCP assignments. Set up the LAGG on both sides, and then reconnect.

Thank you! That works. I unplugged everything and removed any DHCP settings for the FreeNAS server from UniFi. I used the console and deleted both interfaces and created a LAGG. Instead of using DHCP to configure the IP address, I configured it on the FreeNAS console. Sweet!

I realized I had to change settings in a VM that I have. But, I have a slightly just as big problem now. My Plex Media Server cannot access the network. I went into the jail and told it to use lagg0 as the interface, but it still doesn't work. It says it's running correctly, but I can't access it on the network. I tried DHCP and assigning the static IP on FreeNAS. Any ideas? I would assume it's just like configuring most other jails to use lagg0.

 

[IronSheepdog]

I wish I could edit my posts, but I can't. I got it working by manually configuring the Global Configuration under Network with the Default Gateway and Name Servers. I have tried both DCHP and manually configuring the Plex jail. What is odd is that if I got to plex.tv I can access my media and everything. However, if I click on the "Manage" link for the Plex plugin, it uses the network address of http://192.168.10.XXX:32400/web. This link does not work at all. It worked before. Any ideas?

 

[Samuel Tai]

Thank you! That works. I unplugged everything and removed any DHCP settings for the FreeNAS server from UniFi. I used the console and deleted both interfaces and created a LAGG. Instead of using DHCP to configure the IP address, I configured it on the FreeNAS console. Sweet!

I realized I had to change settings in a VM that I have. But, I have a slightly just as big problem now. My Plex Media Server cannot access the network. I went into the jail and told it to use lagg0 as the interface, but it still doesn't work. It says it's running correctly, but I can't access it on the network. I tried DHCP and assigning the static IP on FreeNAS. Any ideas? I would assume it's just like configuring most other jails to use lagg0.

Plex uses bridge0. Stop the jail. Create a bridge0 interface, and bind it to the LAGG interface.

 

[IronSheepdog]

Plex uses bridge0. Stop the jail. Create a bridge0 interface, and bind it to the LAGG interface.

I created bridg0 and binded lagg0 to it. However, I don't see an option for bridge0 in the Plex jail.

 

[Samuel Tai]

The Plex jail automatically connects to bridge0. When the jail is active, look for a vnet0.x interface outside the jail, which will be bound to epair0b inside the jail.

 

[Samuel Tai]

You could also set the Plex jail as follows:

Under vnet_default_interface, switch from auto to your lagg0 interface.

 

[IronSheepdog]

The Plex jail automatically connects to bridge0. When the jail is active, look for a vnet0.x interface outside the jail, which will be bound to epair0b inside the jail.

No vnet interface is showing.

 

[IronSheepdog]

You could also set the Plex jail as follows:

View attachment 38872

Under vnet_default_interface, switch from auto to your lagg0 interface.

I removed the bridge0 I created and did this (because choosing the lagg0 interface isn't an option with bridge0 created). And like I said before, my Plex app and plex.tv can access the server. When I click Manage on the Plex plugin, I can't access it. It's odd.

 

[IronSheepdog]

Anyone have any ideas why this isn't working?

 

[Samuel Tai]

Anyone have any ideas why this isn't working?

There are a couple of possibilities:

1. Your workstation has cached the wrong ARP entry for the Plex jail IP. Make sure when you ping <Plex jail IP>, and then arp -a <Plex jail IP> the MAC address matches the MAC set for the lagg0 interface.
2. In Plex itself, under Settings->Network, have you configured Secure Connections=Required? Then the http Plex management URL won't work.
3. Also, under Settings->Network, what do you have configured for the Preferred network interface? This should match the jail's vnet interface.

 

[IronSheepdog]

It was #2. Thank you very much!