DHCP Disabled on Non-LAGG Interface when LAGG is Established

[57HL&gXjw!]

Hello All,

I am having an issue where when I setup a LEGG for interfaces em0 and em1 with a static IP Address, DHCP for interface re0 is disabled. This is occurring every time I attempt to setup the LAGG interface. The odd part is this is happening to an interface that is not involved in the LAGG group at all.

Any thoughts or suggestions would be very welcome!

Best,
Bry

 

[57HL&gXjw!]

Hello All,

I am having an issue where when I setup a LEGG for interfaces em0 and em1 with a static IP Address, DHCP for interface re0 is disabled. This is occurring every time I attempt to setup the LAGG interface. The odd part is this is happening to an interface that is not involved in the LAGG group at all.

Any thoughts or suggestions would be very welcome!

Best,
Bry

A quick update... every time I attempt to establish this LAGG and test the new network settings, all interfaces fail to connect. The only remedy I have found is to perform a complete reset on the NAS.

Best,
Bry

 

[Samuel Tai]

By any chance, are you trying to put lagg0 and re0 in the same subnet? Within a subnet, you can only have DHCP or static for all interfaces on that subnet.

 

[57HL&gXjw!]

Samuel,

Thank you for the quick reply. Unfortunately in my case this is not the issue.

re0 is configured for my management network on 192.168.1.1/24 while lagg0 I am attempting to configure for my user network 10.1.0.1/24. Whenever I setup this LAGG interface (using physical interfaces em0 and em1), DHCP is suddenly disabled for interface re0. I can see this in the listing of the interfaces as well as when I go into the settings for re0 - where the DHCP option is unchecked.

Thanks again for your help!!!

Best,
Bry

 

[Samuel Tai]

Then you have conflicting default routes. You can only have 1 default route, and it's probably via the LAGG, which overrides the default route via DHCP. In your case, you'll have to go with static addresses on both re0 and lagg0, and will need to set a default route via lagg0, with specific routes via re0 for management.

 

[57HL&gXjw!]

Then you have conflicting default routes. You can only have 1 default route, and it's probably via the LAGG, which overrides the default route via DHCP. In your case, you'll have to go with static addresses on both re0 and lagg0, and will need to set a default route via lagg0, with specific routes via re0 for management.

I want re0 as my management interface at 192.168.1.3(/24)
I want em0 and em1 in a LAGG group as interface lagg0 at 10.1.0.3(/24) - This will be my user network that I will want to bind SMB to.
I will also soon be installing a dual SFP+ adapter. I would like to use [sfp]0 as a single interface to my server network on 172.16.33.3 (/24) - I hope to bind NFS service this IP / interface.


So just to make sure I am clear on what you are suggesting...

1. Setup Static IP for re0 at 192.168.1.3
2. Create LAGG (lagg0) with em0 and em1 at 10.1.0.3
3. Set the default route for lagg0 - next hop 10.1.0.1 (the gateway)
4. Setup static route for re0 - next hop 192.168.1.1 (the gateway)

After I install the new SFP+ adapters, I am assuming I will have to go through the same thing all over again and make sure I setup A) a static IP for [sfp]0 the static routes for the [sfp]0 interface.

Does this all sound about right?

Thanks again for your help and guidance.

Best,
Bryan

 

[57HL&gXjw!]

I want re0 as my management interface at 192.168.1.3(/24)
I want em0 and em1 in a LAGG group as interface lagg0 at 10.1.0.3(/24) - This will be my user network that I will want to bind SMB to.
I will also soon be installing a dual SFP+ adapter. I would like to use [sfp]0 as a single interface to my server network on 172.16.33.3 (/24) - I hope to bind NFS service this IP / interface.


So just to make sure I am clear on what you are suggesting...

1. Setup Static IP for re0 at 192.168.1.3
2. Create LAGG (lagg0) with em0 and em1 at 10.1.0.3
3. Set the default route for lagg0 - next hop 10.1.0.1 (the gateway)
4. Setup static route for re0 - next hop 192.168.1.1 (the gateway)

After I install the new SFP+ adapters, I am assuming I will have to go through the same thing all over again and make sure I setup A) a static IP for [sfp]0 the static routes for the [sfp]0 interface.

Does this all sound about right?

Thanks again for your help and guidance.

Best,
Bryan

An update...

I just tried configuring this again as described. I'm still having the same issues. I can get out to the network (and the internet) from TrueNAS but I cannot connect to it from any network. The minute I reset TrueNAS I can get to it again. I have tried getting all this setup through both cli and via the web interface and still have not had any luck.

I set the default route for 10.1.0.1

 

[57HL&gXjw!]

An update...

I just tried configuring this again as described. I'm still having the same issues. I can get out to the network (and the internet) from TrueNAS but I cannot connect to it from any network. The minute I reset TrueNAS I can get to it again. I have tried getting all this setup through both cli and via the web interface and still have not had any luck.

I set the default route for 10.1.0.1

I think I may have figured this all out... TrueNAS appears to be really picky with the order in which you setup (and connect) interfaces. Resetting the system again, I followed this order to get everything up-and-running:

Physical:

1. Disconnect ALL Network Connections - Do this prior to performing the reset to avoid complications.

VIA CLI:

1. Setup lagg0 via CLI
   • em0 and em1 selected as part of lagg0
   • lagg0 setup as lacp
2. Setup network settings for legg0
   • No DHCP
   • IP Addr & Net: 10.1.0.3/24
3. Setup Default Route
   • 10.1.0.1 (the gateway)

Physical:

1. Connect em0 and em1 to switch for user.lan network - ensure switch ports are configured for LACP Link Aggregation

VIA Web UI:

1. Set the following Network Settings (Global Configuration):
   • Hostname: [updated accordingly]
   • Domain: user.lan
   • Additional Domains: mgmt.lan & storage.lan
   • Nameserver 1: 10.1.0.1 (Gateway)
   • Nameserver 2: 208.67.222.222 (OpenDNS - Now Cisco Umbrella)
   • Nameserver 3: 208.67.220.220 (OpenDNS - Now Cisco Umbrella)
2. Setup Interface re0
   • No DHCP
   • IP Addr & Net: 192.168.1.3/24
3. Test Changes
   • Set testing time to 120 seconds.
   • Make (Test) Configuration Changes
4. Commit Configuration Changes
   • If the above worked and you still have access to TrueNAS on 10.1.0.3, then commit changes.
5. Setup SMB Service to bind to lagg0 IP (10.1.0.3)
   • Services -> SMB -> Advanced Options -> Bind IP Addresses

Physical:

1. Connect re0 to switch for mgmt.lan network.

Web UI:

1. Check to ensure you can still access the Web UI on 10.1.0.3
2. Check to see if Web UI is now accessible on 192.168.1.3
   • I really wish there were a nice way inside TrueNAS / FreeNAS to setup the interface on which the Web UI is accessible. For security purposes, I'd much rather than this interface only accessible via the Management Network (192.168.1.1/24); however, that does not appear to be possible.

I will be going through this again once I have installed the dual SFP+ adapter; however, now that I have the foundational work done, I am hoping this will now be a much smoother process.


Additional Notes:
I have also taken note of some oddities that seem to relate to the Ubiquiti gear that is used for this network. For some reason, Unifi has a habit of assigning DHCP leases to systems that are setup to use a static IP (not a DHCP Reserve, but a static setup on the endpoint) - regardless of the fact that none of these systems are requesting a lease. I am not certain what weird configurations Ubiquiti has made within their gateways that would cause this; but it was causing a great number of issues. The only work around I could find is to set BOTH a static IP on the endpoint (the server) AND set a DHCP Reserve for the MAC Address of the server inside Unifi (with the same address that was statically set on the endpoint). This issue is further exacerbated by how Unifi allows the allocation of DHCP Reserve IPs outside the defined DHCP pool(s). As a long-time network engineer, I can say this is nowhere near now DHCP should be operating on a network; however, everyone knows that Ubiquiti is a little... special...