So does LACP work?

[meowserkat]

So...I read some older posts for LACP and nothing really gave me the indication that it worked or not, there were a bunch of 'try this or that' but nothing that I felt concrete to say LACP does or not not fully work.

I just bought a 'XYRATEX IBM HS-1235E 12 BAY RAID Server XEON L5410 2.3GHz / 16GB / 12TB' from ebay. The unit came with Freenas 11.1-U7. I swapped out the drives with a bunch of 2TB I had from an old Lacie NAS that died. The unit came with 2 onboard NICs, and two 4-port NICS, along with two dual SPF cards with fiber. I pulled the fiber and left the two 4-port NIC. I took one of the onboard NICs and added it to my VLAN for servers. I took 1 port off each of the 4 ports and added it to my VLAN for my ESXi hosts.

On my Cisco 3560 I setup a port-channel mode Active for those two ports on the ESXI VLAN. Inside Freenas I enabled LACP on the two NICs that came up on the ESXi VLAN. Once I did this I lost all access to the Freenas. I was using the Server VLAN IP of the onboard NIC to mange the Freenas at the time. The only way I was able to get access back to the Freenas was to delete the LAGG interface from the console.

 

[Heracles]

Hi,

In short, I would say that LACP does work but does not provide what people are looking for.

LACP will not aggregate the bandwith of many NIC to provide a single network channel equal to the sum of your links.
LACP will work by providing redundancy and multiple paths for many different network access to go over different NIC.

From what you described, I would say that you got something wrong in your config. When things work, there is no problem. When things do not work at all, there is no big problems. It is when things half-work that the situation is at the worst. Here, you were not working at all, so no big problem. Just need to find the single little piece you got wrong and fix it.

Because you did not post any config, I can not go further than that...

Good luck fixing your config,

 

[Mlovelace]

So...I read some older posts for LACP and nothing really gave me the indication that it worked or not, there were a bunch of 'try this or that' but nothing that I felt concrete to say LACP does or not not fully work.

I just bought a 'XYRATEX IBM HS-1235E 12 BAY RAID Server XEON L5410 2.3GHz / 16GB / 12TB' from ebay. The unit came with Freenas 11.1-U7. I swapped out the drives with a bunch of 2TB I had from an old Lacie NAS that died. The unit came with 2 onboard NICs, and two 4-port NICS, along with two dual SPF cards with fiber. I pulled the fiber and left the two 4-port NIC. I took one of the onboard NICs and added it to my VLAN for servers. I took 1 port off each of the 4 ports and added it to my VLAN for my ESXi hosts.

On my Cisco 3560 I setup a port-channel mode Active for those two ports on the ESXI VLAN. Inside Freenas I enabled LACP on the two NICs that came up on the ESXi VLAN. Once I did this I lost all access to the Freenas. I was using the Server VLAN IP of the onboard NIC to mange the Freenas at the time. The only way I was able to get access back to the Freenas was to delete the LAGG interface from the console.

https://www.ixsystems.com/community/threads/jumbo-frames-on-lagg.74503/post-520162

 

[meowserkat]

Here is the config on my switch for those ports. Port 0/3 is my management port.

Code:

interface Port-channel30
 switchport access vlan 125
 switchport mode access
 switchport nonegotiate
!
interface GigabitEthernet0/3
 description iDRAC
 switchport access vlan 120
 switchport mode access
 spanning-tree portfast
!
interface GigabitEthernet0/9
 description Servers
 switchport access vlan 125
 switchport mode access
 switchport nonegotiate
 channel-group 30 mode active
!
interface GigabitEthernet0/10
 description Servers
 switchport access vlan 125
 switchport mode access
 switchport nonegotiate
 channel-group 30 mode active
!



*Apr  4 22:19:08.124: %LINK-3-UPDOWN: Interface GigabitEthernet0/9, changed state to up
*Apr  4 22:19:11.010: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/9, changed state to up
*Apr  4 22:19:11.723: %LINK-3-UPDOWN: Interface GigabitEthernet0/10, changed state to up
*Apr  4 22:19:12.008: %LINK-3-UPDOWN: Interface Port-channel30, changed state to up
*Apr  4 22:19:13.015: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel30, changed state to up
*Apr  4 22:19:13.476: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/10, changed state to up

I am connected to 10.10.120.202 on the web interface.

This is the setup on the FreeNas... So I take em3 & em9 and add to LACP

Adding em3 & em9 to LACP... at this point I lose all access to FreeNas

When I look at the console I see "The web interface cannot be accessed, please check network configuration". I made sure I did not select em4, which is my web interface. But disappears persay, when I add those other two interfaces to the LAGG. Only way I get back in is deleting the LAGG config from the console, then reboot.


Mlovelace I will try your link next, later this afternoon. Thanks.

 

[Mlovelace]

Here is the config on my switch for those ports. Port 0/3 is my management port.

Code:

interface Port-channel30
switchport access vlan 125
switchport mode access
switchport nonegotiate
!
interface GigabitEthernet0/3
description iDRAC
switchport access vlan 120
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet0/9
description Servers
switchport access vlan 125
switchport mode access
switchport nonegotiate
channel-group 30 mode active
!
interface GigabitEthernet0/10
description Servers
switchport access vlan 125
switchport mode access
switchport nonegotiate
channel-group 30 mode active
!



*Apr  4 22:19:08.124: %LINK-3-UPDOWN: Interface GigabitEthernet0/9, changed state to up
*Apr  4 22:19:11.010: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/9, changed state to up
*Apr  4 22:19:11.723: %LINK-3-UPDOWN: Interface GigabitEthernet0/10, changed state to up
*Apr  4 22:19:12.008: %LINK-3-UPDOWN: Interface Port-channel30, changed state to up
*Apr  4 22:19:13.015: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel30, changed state to up
*Apr  4 22:19:13.476: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/10, changed state to up

I am connected to 10.10.120.202 on the web interface.

This is the setup on the FreeNas... So I take em3 & em9 and add to LACP

View attachment 30268

Adding em3 & em9 to LACP... at this point I lose all access to FreeNas
View attachment 30269

When I look at the console I see "The web interface cannot be accessed, please check network configuration". I made sure I did not select em4, which is my web interface. But disappears persay, when I add those other two interfaces to the LAGG. Only way I get back in is deleting the LAGG config from the console, then reboot.


Mlovelace I will try your link next, later this afternoon. Thanks.

The interfaces that are going to be members of the LAGG must be un-configured. Create the LAGG at the freeNAS console, reboot, then configure the new LAGG interface with the network information.

 

[Elliot Dierksen]

This is the setup on the FreeNas... So I take em3 & em9 and add to LACP

When you put interfaces into a LAGG, you must remove all IP configuration from the physical interfaces. You assign the IP's to the LAGG or to a VLAN sub-interface of the LAGG. On the Cisco switch side, all the members of the port-channel interface must have the same configuration as the port-channel interface. To have multiple IP networks/VLAN's, you will have to make those interfaces into trunk ports. You should only allow the VLAN's that are required on that trunk interface

Code:

switchport trunk allowed vlan 120,125

I made an educated guess on the VLAN numbers, but hopefully you get the idea. You may also have some unintended behavior by having multiple IP interfaces on the same network (10.10.125.0/24)

Edit: auto correct erroneously made the 'switchport' directive into two words. :-(

 

[meowserkat]

I do have DHCP enabled on 125. But I see what you are saying about not having them configured. I can try that before I head out this morning.

 

[SweetAndLow]

I use lacp with my sever. I have it setup with 2 10gig sfp+ connections from my Chelsie nic to my ubiquity switch. It's easy to setup once you figure out you have to use the console and reboot.

 

[meowserkat]

SMH, I overlooked where everyone was saying use the console for LAGG. I did this and still didn't have a web interface IP, but I configured a static and was able to get back in. Also the LAGG is showing LACP on those two interfaces. I turned the ports back on in my switch and the LACP the port-channel protocol is up.

Code:

*Apr  5 00:06:53.769: %LINK-3-UPDOWN: Interface GigabitEthernet0/9, changed state to down
*Apr  5 00:06:53.778: %LINK-3-UPDOWN: Interface GigabitEthernet0/10, changed state to down
*Apr  5 00:06:56.126: %LINK-3-UPDOWN: Interface GigabitEthernet0/10, changed state to up
*Apr  5 00:06:56.236: %LINK-3-UPDOWN: Interface GigabitEthernet0/9, changed state to up
*Apr  5 00:06:58.953: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/9, changed state to up
*Apr  5 00:06:59.046: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/10, changed state to up
*Apr  5 00:06:59.952: %LINK-3-UPDOWN: Interface Port-channel30, changed state to up
*Apr  5 00:07:00.958: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel30, changed state to up

Thank you everyone for the quick responses and amazing help! Obviously, I am a newbie to FreeNAS, but I can tell already this an amazing community.