TrueNAS-SCALE-22.12.4.2 - Can't assign static IP to bond0 (lagg) interface

perryb

Cadet
Joined
Apr 10, 2018
Messages
9
I have two network interfaces setup for LAGG on my TrueNAS SCALE. Everything works fine as long as I use a dhcp reservation on my dhcp server to assign an IP address to the bond0 interface. Now I want to set the change the IP address AND set it statically on the TrueNAS box. However, I cannot set the IP address manually etiher through the web GUI, the menu driven CLI interface, or from the actual CLI. Whenever I try to set the alias for the bond0 interface, etiher by the Web GUI or the CLI menu, TrueNAS throws a runtime error. From the Web GUI, I got to:

network/intefaces/bond0
go down to bottom of the bond0 interface pop-out to the "add alias" section (by the way, the DHCP radio button on the pop-out for bond0 is unchecked and I still get a dhcp assined ip), enter the new ip address (in my case I enter 192.168.0.3 /24). When I click save a runtime error window pops up that says the following:

Code:
 Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 184, in call_method
    result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self)
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1317, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/lib/python3/dist-packages/middlewared/service.py", line 951, in update
    rv = await self.middleware._call(
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1317, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1247, in nf
    res = await f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1379, in nf
    return await func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/network.py", line 1253, in do_update
    lag_id = await self.middleware.call(
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1368, in call
    return await self._call(

  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1317, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1379, in nf
    return await func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/datastore/write.py", line 103, in update
    raise RuntimeError(f'{len(rows)} found, expecting one')
RuntimeError: 0 found, expecting one



When I close bothe the error and bond0 configuration windows, I am now able the select the "Test Changes" button. When I select the "Test Changes," it thinks for about 10 seconds, then nothing changes and it keeps the orginal dhcp assigned IP address. Essentially the same theing happens when I try to change it for the CLI menu. When I try to get into the CLI /etc/network/interfaces section, it's very empty - notthing configured,all files are empty.

I've serached the boards and Google about this problem and while there are those that have problems setting a static IP on their trueNAS SCALE, none of them have my specific issue. Can anyone help me get around this problem and statically assign the IP address information to my (Lagg) bond0 interface?

Thank you
 

PhilD13

Patron
Joined
Sep 18, 2020
Messages
203
Adding a LAGG interface will prevent the SMB service from using SMB Multichannel. If using SMB shares you might investigate that instead of using LAGG as it might be easier to setup. You can't use both at once so you have to select one or the other.

I don't remember everythig I did or tried to do to get LAGG working but do remember the LAGG setup was a real bear (on and earlier Bluefin version) and it should not have been that difficult, taken multiple tries even using the docs, or taken as long as it did.

I do remember that I setup the switch LAGG first with the 2 switch ports I wanted to use for the LAGG, then in Truenas after setting a static ip on the first interface, I had to reboot after testing and saving. I unplugged the first interface, then I set the second interface to static then reboot after testing and saving it. Then I added an NEW interface of type "Link Aggreation" then setup the LAGG settings in Truenas to match the switch LAGG protocol and set up the LAGG with it's own static ip address different from the 2 interfaces, and then added the previous two physical interfaces I had set to static as Aliases in the LAGG setup, tested, saved, plugged both cables into the switch and rebooted.

You add a new interface set to Link Aggreation and set the aliases in the new interface not the individual interfaces.
 

perryb

Cadet
Joined
Apr 10, 2018
Messages
9
Thanks for the reply and maybe I completely misunderstood your response. I don't have an issue using LAGG or using SMB with LAGG, no problem at all there. In fact I can measure the full utilization of the 2x1GB ports while transferring files from the SMB (Cifs) service of my TrueNAS server - if I unplug one of the aggregated ports, it keeps on moving packets as it should. My problem is setting the bond0 interface, which was established during my LAGG configuration and setup, to use a static IP address. It works just fine using a dhcp assigned address, but I have no control over enabling or disabling dhcp, or giving the bond0 interface a static IP address. When I try to set the bond0 to a static IP, I get that runtime error I posted in my original message. I can continue to use the dhcp assigned address and make sure it always gets the same IP using a dhcp reservation, but it concerns me to know that I have no control over where and how the dhcp is enabled/disabled (read my initial post, the dhcp radio button is unchecked on the web gui bond0 interface configuration section, but it still grabs a dhcp address from the server) and the fact that something is definitely broken in the code.

It would help tremendously if I knew where in the TrueNAS files system that all the network configurations and dhcp lived. As I stated in my original post, these configurations do not exists under /etc/network.... or /etc/dhcp, but the darn thing is grabbing and using an IP address and storing it somewhere!
 

PhilD13

Patron
Joined
Sep 18, 2020
Messages
203
I do remember that I setup the switch LAGG first with the 2 switch ports I wanted to use for the LAGG, then in Truenas after setting a static ip on the first interface, I had to reboot after testing and saving. I unplugged the first interface, then I set the second interface to static then reboot after testing and saving it. Then I added an NEW interface of type "Link Aggreation" then setup the LAGG settings in Truenas to match the switch LAGG protocol and set up the LAGG with it's own static ip address different from the 2 interfaces, and then added the previous two physical interfaces I had set to static as Aliases in the LAGG setup, tested, saved, plugged both cables into the switch and rebooted.
I don't know where the network files are or if they are exposed at all. I think Truenas wants it that way for various support reasons.

The above quote is perhaps not correct (faulty memory?) and I'll need to decide if I wish to alter it or do something else to it. It was over 6 months ago when I had a catastropic failure in a QNAP backplane on the primary storage server and decided to move the data to a used Supermicro server running Truenas Scale. I was under a time crunch on two other unrelated projects going on at the same time with seperate issues and only partially read the docs or did not fully understand them at the time. I did do some reading after I posted the info and the above quote is probably way off of what is supposed to go on.

I set all my servers with static ip addresses but apparently in Truenas you should leave the interfaces as they came and don't try to set static ip addresses for the interfaces. Set the static ip in the lagg setup instead.

Try this. I think it may work for you
In the setup for adding a new interface

Select Type: Link Aggregation Interfaces
On the setup page for the new lagg interface:
Select the interfaces you want to use in the link aggregation. --> pick both interfaces to use from the dropdown.
Further down the page under Aliases set the static ip address desired for the lagg interface. The alias can be an IPv4 or IPv6 address
Configure the rest of the lagg interface settings as necessary
Save and reboot

Warning: Link Aggregation creation fails if any of the selected interfaces have been manually configured.

I apparently missed that line of information when I did my setup. Apparently, I had tried configuring the interfaces as static to start and being only partially successful. I then wound up going about it in a cumbersome and wrong way and that part was what I remembered.

The suggestion to use Multichannel over lagg was just thrown out there in case it was not known to be an option available.

[edit 12/23/23: I was unable to make the above work on a old server updated to the latest Cobia, so I am at this time unable to help solve the issue. Maybe someone else can and I can learn something too.]
 
Last edited:

perryb

Cadet
Joined
Apr 10, 2018
Messages
9
Total nightmare, why can't TrueNAS get networking right. I ended up trying to rebuild (start over) with building a lagg interface for my two TrueNAS NICs. There is no long and option for doing this from the CLI menu like it was when I first built that LAGG in a previous version of TrueNAS. Now it seems the only option is to do it through the web GUI which is riddled with bugginess. Here's what I now get when I try to build a LAGG interface on my TrueNAS SCALE:

Code:
Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
sqlite3.IntegrityError: UNIQUE constraint failed: network_lagginterfacemembers.lagg_physnic

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 201, in call_method
    result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1342, in _call
    return await methodobj(*prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 169, in create
    return await self.middleware._call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1342, in _call
    return await methodobj(*prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 194, in nf
    rv = await func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 44, in nf
    res = await f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/network.py", line 802, in do_create
    lagports_ids += await self.__set_lag_ports(lag_id, data['lag_ports'])
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/network.py", line 1149, in __set_lag_ports
    await self.middleware.call(
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1399, in call
    return await self._call(
           ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1342, in _call
    return await methodobj(*prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/datastore/write.py", line 62, in insert
    result = await self.middleware.call(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1399, in call
    return await self._call(
           ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1353, in _call
    return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1251, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/datastore/connection.py", line 106, in execute_write
    result = self.connection.execute(sql, binds)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1365, in execute
    return self._exec_driver_sql(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1669, in _exec_driver_sql
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: network_lagginterfacemembers.lagg_physnic
[SQL: INSERT INTO network_lagginterfacemembers (lagg_ordernum, lagg_physnic, lagg_interfacegroup_id) VALUES (?, ?, ?)]
[parameters: (0, 'enp1s0f0', 4)]
(Background on this error at: https://sqlalche.me/e/14/gkpj)


I don't know what is going on and why they can't get this right. I'm just going to have to hobble along on a single interface until this gets fixed - super furstrating.
 
Top