60+ emails today for "Failed to get properties: Transport endpoint is not connected"

[ctag]

I'm curious if anyone else has seen this. Today I've been getting clusters of alert emails from my server every hour or so.

New alerts:

• Failed to get properties: Transport endpoint is not connected

Current alerts:

• Failed to check for alert NTLMv1Authentication: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/middlewared/plugins/alert.py", line 808, in __run_source alerts = (await alert_source.check()) or [] ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/alert/source/smb.py", line 76, in check if not await self.middleware.call('service.started', 'cifs'): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1399, in call return await self._call( ^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1342, in _call return await methodobj(*prepared_call.args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 44, in nf res = await f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/plugins/service.py", line 201, in started state = await service_object.get_state() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/plugins/service_/services/base.py", line 37, in get_state return await self.middleware.run_in_thread(self._get_state_sync) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1254, in run_in_thread return await self.run_in_executor(self.thread_pool_executor, method, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1251, in run_in_executor return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/plugins/service_/services/base.py", line 42, in _get_state_sync state = unit.Unit.ActiveState ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pystemd/base.py", line 191, in _call return func(self, name, *args) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pystemd/base.py", line 128, in _get_property return bus.get_property( ^^^^^^^^^^^^^^^^^ File "pystemd/dbuslib.pyx", line 482, in pystemd.dbuslib.DBus.get_property pystemd.dbusexc.DBusTimeoutError: [err -110]: b'Connection timed out'
• Failed to check for alert SMBLegacyProtocol: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/middlewared/plugins/alert.py", line 808, in __run_source alerts = (await alert_source.check()) or [] ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/alert/source/smb.py", line 53, in check if not await self.middleware.call('service.started', 'cifs'): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1399, in call return await self._call( ^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1342, in _call return await methodobj(*prepared_call.args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 44, in nf res = await f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/plugins/service.py", line 201, in started state = await service_object.get_state() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/plugins/service_/services/base.py", line 37, in get_state return await self.middleware.run_in_thread(self._get_state_sync) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1254, in run_in_thread return await self.run_in_executor(self.thread_pool_executor, method, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1251, in run_in_executor return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/plugins/service_/services/base.py", line 40, in _get_state_sync unit = self._get_systemd_unit() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/middlewared/plugins/service_/services/base.py", line 68, in _get_systemd_unit unit.load() File "/usr/lib/python3/dist-packages/pystemd/base.py", line 90, in load unit_xml = self.get_introspect_xml() ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pystemd/base.py", line 75, in get_introspect_xml bus.call_method( File "pystemd/dbuslib.pyx", line 446, in pystemd.dbuslib.DBus.call_method pystemd.dbusexc.DBusTimeoutError: [err -110]: b'Connection timed out'
• Failed to get properties: Transport endpoint is not connected

 

[ctag]

Alerts continue. Rebooting the whole system this morning didn't help.

This is on TrueNAS-SCALE-23.10.1.3.

Searching for "Transport endpoint is not connected" returns a lot of results for systemd, but I don't see anything directly related to Truenas.