Crash of middleware on TrueNAS 12.0-U8.1

[Samuel Mutel]

Hello,

Sometimes the connection through the web ui of our TrueNas is not responding anymore.
Restarting middleware service and nginx restore the access but not all the time. Sometimes we need to reboot the truenas completely.

I grab some logs below but not sure it's related to our issue:

[2022/05/23 12:26:03] (ERROR) middlewared.job.run():379 - Job <bound method accepts.<locals>.wrap.<locals>.nf of <middlewared.plugins.rsync.RsyncTaskService object at 0x819e75dc0>> failed
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 367, in run
await self.future
File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 405, in __run_body
rv = await self.middleware.run_in_thread(self.method, *([self] + args))
File "/usr/local/lib/python3.9/site-packages/middlewared/utils/run_in_thread.py", line 10, in run_in_thread
return await self.loop.run_in_executor(self.run_in_thread_executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 979, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/rsync.py", line 712, in run
raise CallError(
middlewared.service_exception.CallError: [EFAULT] rsync command returned 23. Check logs for further information.
[2022/05/23 12:27:34] (ERROR) middlewared.job.run():379 - Job <bound method accepts.<locals>.wrap.<locals>.nf of <middlewared.plugins.rsync.RsyncTaskService object at 0x819e75dc0>> failed
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 367, in run
await self.future
File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 405, in __run_body
rv = await self.middleware.run_in_thread(self.method, *([self] + args))
File "/usr/local/lib/python3.9/site-packages/middlewared/utils/run_in_thread.py", line 10, in run_in_thread
return await self.loop.run_in_executor(self.run_in_thread_executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 979, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/rsync.py", line 712, in run
raise CallError(
middlewared.service_exception.CallError: [EFAULT] rsync command returned 23. Check logs for further information.