Active Directory - MiddlewareError Timeout

[ABoehm]

Our FreeNAS stopped binding to Active Directory after operating for about 1.5 years without issues.

over the last few weeks we started to see failures with the SMB shares that we use as a backup target. Restarting the services on the FreeNAS would usually restore connectivity with AD and get it back online.

In the GUI I get [MiddlewareError: Active Directory start timed out after 90 seconds.]

I have tried multiple sets of AD credentials. I attempted deleting the computer account from AD and allowing it to recreate. It did recreate, but the errors persist.

I ran the following from the Shell based on some other threads and see many timeouts.

root@FreeNAS01:~ # /etc/directoryservice/ActiveDirectory/ctl start
Call timeout
False
Call timeout
Join is OK
Traceback (most recent call last):
File "/usr/local/bin/midclt", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 552, in main
with Client(uri=args.uri) as c:
File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 313, in __init__
self._ws.connect()
File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 170, in connect
rv = super(WSClient, self).connect()
File "/usr/local/lib/python3.6/site-packages/ws4py/client/__init__.py", line 215, in connect
bytes = self.sock.recv(128)
socket.timeout: timed out
Traceback (most recent call last):
File "/usr/local/bin/midclt", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 552, in main
with Client(uri=args.uri) as c:
File "/usr/local/lib/python3.6/site-packages/middlewared/client/client.py", line 316, in __init__
raise ClientException('Failed connection handshake')
middlewared.client.client.ClientException: Failed connection handshake
Call timeout
Call timeout
rm: /tmp/.ad_start: No such file or directory
root@FreeNAS01:~ # wbinfo -t
could not obtain winbind interface details: WBC_ERR_WINBIND_NOT_AVAILABLE
could not obtain winbind domain name!
checking the trust secret for domain (null) via RPC calls failed
failed to call wbcCheckTrustCredentials: WBC_ERR_WINBIND_NOT_AVAILABLE
Could not check secret

I am wondering if the issues are performance related. I keep notice long load and refresh times in the dashboard. Example, when I check the dashboard for CPU and Memory utilization, there graphs eventually load without any data. When I check for updates, it takes minutes to complete a refresh. I updated to FreeNAS-11.2-U6 in hope that some of the fixes might help, but no improvement.

Any assistance would be appreciated.

Thank you,

 

[ABoehm]

By the way, this is on a FreeNAS Mini 2.0, not a home build.

 

[ABoehm]

kernel: Failed to write core file for process smbd (error 28)
syslog-ng[2618]: I/O error occurred while writing; fd='24', error='No space left on device (28)'
syslog-ng[2618]: Suspending write operation because of an I/O error; fd='24', time_reopen='60'

The Backups ZFS Pool is at 6.69 MiB of free space. However, that shouldn't impact the FreeNAS OS and AD bind should it? I need to get SMB up so our backup utility can clean up old snapshots based on retention policies.

 

[dlavigne]

Were you able to resolve this?

 

[ABoehm]

We did resolve the issue. The timeout issues with AD were all directly a result of low disk space on the storage pool. They pool was creating performance issues that impacted the OS.

We removed directories through the shell to free up space and after letting it settle for about 13 minutes, services started restoring to normal.