Code:
[2021/02/11 07:30:05] (INFO) middlewared.__init__():792 - Starting TrueNAS-12.0-U2 middleware
[2021/02/11 08:30:14] (DEBUG) middlewared.setup():1639 - Timezone set to Europe/Berlin
[2021/02/11 08:30:17] (WARNING) middlewared._loop_monitor_thread():1415 - Task seems blocked:
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.8/site-packages/pysnmp/carrier/asyncore/dgram/base.py", line 10, in <module>
from pysnmp.carrier.asyncore.base import AbstractSocketTransport
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 779, in exec_module
File "<frozen importlib._bootstrap_external>", line 874, in get_code
File "<frozen importlib._bootstrap_external>", line 973, in get_data
[2021/02/11 08:30:19] (WARNING) middlewared._loop_monitor_thread():1415 - Task seems blocked:
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.8/site-packages/Cryptodome/Hash/SHA1.py", line 29, in <module>
_raw_sha1_lib = load_pycryptodome_raw_lib("Cryptodome.Hash._SHA1",
File "/usr/local/lib/python3.8/site-packages/Cryptodome/Util/_raw_api.py", line 296, in load_pycryptodome_raw_lib
return load_lib(pycryptodome_filename(dir_comps, filename),
File "/usr/local/lib/python3.8/site-packages/Cryptodome/Util/_raw_api.py", line 102, in load_lib
lib = ffi.dlopen(name)
File "/usr/local/lib/python3.8/site-packages/cffi/api.py", line 150, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/usr/local/lib/python3.8/site-packages/cffi/api.py", line 832, in _make_ffi_library
backendlib = _load_backend_lib(backend, libname, flags)
File "/usr/local/lib/python3.8/site-packages/cffi/api.py", line 818, in _load_backend_lib
path = ctypes.util.find_library(name)
File "/usr/local/lib/python3.8/ctypes/util.py", line 206, in find_library
data = proc.stdout.read()
[2021/02/11 08:30:21] (WARNING) middlewared._loop_monitor_thread():1415 - Task seems blocked:
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.8/site-packages/Cryptodome/Cipher/AES.py", line 66, in <module>
_raw_aes_lib = load_pycryptodome_raw_lib("Cryptodome.Cipher._raw_aes",
File "/usr/local/lib/python3.8/site-packages/Cryptodome/Util/_raw_api.py", line 296, in load_pycryptodome_raw_lib
return load_lib(pycryptodome_filename(dir_comps, filename),
File "/usr/local/lib/python3.8/site-packages/Cryptodome/Util/_raw_api.py", line 102, in load_lib
lib = ffi.dlopen(name)
File "/usr/local/lib/python3.8/site-packages/cffi/api.py", line 150, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/usr/local/lib/python3.8/site-packages/cffi/api.py", line 832, in _make_ffi_library
backendlib = _load_backend_lib(backend, libname, flags)
File "/usr/local/lib/python3.8/site-packages/cffi/api.py", line 818, in _load_backend_lib
path = ctypes.util.find_library(name)
File "/usr/local/lib/python3.8/ctypes/util.py", line 199, in find_library
proc = subprocess.Popen(('/sbin/ldconfig', '-r'),
File "/usr/local/lib/python3.8/subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/lib/python3.8/subprocess.py", line 1658, in _execute_child
part = os.read(errpipe_read, 50000)
[2021/02/11 08:30:25] (DEBUG) middlewared.setup():2835 - Certificate setup for System complete
[2021/02/11 08:30:30] (INFO) middlewared.devd_listen():58 - devd connection established
[2021/02/11 08:30:31] (DEBUG) middlewared.__plugins_setup():907 - All plugins loaded
[2021/02/11 08:30:31] (DEBUG) middlewared.__initialize():1499 - Accepting connections
[2021/02/11 08:30:32] (INFO) DiskService.sync_all():94 - Found disks: {'ada4': {'name': 'ada4', 'ident': 'FF73074A0F0800757474', 'lunid': None, 'serial': 'FF73074A0F0800757474'}, 'ada3': {'name': 'ada3', 'ident': 'WD-WCC4J3304069', 'lunid': '50014ee25f4d194e', 'serial': 'WD-WCC4J3304069'}, 'ada2': {'name': 'ada2', 'ident': 'WD-WCC4J3324157', 'lunid': '50014ee25f4d5063', 'serial': 'WD-WCC4J3324157'}, 'ada1': {'name': 'ada1', 'ident': 'WD-WCC4J3UPHZXP', 'lunid': '50014ee2b712a30f', 'serial': 'WD-WCC4J3UPHZXP'}, 'ada0': {'name': 'ada0', 'ident': 'WD-WCC4J5YEDYEJ', 'lunid': '50014ee20c678774', 'serial': 'WD-WCC4J5YEDYEJ'}}
[2021/02/11 08:30:38] (DEBUG) EtcService.generate():378 - No new changes for /etc/krb5.conf
[2021/02/11 08:30:39] (WARNING) builtins.add_bind_interfaces():116 - IP address [10.19.42.40] is no longer in use and should be removed from SMB configuration.
[2021/02/11 08:30:40] (DEBUG) EtcService.generate():378 - No new changes for /etc/pam.d/sshd
[2021/02/11 08:30:40] (DEBUG) EtcService.generate():329 - mako:local/users.oath file removed.
[2021/02/11 08:30:40] (DEBUG) EtcService.generate():329 - mako:local/openvpn/server/openvpn_server.conf file removed.
[2021/02/11 08:30:40] (DEBUG) EtcService.generate():329 - mako:local/openvpn/client/openvpn_client.conf file removed.
[2021/02/11 08:30:40] (DEBUG) EtcService.generate():329 - mako:wireguard/wg0.conf file removed.
[2021/02/11 08:30:44] (DEBUG) middlewared.dumpdev_configure():24 - Configured "/dev/ada3p1" device for crash dumps.
[2021/02/11 08:30:46] (WARNING) builtins.add_bind_interfaces():116 - IP address [10.19.42.40] is no longer in use and should be removed from SMB configuration.
[2021/02/11 08:30:46] (DEBUG) EtcService.generate():378 - No new changes for /etc/local/smb4.conf
[2021/02/11 08:30:46] (DEBUG) EtcService.generate():378 - No new changes for /etc/local/smbusername.map
[2021/02/11 08:30:47] (DEBUG) EtcService.generate():378 - No new changes for /etc/group
[2021/02/11 08:30:47] (DEBUG) EtcService.generate():378 - No new changes for /etc/security/pam_winbind.conf
[2021/02/11 08:30:48] (DEBUG) EtcService.generate():378 - No new changes for /etc/master.passwd
[2021/02/11 08:30:49] (DEBUG) EtcService.generate():378 - No new changes for /etc/local/proftpd.conf
[2021/02/11 08:30:49] (DEBUG) EtcService.generate():378 - No new changes for /etc/local/apache24/httpd.conf
[2021/02/11 08:30:50] (DEBUG) EtcService.generate():378 - No new changes for /etc/local/apache24/Includes/webdav.conf
[2021/02/11 08:30:50] (DEBUG) EtcService.generate():378 - No new changes for /etc/local/smb4_share.conf
[2021/02/11 08:30:50] (DEBUG) middlewared.interface_pre_sync_hook():1525 - No HA hardware detected, skipping interfaces setup.
[2021/02/11 08:30:50] (INFO) InterfaceService.lag_setup():13 - Setting up lagg0
[2021/02/11 08:30:50] (INFO) InterfaceService.lag_setup():35 - lagg0: changing protocol to 4
[2021/02/11 08:30:51] (DEBUG) InterfaceService.configure():159 - lagg0: adding 10.19.42.40/255.255.252.0
[2021/02/11 08:30:51] (INFO) InterfaceService.sync():1839 - Interfaces in database: lagg0
[2021/02/11 08:30:51] (DEBUG) EtcService.generate():378 - No new changes for /etc/dhclient.conf
[2021/02/11 08:30:53] (DEBUG) EtcService.generate():378 - No new changes for /etc/nsswitch.conf
[2021/02/11 08:30:53] (DEBUG) EtcService.generate():378 - No new changes for /etc/local/nslcd.conf
[2021/02/11 08:30:54] (DEBUG) EtcService.generate():378 - No new changes for /etc/krb5.conf
[2021/02/11 08:30:57] (DEBUG) EtcService.generate():378 - No new changes for /etc/krb5.conf
[2021/02/11 08:30:57] (DEBUG) EtcService.generate():378 - No new changes for /etc/krb5.conf
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task alert.flush_alerts to run every 3600 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task alert.process_alerts to run every 60 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task certificate.renew_certs to run every 86400 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task certificateauthority.crl_generation to run every 86400 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task kerberos.keytab.check_updated_keytab to run every 3600 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task kmip.sync_keys to run every 86400 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task mail.send_mail_queue to run every 600 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task plugin.periodic_plugin_update to run every 86400 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task plugin.retrieve_versions_for_repos to run every 86400 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task pool.dataset.sync_db_keys to run every 86400 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task smb.sharesec.check_share_info_tdb to run every 3600 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task truecommand.health_check to run every 1800 seconds
[2021/02/11 08:30:58] (DEBUG) middlewared._setup_periodic_tasks():920 - Setting up periodic task zettarepl.flush_state to run every 3600 seconds
[2021/02/11 08:30:58] (DEBUG) JailService.start_on_boot():1624 - Starting jails on boot: PENDING
[2021/02/11 08:30:59] (DEBUG) JailService.start_on_boot():1626 - Starting jails on boot: SUCCESS
[2021/02/11 08:30:59] (ERROR) iocage.callback():67 - Failed to update git repository: fatal: unable to access 'https://github.com/ix-plugin-hub/iocage-plugin-index.git/': Could not resolve host: github.com
[2021/02/11 08:30:59] (ERROR) iocage.callback():67 - Not cloning https://github.com/ix-plugin-hub/iocage-plugin-index.gitas git-pull failed due to network issues.
[2021/02/11 08:31:00] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): pkg.freebsd.org:80
[2021/02/11 08:31:00] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): pkg.cdn.trueos.org:80
[2021/02/11 08:31:00] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): pkg.freebsd.org:80
[2021/02/11 08:31:00] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): pkg.freebsd.org:80
[2021/02/11 08:31:00] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): pkg.cdn.trueos.org:80
[2021/02/11 08:31:15] (ERROR) iocage.callback():67 - Failed to update git repository: fatal: unable to access 'https://github.com/freenas/iocage-ix-plugins.git/': Could not resolve host: github.com
[2021/02/11 08:31:15] (ERROR) iocage.callback():67 - Not cloning https://github.com/freenas/iocage-ix-plugins.gitas git-pull failed due to network issues.
[2021/02/11 08:31:15] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): pkg.freebsd.org:80
[2021/02/11 08:31:15] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): pkg.freebsd.org:80
[2021/02/11 08:31:15] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): pkg.cdn.trueos.org:80
[2021/02/11 08:31:15] (DEBUG) urllib3.connectionpool._new_conn():221 - Starting new HTTP connection (1): download.truenas.com:80
[2021/02/11 08:31:15] (DEBUG) urllib3.connectionpool._make_request():428 - http://pkg.cdn.trueos.org:80 "GET /iocage/unstable/packagesite.txz HTTP/1.1" 200 302704
[2021/02/11 08:31:15] (DEBUG) urllib3.connectionpool._make_request():428 - http://pkg.freebsd.org:80 "GET /FreeBSD:12:amd64/latest/packagesite.txz HTTP/1.1" 200 6412488
[2021/02/11 08:31:15] (DEBUG) urllib3.connectionpool._make_request():428 - http://pkg.freebsd.org:80 "GET /FreeBSD:12:amd64/latest/packagesite.txz HTTP/1.1" 200 6412488
[2021/02/11 08:31:15] (DEBUG) urllib3.connectionpool._make_request():428 - http://download.truenas.com:80 "GET /plugins/asigra/stable/11.3-RELEASE/packagesite.txz HTTP/1.1" 200 16472
[2021/02/11 08:59:16] (DEBUG) urllib3.connectionpool._new_conn():955 - Starting new HTTPS connection (1): update-master.ixsystems.com:443
[2021/02/11 08:59:22] (WARNING) UpdateService.get_trains_data():102 - Failed to retrieve trains redirection
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 156, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 665, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 334, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 161, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.VerifiedHTTPSConnection object at 0x81eba5cd0>, 'Connection to update-master.ixsystems.com timed out. (connect timeout=5)')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 719, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='update-master.ixsystems.com', port=443): Max retries exceeded with url: /TrueNAS/trains_redir.json (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x81eba5cd0>, 'Connection to update-master.ixsystems.com timed out. (connect timeout=5)'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/update_/trains_freebsd.py", line 100, in get_trains_data
redir_trains = self._get_redir_trains()
File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/update_/trains_freebsd.py", line 141, in _get_redir_trains
r = requests.get(
File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 504, in send
raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='update-master.ixsystems.com', port=443): Max retries exceeded with url: /TrueNAS/trains_redir.json (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x81eba5cd0>, 'Connection to update-master.ixsystems.com timed out. (connect timeout=5)'))
[2021/02/11 08:59:22] (DEBUG) freenasOS.Configuration.TryGetNetworkFile():606 - TryGetNetworkFile(['https://update.ixsystems.com/TrueNAS/trains.txt', 'https://update-master.ixsystems.com/TrueNAS/trains.txt'])
[2021/02/11 08:59:22] (DEBUG) urllib3.connectionpool._new_conn():955 - Starting new HTTPS connection (1): update.ixsystems.com:443
[2021/02/11 08:59:23] (DEBUG) urllib3.connectionpool._make_request():428 - https://update.ixsystems.com:443 "GET /TrueNAS/trains.txt HTTP/1.1" 301 169
[2021/02/11 08:59:23] (DEBUG) urllib3.connectionpool._new_conn():955 - Starting new HTTPS connection (1): update.freenas.org:443
[2021/02/11 08:59:23] (DEBUG) urllib3.connectionpool._make_request():428 - https://update.freenas.org:443 "GET /TrueNAS/trains.txt HTTP/1.1" 200 377
[2021/02/11 08:59:23] (DEBUG) freenasOS.Configuration.TryGetNetworkFile():745 - TryGetNetworkFile(['https://update.ixsystems.com/TrueNAS/trains.txt', 'https://update-master.ixsystems.com/TrueNAS/trains.txt']): Read 377 bytes total
[2021/02/11 08:59:23] (DEBUG) freenasOS.Configuration.TryGetNetworkFile():606 - TryGetNetworkFile(['https://update-master.ixsystems.com/TrueNAS/TrueNAS-12.0-STABLE/LATEST'])
[2021/02/11 08:59:23] (DEBUG) urllib3.connectionpool._new_conn():955 - Starting new HTTPS connection (1): update-master.ixsystems.com:443
[2021/02/11 08:59:23] (DEBUG) urllib3.connectionpool._make_request():428 - https://update-master.ixsystems.com:443 "GET /TrueNAS/TrueNAS-12.0-STABLE/LATEST HTTP/1.1" 200 1602
[2021/02/11 08:59:23] (DEBUG) freenasOS.Configuration.TryGetNetworkFile():745 - TryGetNetworkFile(['https://update-master.ixsystems.com/TrueNAS/TrueNAS-12.0-STABLE/LATEST']): Read 1602 bytes total
[2021/02/11 08:59:23] (DEBUG) freenasOS.Configuration.TryGetNetworkFile():606 - TryGetNetworkFile(['https://update-master.ixsystems.com/updates/ix_crl.pem'])
[2021/02/11 08:59:23] (DEBUG) urllib3.connectionpool._new_conn():955 - Starting new HTTPS connection (1): update-master.ixsystems.com:443
[2021/02/11 08:59:24] (DEBUG) urllib3.connectionpool._make_request():428 - https://update-master.ixsystems.com:443 "GET /updates/ix_crl.pem HTTP/1.1" 200 1028
[2021/02/11 08:59:24] (DEBUG) freenasOS.Configuration.CheckFreeSpace():77 - CheckFreeSpace(path=/tmp/tmp8khzpzq0.pem, pool=None, required=1028)
[2021/02/11 08:59:24] (DEBUG) freenasOS.Configuration.TryGetNetworkFile():745 - TryGetNetworkFile(['https://update-master.ixsystems.com/updates/ix_crl.pem']): Read 1028 bytes total
[2021/02/11 08:59:24] (DEBUG) freenasOS.Configuration.TryGetNetworkFile():606 - TryGetNetworkFile(['https://update.ixsystems.com/TrueNAS/Validators/ValidateUpdate-mg3ef525.txt', 'https://update-master.ixsystems.com/TrueNAS/Validators/ValidateUpdate-mg3ef525.txt'])
[2021/02/11 08:59:24] (DEBUG) urllib3.connectionpool._new_conn():955 - Starting new HTTPS connection (1): update.ixsystems.com:443
[2021/02/11 08:59:25] (DEBUG) urllib3.connectionpool._make_request():428 - https://update.ixsystems.com:443 "GET /TrueNAS/Validators/ValidateUpdate-mg3ef525.txt HTTP/1.1" 301 169
[2021/02/11 08:59:25] (DEBUG) urllib3.connectionpool._new_conn():955 - Starting new HTTPS connection (1): update.freenas.org:443
[2021/02/11 08:59:25] (DEBUG) urllib3.connectionpool._make_request():428 - https://update.freenas.org:443 "GET /TrueNAS/Validators/ValidateUpdate-mg3ef525.txt HTTP/1.1" 200 185
[2021/02/11 08:59:25] (DEBUG) freenasOS.Configuration.CheckFreeSpace():77 - CheckFreeSpace(path=/tmp/tmp2ihmflkk, pool=None, required=185)
[2021/02/11 08:59:25] (DEBUG) freenasOS.Configuration.TryGetNetworkFile():745 - TryGetNetworkFile(['https://update.ixsystems.com/TrueNAS/Validators/ValidateUpdate-mg3ef525.txt', 'https://update-master.ixsystems.com/TrueNAS/Validators/ValidateUpdate-mg3ef525.txt']): Read 185 bytes total
[2021/02/11 09:01:48] (DEBUG) ActiveDirectoryService.get_n_working_servers():169 - Request for [2] of server type [DOMAINCONTROLLER] returned: [{'host': 'dc2.corp.DOMAIN.com', 'port': 389}, {'host': 'dc1.corp.DOMAIN.com', 'port': 389}]
[2021/02/11 09:01:48] (WARNING) ActiveDirectoryService.conn_check():845 - Failed to get DC info from winbindd:
[2021/02/11 09:01:48] (WARNING) ActiveDirectoryService._extend_creds():201 - Failed to initialize secrets for domain [CORP.DOMAIN.COM]. attempting to use credentials from config file.
The option to leave the domain is not available, probably due to the failed join attempt. Is there a way to do it from the shell?