Panic reboot during pool import

Joined
May 8, 2015
Messages
2
20200314 System Configuration:
AMD Phenom 9600
ASUS M4A785-M motherboard
8Gig DDR2-800 ECC Ram
Lsi SAS9211-8I with IT firmware
Boot drive: Mirrored 16g SanDisk USB Flash
FreeNAS 11.3-U1

20200309 configuration
Same as above except boot drive Trancend 32gb SSD
FreeNAS 11.3-RELEASE

Pool info:
4 - 1Tb HDD Seagate Pipeline installed 20200206
Pool arranged as 2 mirrored pairs total storage 2 Tb only 30% full.

On 0309 kept getting alert emails:
Code:
FreeNAS @ ceasar.local

New alerts:
* Failed to check for alert Samba4:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 381, in execute
execute = super().execute(query, params)
sqlite3.OperationalError: Error while building row_cast_map

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/alert.py", line 650, in __run_source
alerts = (await alert_source.check()) or []
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/../alert/source/samba4.py", line 27, in check
if not await self.middleware.call("datastore.query", "storage.volume"):
File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1127, in call
app=app, pipes=pipes, job_on_progress_cb=job_on_progress_cb, io_thread=True,
File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1091, in _call
return await run_method(methodobj, *args)
File "/usr/local/lib/python3.7/site-packages/middlewared/utils/run_in_thread.py", line 10, in run_in_thread
return await self.loop.run_in_executor(self.run_in_thread_executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.7/site-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/schema.py", line 964, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/datastore.py", line 192, in query
qs, options.get('extend'), options.get('extend_context'), options.get('prefix'), options.get('select'),
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/datastore.py", line 95, in __queryset_serialize
for i in qs:
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 256, in __iter__
self._fetch_all()
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1087, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 54, in __iter__
results = compiler.execute_sql()
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.7/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 381, in execute
execute = super().execute(query, params)
django.db.utils.OperationalError: Error while building row_cast_map


Current alerts:
* Scrub of pool 'freenas-boot' finished.
* A system update is available. Go to System -> Update to download and apply the update.
* Failed to check for alert Samba4:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 381, in execute
execute = super().execute(query, params)
sqlite3.OperationalError: Error while building row_cast_map

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/alert.py", line 650, in __run_source
alerts = (await alert_source.check()) or []
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/../alert/source/samba4.py", line 27, in check
if not await self.middleware.call("datastore.query", "storage.volume"):
File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1127, in call
app=app, pipes=pipes, job_on_progress_cb=job_on_progress_cb, io_thread=True,
File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1091, in _call
return await run_method(methodobj, *args)
File "/usr/local/lib/python3.7/site-packages/middlewared/utils/run_in_thread.py", line 10, in run_in_thread
return await self.loop.run_in_executor(self.run_in_thread_executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.7/site-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/schema.py", line 964, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/datastore.py", line 192, in query
qs, options.get('extend'), options.get('extend_context'), options.get('prefix'), options.get('select'),
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/datastore.py", line 95, in __queryset_serialize
for i in qs:
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 256, in __iter__
self._fetch_all()
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1087, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 54, in __iter__
results = compiler.execute_sql()
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.7/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 381, in execute
execute = super().execute(query, params)
django.db.utils.OperationalError: Error while building row_cast_map


Then followed by the following email alert:
Code:
FreeNAS @ ceasar.local

The following alert has been cleared:
* Failed to check for alert Samba4:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 381, in execute
execute = super().execute(query, params)
sqlite3.OperationalError: Error while building row_cast_map

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/alert.py", line 650, in __run_source
alerts = (await alert_source.check()) or []
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/../alert/source/samba4.py", line 27, in check
if not await self.middleware.call("datastore.query", "storage.volume"):
File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1127, in call
app=app, pipes=pipes, job_on_progress_cb=job_on_progress_cb, io_thread=True,
File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1091, in _call
return await run_method(methodobj, *args)
File "/usr/local/lib/python3.7/site-packages/middlewared/utils/run_in_thread.py", line 10, in run_in_thread
return await self.loop.run_in_executor(self.run_in_thread_executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.7/site-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/schema.py", line 964, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/datastore.py", line 192, in query
qs, options.get('extend'), options.get('extend_context'), options.get('prefix'), options.get('select'),
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/datastore.py", line 95, in __queryset_serialize
for i in qs:
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 256, in __iter__
self._fetch_all()
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1087, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 54, in __iter__
results = compiler.execute_sql()
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.7/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 381, in execute
execute = super().execute(query, params)
django.db.utils.OperationalError: Error while building row_cast_map


Current alerts:
* Scrub of pool 'freenas-boot' finished.
* A system update is available. Go to System -> Update to download and apply the update.



I would get an email every 5 minutes. My email was blowing up. I connected to the web ui and shut the server down. After doing some research I thought maybe my SSD was failing. So pulled the ssd and installed the flash drives and did a clean install of FreeNAS.

Now when importing the pool the following messages appear on screen
20200309_150558.jpg

The system continually reboots repeatedly.

I tried mounting the pool using: zpool import -o readonly=on Sch_pool2 in the shell terminal in the UI.

FreeNAS does panic reboot.

Just realized I may have had a typoed that command. I'll get back to it.

I'll post this anyways in hopes that there is something I am missing here or can offer guidance.

This hardware except for the ssd, new usb flash, and hdd has been running for 8 years with 6 500gb hdds in a z3.
 
Top