HTTPDBindAddress errors - likely something else under the hood

jacob42

jacob42

Cadet
Joined
Feb 21, 2015
Messages
2
I've been ignoring a problem I've had for some time, mainly because it doesn't seem to affect anything. The problem is now starting to bug me, but on investigation I can't figure out where to start. I've searched for some lines but nothing definitive / people not updating their tickets. Also almost none of the other problems reported relate to HTTPDBindAddress, they seem to be more under-the-hood python bugs which I don't have access to, or don't really want to touch core files.

I receive 2 emails about twice a day. First email:

Code:
New alerts:
* Unable to run alert source 'HTTPDBindAddress'
Traceback (most recent call last):
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 358, in __run_source
   alerts = (await alert_source.check()) or []
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/source/httpd_bindaddr.py", line 9, in check
   settings = await self.middleware.call("datastore.query", "system.settings", None, {"get": True})
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1047, in call
   return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 997, in _call
   return await run_method(methodobj, *args)
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 943, in run_in_thread
   return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
 File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
   result = self.fn(*self.args, **self.kwargs)
 File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
   return f(*args, **kwargs)
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 171, in query
   qs, extend=options.get('extend'), field_prefix=options.get('prefix')
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 88, in __queryset_serialize
   for i in qs:
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __iter__
   self._fetch_all()
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1087, in _fetch_all
   self._result_cache = list(self.iterator())
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 54, in __iter__
   results = compiler.execute_sql()
 File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
   cursor.execute(sql, params)
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
   six.reraise(dj_exc_type, dj_exc_value, traceback)
 File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
   raise value.with_traceback(tb)
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
django.db.utils.OperationalError: database is locked


Alerts:
* Unable to run alert source 'HTTPDBindAddress'
Traceback (most recent call last):
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 358, in __run_source
   alerts = (await alert_source.check()) or []
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/source/httpd_bindaddr.py", line 9, in check
   settings = await self.middleware.call("datastore.query", "system.settings", None, {"get": True})
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1047, in call
   return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 997, in _call
   return await run_method(methodobj, *args)
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 943, in run_in_thread
   return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
 File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
   result = self.fn(*self.args, **self.kwargs)
 File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
   return f(*args, **kwargs)
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 171, in query
   qs, extend=options.get('extend'), field_prefix=options.get('prefix')
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 88, in __queryset_serialize
   for i in qs:
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __iter__
   self._fetch_all()
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1087, in _fetch_all
   self._result_cache = list(self.iterator())
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 54, in __iter__
   results = compiler.execute_sql()
 File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
   cursor.execute(sql, params)
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
   six.reraise(dj_exc_type, dj_exc_value, traceback)
 File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
   raise value.with_traceback(tb)
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
django.db.utils.OperationalError: database is locked



And about 20 seconds later, the second email:

Code:
Gone alerts:
* Unable to run alert source 'HTTPDBindAddress'
Traceback (most recent call last):
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 358, in __run_source
   alerts = (await alert_source.check()) or []
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/source/httpd_bindaddr.py", line 9, in check
   settings = await self.middleware.call("datastore.query", "system.settings", None, {"get": True})
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1047, in call
   return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 997, in _call
   return await run_method(methodobj, *args)
 File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 943, in run_in_thread
   return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
 File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
   result = self.fn(*self.args, **self.kwargs)
 File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
   return f(*args, **kwargs)
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 171, in query
   qs, extend=options.get('extend'), field_prefix=options.get('prefix')
 File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 88, in __queryset_serialize
   for i in qs:
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __iter__
   self._fetch_all()
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1087, in _fetch_all
   self._result_cache = list(self.iterator())
 File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 54, in __iter__
   results = compiler.execute_sql()
 File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
   cursor.execute(sql, params)
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
   six.reraise(dj_exc_type, dj_exc_value, traceback)
 File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
   raise value.with_traceback(tb)
 File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
 File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
django.db.utils.OperationalError: database is locked


Alerts:


Machine is a light duty Plex box:
FreeNAS-11.2-RC1
Core i3-7100
16gb RAM

Any thoughts? Any recommendations on where to start scoping this problem?
 
D

dlavigne

Guest
Code:
sqlite3.OperationalError: database is locked


The usual culprit is that the boot device is dying. Does a fresh install of the same version to a different stick resolve the issue? If it does, you can restore your config to the new stick.
 
You must log in or register to reply here.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "HTTPDBindAddress errors - likely something else under the hood"

Similar threads

hiro5id
My FreeNAS-11.2-RELEASE-U1 appears to be working fine. However i'm getting these errors emailed to me.
Replies
3
Views
2K
hiro5id
hiro5id
Z
smartd Issue
Replies
8
Views
3K
dlavigne
D
J
New Alerts and gone alerts
Replies
6
Views
3K
mistermanko
mistermanko
bigdadda06
Unable to run alert source 'HasUpdate‘
Replies
6
Views
3K
Inxsible
I
O
freenas.local: Unscheduled system reboot
Replies
3
Views
3K
Chris Moore
Chris Moore
Top