Error updating 9.3 to 9.10

[Hi-Liter]

I got the following error in data/update.failed after trying to update FreeNAS-9.3-STABLE-201604120241 to 9.10 today:

Code:

FATAL ERROR - The following SQL query failed: INSERT INTO "_south_new_storage_encrypteddisk" ("encrypted_volume_id", "id", "encrypted_provider", "encrypted_disk_id") SELECT "encrypted_volume_id", "id", "encrypted_provider", "encrypted_disk_id" FROM "storage_encrypteddisk";
The error was: NOT NULL constraint failed: _south_new_storage_encrypteddisk.encrypted_disk_id
Running migrations for api:
- Nothing to migrate.
- Loading initial data for api.
Installed 0 object(s) from 0 fixture(s)
Running migrations for freeadmin:
- Nothing to migrate.
- Loading initial data for freeadmin.
Installed 0 object(s) from 0 fixture(s)
Running migrations for jails:
- Migrating forwards to 0036_templates_to_10_3_release.
> jails:0034_templates_to_10
- Migration 'jails:0034_templates_to_10' is marked for no-dry-run.
> jails:0035_templates_mtree_to_10
- Migration 'jails:0035_templates_mtree_to_10' is marked for no-dry-run.
> jails:0036_templates_to_10_3_release
- Migration 'jails:0036_templates_to_10_3_release' is marked for no-dry-run.
- Loading initial data for jails.
Installed 0 object(s) from 0 fixture(s)
Running migrations for vcp:
- Nothing to migrate.
- Loading initial data for vcp.
Installed 0 object(s) from 0 fixture(s)
Running migrations for support:
- Nothing to migrate.
- Loading initial data for support.
Installed 0 object(s) from 0 fixture(s)
Running migrations for plugins:
- Nothing to migrate.
- Loading initial data for plugins.
Installed 0 object(s) from 0 fixture(s)
Running migrations for sharing:
- Nothing to migrate.
- Loading initial data for sharing.
Installed 0 object(s) from 0 fixture(s)
Running migrations for tasks:
- Nothing to migrate.
- Loading initial data for tasks.
Installed 0 object(s) from 0 fixture(s)
Running migrations for services:
- Migrating forwards to 0192_netbiosname_split.
> services:0191_auto__add_field_cifs_cifs_srv_netbiosname_b__add_field_cifs_cifs_srv_n
> services:0192_netbiosname_split
- Migration 'services:0192_netbiosname_split' is marked for no-dry-run.
- Loading initial data for services.
Installed 0 object(s) from 0 fixture(s)
Running migrations for account:
- Nothing to migrate.
- Loading initial data for account.
Installed 0 object(s) from 0 fixture(s)
Running migrations for network:
- Migrating forwards to 0019_auto__del_field_interfaces_int_carp.
> network:0019_auto__del_field_interfaces_int_carp
- Loading initial data for network.
Installed 0 object(s) from 0 fixture(s)
Running migrations for storage:
- Migrating forwards to 0058_auto__del_field_disk_id__chg_field_disk_disk_identifier__add_unique_di.
> storage:0056_remove_disabled_encryption
- Migration 'storage:0056_remove_disabled_encryption' is marked for no-dry-run.
> storage:0057_migrate_disk_id_to_identifier
- Migration 'storage:0057_migrate_disk_id_to_identifier' is marked for no-dry-run.
! Error found during real run of migration! Aborting.

! Since you have a database that does not support running
! schema-altering statements in transactions, we have had
! to leave it in an interim state between migrations.

! You *might* be able to recover with:   (migration cannot be dry-run; cannot discover commands)
! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
Error in migration: storage:0057_migrate_disk_id_to_identifier
Traceback (most recent call last):
  File "/usr/local/www/freenasUI/manage.py", line 42, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
    ignore_ghosts = ignore_ghosts,
  File "/usr/local/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 256, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 331, in migrate_many
    result = self.migrate(migration, database)
  File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate
    result = self.run(migration, database)
  File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run
    return self.run_migration(migration, database)
  File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 84, in run_migration
    migration_function()
  File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 60, in <lambda>
    return (lambda: direction(orm))
  File "/usr/local/www/freenasUI/../freenasUI/storage/migrations/0057_migrate_disk_id_to_identifier.py", line 38, in forwards
    db.alter_column(u'storage_encrypteddisk', 'encrypted_disk_id', self.gf('django.db.models.fields.CharField')(max_length=100))
  File "/usr/local/lib/python2.7/site-packages/south/db/sqlite3.py", line 232, in alter_column
    name: self._column_sql_for_create(table_name, name, field, explicit_name),
  File "/usr/local/lib/python2.7/site-packages/south/db/generic.py", line 47, in _cache_clear
    return func(self, table, *args, **opts)
  File "/usr/local/lib/python2.7/site-packages/south/db/sqlite3.py", line 113, in _remake_table
    self._copy_data(table_name, temp_name, renames, added)
  File "/usr/local/lib/python2.7/site-packages/south/db/sqlite3.py", line 149, in _copy_data
    self.quote_name(src),
  File "/usr/local/lib/python2.7/site-packages/south/db/generic.py", line 282, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/www/freenasUI/../freenasUI/freeadmin/sqlite3_ha/base.py", line 372, in execute
    execute = Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: NOT NULL constraint failed: _south_new_storage_encrypteddisk.encrypted_disk_id

In most tabs on the GUI, I do not get anything displayed after the system rebooted following the failed update, except

{"error": true, "events": [], "message": "Error: no such column: system_advanced.adv_graphite"}

 

[dlavigne]

This will be fixed in the next SU: https://bugs.freenas.org/issues/14602. In the meantime, you can boot into a previous BE.

 

[Chris Maier]

Also hit the same bug, looking forward for the fix. Will keep an eye on https://bugs.freenas.org/issues/14602. I assume the fix will be applied on top of the 9.10 installation or do i have to do another upgrade from 9.3?

 

[dlavigne]

As far as I can tell, this error was in the config db migration script so only affects systems upgrading from 9.3 to 9.10.

 

[Chris Maier]

Ok, i see the bug fix in the 9.3 SU branch. I'll then have to update 9.3 and redo the upgrade. Thx

 

[RoboKaren]

Ran into the same bug as I tried to upgrade today. Is there a fix in the works? I had updated to the very latest 9.3 before going to 9.10 -- so if there's been a fix, it doesn't work.

 

[Alexandru Badea]

Hello,

I have the exact same problem after upgrading 9.3 to 9.10. Also, the CIFS service is not working and I cannot start it in the GUI (I get a an error). This is bad because it's on a production server and I need to at least make the files browsable until Monday. Any temporary fix for that?

Thank you.

 

[RoboKaren]

I used the zfs/GRUB rollback feature to rollback to 9.3. Thank goodness for that!

 

[Alexandru Badea]

I used the zfs/GRUB rollback feature to rollback to 9.3. Thank goodness for that!

I am a noob when it comes to FreeNAS. Can you please explain how to do that? I will be grateful.

 

[dlavigne]

Which version did you try to upgrade from and to? Have you done any downgrades?

 

[dlavigne]

Reboot and select an earlier boot entry from the boot menu (the one before the failed upgrade).

 

[RoboKaren]

When you boot, a screen like this should briefly (1 second or so) pop up. Be sure to hit the up/down key to pause the screen. Select the last version of 9.3 before you upgraded to 9.10.

 

[Alexandru Badea]

Thank you very much, I've found a previous version and got everything working. I'll try upgrading to 9.10 by booting from the latest iso rather than upgrading through the GUI.

All the best.

 

[Alexandru Badea]

An upgrade from the 9.10 iso CD busted my FreeNAS (can't access any option, just like OP mentioned). I had to do a fresh install and import the existing disks. The saved config from 9.3 didn't work and I had to redo everything. At least now it functions and the files are accessible.

 

[needo]

I had this same issue, but I was going from 9.10 to a later version of 9.10.

I know this won't work for everyone but it worked for me.

* I made the boot environment - 9.10-STABLE-201603252134 active and rebooted booting to it.
* I did not unlock my pool but went to Updates and installed last night's update. And rebooted
* That SQL error went away.
* I then ran into the error outlined in... https://bugs.freenas.org/issues/14789
* I applied the patch, rebooted again.
* Unlocked my pool and all is working correctly.

Hope this helps.

 

[Fabiano Banhi]

I had a similar issue but, unfortunately, I can't solve it using the previous infos.

I updated my system from 9.3 to 9.10. After that, I can't access the system tab. The answer is:

{"error": true, "events": [], "message": "Error: no such column: system_advanced.adv_graphite"}

Also has an update failed message: Update failed. Check /data/update.failed for further details.

Here is the file content:

FATAL ERROR - The following SQL query failed: ALTER TABLE "services_fiberchanneltotarget" RENAME TO "services_fibrechanneltotarget";
The error was: no such table: services_fiberchanneltotarget
Running migrations for api:
- Nothing to migrate.
- Loading initial data for api.
Installed 0 object(s) from 0 fixture(s)
Running migrations for freeadmin:
- Nothing to migrate.
- Loading initial data for freeadmin.
Installed 0 object(s) from 0 fixture(s)
Running migrations for support:
- Nothing to migrate.
- Loading initial data for support.
Installed 0 object(s) from 0 fixture(s)
Running migrations for vcp:
- Nothing to migrate.
- Loading initial data for vcp.
Installed 0 object(s) from 0 fixture(s)
Running migrations for directoryservice:
- Migrating forwards to 0060_auto__del_field_nt4_nt4_netbiosname.
> services:0184_auto__del_fiberchanneltotarget__add_fibrechanneltotarget
> services:0185_auto__add_field_iscsitargetextent_iscsi_target_extent_ro
> services:0186_auto__add_field_ssh_ssh_kerberosauth
> services:0187_auto__add_field_nfs_nfs_srv_16
> services:0188_auto__add_field_ssh_ssh_bindiface
> services:0189_auto__add_field_nfs_nfs_srv_v4_v3owner
> services:0190_dup_webdav
- Migration 'services:0190_dup_webdav' is marked for no-dry-run.
> services:0191_auto__add_field_cifs_cifs_srv_netbiosname_b__add_field_cifs_cifs_srv_n
> services:0192_netbiosname_split
- Migration 'services:0192_netbiosname_split' is marked for no-dry-run.
> directoryservice:0058_auto__del_field_activedirectory_ad_netbiosname_b__del_field_activedire
> directoryservice:0059_auto__del_field_ldap_ldap_netbiosname_a__del_field_ldap_ldap_netbiosna
> directoryservice:0060_auto__del_field_nt4_nt4_netbiosname
- Loading initial data for directoryservice.
Installed 0 object(s) from 0 fixture(s)
Running migrations for storage:
- Migrating forwards to 0059_encrypted_disk_id_null.
> storage:0054_migrate_follow_delete
! Error found during real run of migration! Aborting.

! Since you have a database that does not support running
! schema-altering statements in transactions, we have had
! to leave it in an interim state between migrations.

! You *might* be able to recover with:
! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
Error in migration: storage:0054_migrate_follow_delete
Traceback (most recent call last):
File "/usr/local/www/freenasUI/manage.py", line 42, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
ignore_ghosts = ignore_ghosts,
File "/usr/local/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app
success = migrator.migrate_many(target, workplan, database)
File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 256, in migrate_many
result = migrator.__class__.migrate_many(migrator, target, migrations, database)
File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 331, in migrate_many
result = self.migrate(migration, database)
File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate
result = self.run(migration, database)
File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run
return self.run_migration(migration, database)
File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 84, in run_migration
migration_function()
File "/usr/local/lib/python2.7/site-packages/south/migration/migrators.py", line 60, in <lambda>
return (lambda: direction(orm))
File "/usr/local/www/freenasUI/../freenasUI/storage/migrations/0054_migrate_follow_delete.py", line 12, in forwards
for repl in orm.Replication.objects.all():
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 96, in __iter__
self._fetch_all()
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/../freenasUI/freeadmin/sqlite3_ha/base.py", line 372, in execute
execute = Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such column: storage_replication.repl_resetonce

Any ideas to help me ?

Thanks in advance.

Best,

 

[PhilZJ81]

I just tried the update as well, and got the same error. I was seeing also an error frequently displaying on the display connected to the nas.
unfortunately, i'm not sure how to navigate to view the nice error log like you guys list here.

any help is much appreciated

 

[MDKRUSH]

Same here.