Best method to recover from the dreaded "An unhandled exception was thrown by the application." web

[Bigtexun]

So I have a FreeNAS system that was healthy, happy and stable for a long time. Last week I was doing spring cleaning in my office where the server is, and I'm sure I must have done something to upset my server and so it is getting back at me by refusing to display it's web page, only giving me "An unhandled exception was thrown by the application." to a browser.

My two 11 drive zfs r3 file systems appear to be intact, all of the data is there, so I'm not in a panic. In fact, applications running in jails, like plex, seem to be working normally.

The last couple of scrubs logged no errors.

The last thing I remember doing from the web console was applying updates, but that was a good month or two ago, so I haven't paid the server much attention at all.

Today I could not ping it, and when I plugged in a monitor it looked like it was just a bunch of normal syslog stuff, except that the video page was displaying funky, with some garbage on the screen that looks like what you sometimes see when you reboot, and you see for a split second an old video page right before the video card initializes... So I'm guessing the system glitched or just hung till I noticed it and rebooted. But that doesn't really matter, obviously something is hozed, and needs basic repair. Do I need to reinstall? Or is there some less disruptive thing I can do? I know my data will probably survive, but my wife will be annoyed if she has to redo all of the work she has done to set up plex.

What does "create backup" do from the console? Will that back up settings so I can install then do a restore? Obviously I can get to the console and jail apps, so the machine is mostly running...

Thanks for any advice you can offer...

George

 

[Bidule0hm]

Sounds like a failing boot drive. The first thing I'd try is to re-install and restore the config you hopefully backed-up in the past ;)

 

[Bigtexun]

So I download the 9.3 iso, connect it with the IPMI virtual media widget, and reinstall... Took forever, but gave me the option to preserve the configuration db.

Of course it came up complaining that I had firmware 20 and driver 16 for the drive controller, but I'm running GUI updates now to being it back to where it should be.

So I didn't swap the flash stick, it really doesn't look like that was the problem, but of course it could have been. I'm guessing a power disruption that made it through the UPS may have done something... we have had constant electrical storms for the last couple months... record rains to recover from a record drought... we have seen other issues with the power, both here and at the office, so it is plausible that it was a power-induced issue... But it *could* be from anything I suppose. I'm standing up a second FreeNAS system here over the next few weeks, so I'll have a secondary backup... Like I do at the office for my FreeNAS SAN...

Now I'm getting some errors during the update process, but when I google these I see they seem to be common from people upgrading. Of course they were having serious issues, but I'm not so far... so I'll assume they are like 75% of the other strange syslog errors, as not meaning anything bad per se...

Code:

May 14 21:09:03 freenas alert.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80ba1e5d0>' failed: no such column: storage_replication.repl_resetonce
May 14 21:09:16 freenas manage.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x8100491d0>' failed: no such column: storage_replication.repl_resetonce
May 14 21:10:04 freenas alert.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80ba1e5d0>' failed: no such column: storage_replication.repl_resetonce

So hopefully I will get through the update process without any need to do a more extensive repair...

George

 

[EM1L]

Hey George,

how did it go what happened with your error, how did you resolve it? This is what I get:

Code:

Aug 23 00:18:05 freenas alert.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80ba36610>' failed: no such column: storage_replication.repl_resetonce
Aug 23 00:19:05 freenas alert.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80ba36610>' failed: no such column: storage_replication.repl_resetonce
Aug 23 00:19:25 freenas manage.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80f7e7550>' failed: no such column: storage_replication.repl_resetonce
Aug 23 00:20:05 freenas alert.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80ba36610>' failed: no such column: storage_replication.repl_resetonce
Aug 23 00:21:05 freenas alert.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80ba36610>' failed: no such column: storage_replication.repl_resetonce
Aug 23 00:22:05 freenas alert.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80ba36610>' failed: no such column: storage_replication.repl_resetonce
Aug 23 00:23:04 freenas alert.py: [system.alert:172] Alert module '<replication_status.ReplicationStatusAlert object at 0x80ba36610>' failed: no such column: storage_replication.repl_resetonce

I am getting ththis error following fresh install on new boot flash drive. Basically I had corrupt boot volume (you can't trust even memory flash drives :))
So I installed 9.3 this time on two new memory flash drives - mirrored this time - I learned my lesson.
Booted into the fresh install and simply uploaded my old config - the db update was reported as successful and initiated another reboot. The error started to be reported shortly after this final reboot...

Not sure now what to do, but your post came on top of my searches :)

Any pinters will be apreciated thanks for your help!

Thanks,
Emil