Stuck at Beginning ZFS volume imports

[Javierus]

(Edited to add the system specifications and other needed data)

I've upgraded a FreeNAS from 9.2 to 9.3, and after rebooting it gets stuck at

Code:

Beginning ZFS volume imports

     Importing 152162520229915532

txg 10493740 open pool version 5000; software version 5000/5; uts 9.3-RELEASE-p
txg 10493742 import pool version 5000; software version 5000/5; uts 9.3-RELEASE
slow spa_sync: started 68789 seconds ago, calls 13560

The seconds count increases as time passes, and the calls count also does it.

if I press Ctrl+T I get something like:

Code:

load: 0.00 cmd: zfs 734 [] 68792.32r 0.00u 0.00s 0% 3008k

The system has 6x3TB disks in RAIDZ2, and had 8GB RAM. After reading some on the forums, powered off the system, removed the RAM, and inserted 16GB. No luck.

If I use a new USB stick with a fresh FreeNAS 9.3 install, from the GUI I can start the volume import, but the step 2 takes forever and does not seem to end.

After starting the import, if I open a shell from putty or the physical console, a zpool status shows all normal: the state is ONLINE for the pool, the VDEV and all the disks. Also says "No known data errors"

From the shell, I can navigate to the tank, the datasets, the folders, and see the files.

I can enter the .zfs folder on the tank, but if I try to list the contents of the snapshot folder, the console gets stuck waiting for the command to finish.

And we just discovered we need the snapshots, cause something was lost at some point, but should be on the snapshots.

Any help on recovering this would be appreciated.


System:
Motherboard: Gigabyte GA-Z77M-D3H
NIC: Integrated on motherboard, Atheros chip AR8151 v2.0
RAM: 16GB, non-ECC
CPU: i7-3770 @ 3,4GHz
Disks: 6 x 3TB, plugged to the motherboard
At start the disks where Seagate, but as time passed I changed some of them to WD Red
* 2x Seagate 3TB SATA (ST3000DM001)
* 4x Western Digital NAS 3TB: WDC WD30EFRX

ZFS:
Filesystem version 5
ZFS storage pool version: feature support (5000)
The pool was vreated with 512b size, cause when I created it the GUI created it with that size. Since a given FreeNAS upgrade (don't remember wich on), I use the Ashift tunables needed to make it work.

Boot:
USB 8GB. Kingston data traveler 3.0

CLI:
I tried zpool Import and zpool Export
Now I do know is a mistake, after reading Cyberjock newbie guide, but what is done, is done.

Upgrade:
Standard from the GUI

Thanks again for your time and patience :)

 

[DrKK]

Per the forum rules, can we please be told exactly what the system is, CPU, motherboard, and other sundry information that may affect the answer? How did you perform the upgrade? A standard GUI upgrade? Have you done anything at all to the pool from the CLI?

Thanks.

 

[Javierus]

(posted the system specs; now moved them to first post)

 

[Robert Trevellyan]

From the shell, I can navigate to the tank, the datasets, the folders, and see the files.

Then your data are probably safe.

if I try to list the contents of the snapshot folder, the console gets stuck waiting for the command to finish.

How long did you wait? Maybe there are so many snapshots that it takes a really long time to list them.

Your system is not an ideal platform for FreeNAS, but at least it isn't under-resourced.

Have you tried installing the version you were running before on your original USB drive? There are versions that require a step by step upgrade path rather than a direct upgrade to 9.3.

 

[Javierus]

Then your data are probably safe.

How long did you wait? Maybe there are so many snapshots that it takes a really long time to list them.

Your system is not an ideal platform for FreeNAS, but at least it isn't under-resourced.

Have you tried installing the version you were running before on your original USB drive? There are versions that require a step by step upgrade path rather than a direct upgrade to 9.3.

I'm guessing that the snapshot number might be around a couple thousands, and waited around 3 or 4 hours. Then the lights went down, and on restart didn't wanted to try listing snapshots again, just in case I would slow down the system even more.

 

[Robert Trevellyan]

Then the lights went down

What does this mean?

Have you tried installing the version you were running before on your original USB drive?

 

[Javierus]

What does this mean?

A power outage, literally.
About installing the version I was running (8.2), Would it work? The zpool get version shows me "-".
Btw, "zpool list" hangs also.
More info: looking through the GUI Reporting tool, it shows zero disk activity, and around 5% CPU usage.

 

[Robert Trevellyan]

OK, your first post says you upgraded from 9.2 to 9.3, but now you seem to be saying you went from 8.2 to 9.3. If so, it's no surprise that you ran into trouble. You have to go stepwise from 8.x to 9.3. Even some 9.x branches require stepwise upgrade to 9.3.

About installing the version I was running (8.2), Would it work?

I would expect installing the version you were running immediately before the upgrade to work.

 

[Javierus]

OK, your first post says you upgraded from 9.2 to 9.3, but now you seem to be saying you went from 8.2 to 9.3. If so, it's no surprise that you ran into trouble. You have to go stepwise from 8.x to 9.3. Even some 9.x branches require stepwise upgrade to 9.3.

I would expect installing the version you were running immediately before the upgrade to work.

It was 9.2; the 8 was a typo. Besides the biz tuff, I had also a jail with PLEX.

 

[Robert Trevellyan]

It was 9.2; the 8 was a typo. Besides the biz tuff, I had also a jail with PLEX.

OK, so try whatever version it was before.

 

[cyberjock]

I realize I'm a couple of weeks late.. but did you do a RAM test? Your hardware is not recommended, so I can't say I'm too surprised that you are having problems. Using poorly supported hardware often causes unintended consequences.