Need help recovering from crashed root volume

Status
Not open for further replies.

rhing

Cadet
Joined
Sep 18, 2013
Messages
2
Greetings all.

Here's a description of where I'm at:
1) Discovered I couldn't access any of my FreeNAS drives (Two RAIDZ2 volumes)
2) Found a kernel panic message at the FreeNAS terminal
3) Restarted the system and no operating system was found
4) Discovered the USB flash drive completely dead
5) Downloaded and installed FreeNAS 9.1.1 to another working USB drive
6) Used "Auto-Import" from the GUI to try to recover previous ZFS volumes
7) "Auto-Import" was able to find the first volume but not the second

I've been reading the forums and trying to decipher what I need to do but I'm still a bit lost and I'm not thinking straight right now because I'm in a bit of a panic state.

From what I've seen those offering support have asked for this information:

Code:
[root@freenas] /mnt# camcontrol devlist
<WDC WD20EARS-22MVWB0 51.0AB51>    at scbus0 target 0 lun 0 (ada0,pass0)
<ST3000DM001-1CH166 CC24>          at scbus1 target 0 lun 0 (ada1,pass1)
<WDC WD20EARS-22MVWB0 51.0AB51>    at scbus2 target 0 lun 0 (ada2,pass2)
<WDC WD20EARS-00MVWB0 50.0AB50>    at scbus3 target 0 lun 0 (ada3,pass3)
<Marvell Console 1.01>            at scbus7 target 0 lun 0 (pass4)
<ST3000DM001-1CH166 CC24>          at scbus8 target 0 lun 0 (ada4,pass5)
<ST3000DM001-1CH166 CC26>          at scbus10 target 0 lun 0 (ada5,pass6)
<Marvell Console 1.01>            at scbus15 target 0 lun 0 (pass7)
<SAMSUNG HD204UI 1AQ10001>        at scbus16 target 0 lun 0 (ada6,pass8)
<ST3000DM001-1CH166 CC24>          at scbus17 target 0 lun 0 (ada7,pass9)
<ST3000DM001-1CH166 CC24>          at scbus18 target 0 lun 0 (ada8,pass10)
<ST3000DM001-1CH166 CC26>          at scbus19 target 0 lun 0 (ada9,pass11)
<WDC WD20EARS-00J2GB0 80.00A80>    at scbus20 target 0 lun 0 (ada10,pass12)
<WDC WD10EALX-009BA0 15.01H15>    at scbus22 target 0 lun 0 (ada11,pass13)
< Patriot Memory PMAP>            at scbus30 target 0 lun 0 (pass14,da0)


Code:
[root@freenas] /mnt# gpart show
=>        34  3907029101  ada0  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)
 
=>        34  5860533101  ada1  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>        34  3907029101  ada2  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)
 
=>        34  3907029101  ada3  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)
 
=>        34  5860533101  ada4  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>        34  5860533101  ada5  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>        34  3907029101  ada6  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)
 
=>        34  5860533101  ada7  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>        34  5860533101  ada8  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>        34  5860533101  ada9  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>        34  3907029101  ada10  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304      1  freebsd-swap  (2.0G)
    4194432  3902834703      2  freebsd-zfs  (1.8T)
 
=>        34  1953525101  ada11  GPT  (931G)
          34          94        - free -  (47k)
        128    4194304      1  freebsd-swap  (2.0G)
    4194432  1949330703      2  freebsd-ufs  (929G)
 
=>      63  15515585  da0  MBR  (7.4G)
        63  1930257    1  freebsd  [active]  (942M)
  1930320        63      - free -  (31k)
  1930383  1930257    2  freebsd  (942M)
  3860640      3024    3  freebsd  (1.5M)
  3863664    41328    4  freebsd  (20M)
  3904992  11610656      - free -  (5.5G)
 
=>      0  1930257  da0s1  BSD  (942M)
        0      16        - free -  (8.0k)
      16  1930241      1  !0  (942M)


Code:
[root@freenas] /mnt# glabel status
                                      Name  Status  Components
gptid/b011fd83-c0a2-11e0-897e-00e018e48620    N/A  ada0p2
gptid/862bdfcf-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada1p2
gptid/ae94e17b-c0a2-11e0-897e-00e018e48620    N/A  ada2p2
gptid/af78266b-c0a2-11e0-897e-00e018e48620    N/A  ada3p2
gptid/879db021-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada4p2
gptid/880590f8-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada5p1
gptid/881d609c-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada5p2
                            gpt/swap-ada4    N/A  ada6p1
gptid/b082960b-c0a2-11e0-897e-00e018e48620    N/A  ada6p1
gptid/b08e3a3a-c0a2-11e0-897e-00e018e48620    N/A  ada6p2
gptid/8707ecde-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada7p1
gptid/872106d2-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada7p2
gptid/868caa9f-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada8p1
gptid/86a578d5-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada8p2
gptid/8595363a-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada9p1
gptid/85b02854-c9a0-11e2-bad5-50e549c7ad3f    N/A  ada9p2
                            gpt/swap-ada0    N/A  ada10p1
gptid/adced8e5-c0a2-11e0-897e-00e018e48620    N/A  ada10p1
gptid/ade611e2-c0a2-11e0-897e-00e018e48620    N/A  ada10p2
gptid/7463b438-dda3-11e2-b371-50e549c7ad3f    N/A  ada11p1
                    ufsid/51c9a8f78ba54025    N/A  ada11p2
                            ufs/Downloads    N/A  ada11p2
gptid/74720125-dda3-11e2-b371-50e549c7ad3f    N/A  ada11p2
                            ufs/FreeNASs3    N/A  da0s3
                            ufs/FreeNASs4    N/A  da0s4
                            ufs/FreeNASs1a    N/A  da0s1a


That "Downloads" UFS volume is a single disk. I have no problems with that one I can import it and use the data readily.

The issue is with the the 6 ST3000DM0001* drives. Those drives were used to create a single RAIDZ2 volume. I need that volume back and need to know if there's any information on the drives themselves that will allow me to re-build that volume in FreeNAS.

I believe the drives are healthy and in good working order. The only failure was in the root drive.

Sadly my last FreeNAS db backup was from before I generated the second RAIDZ2 volume (let the scolding begin :P ). The data on that second volume is extremely valuable to me.

As for my knowledge level, I have only a beginners understanding of ZFS and FreeNAS, but I was able to install and configure transmission on the root FreeNAS drive and VI is my preferred editor (purely as an indicator of my level of experience and knowledge).

Thanks for reading my thread; any and all help would be greatly appreciated.

~ R
 

rhing

Cadet
Joined
Sep 18, 2013
Messages
2
I was doing some more research tonight and it seems that the ZFS drives themselves should contain metadata describing the zpool that they belong to, thus enabling the pool to be recreated.

Clearly this was the case for the first volume which was automatically imported, but not so for the second volume. Perhaps there's a way to repair the metadata on the drives?

Just thinking out loud here...
 
Status
Not open for further replies.
Top