2way mirror died, looking for help

Status
Not open for further replies.
korund

korund

Cadet
Joined
Jan 12, 2014
Messages
4
Hi guys,

Seems like today my precious mirror was put into unreadable state. Here is what happend:

1. I had 2-way mirror (named "stone"), Freenas 9.2 on flashdrive, 8Gb ECC ram.
2. Connected 3rd drive and tried to extend mirror to 3-way via WebGui. Looks like I did it wrong because 3rd drive become connected as "Stripe".
3. I did "zpool export" and (stupid idea) wiped 3rd drive using "quick wipe" option in WebGui.
4. Now I'm unable to import mirror and following error is returned:
Code:
cannot import 'stone': one or more devices is currently unavailable


What we have:

1. Two original disks
2. 3rd disk untouched after "quick wipe"
3. Before wipe mirror looked like this:

Code:
  pool: stone
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 1h13m with 0 errors on Sun Dec 15 01:13:57 2013
config:
 
        NAME                                            STATE    READ WRITE CKSUM
        stone                                          ONLINE      0    0    0
          mirror-0                                      ONLINE      0    0    0
            gptid/7259140f-44ab-11e3-9064-6805ca195278  ONLINE      0    0    0
            gptid/72c4474f-44ab-11e3-9064-6805ca195278  ONLINE      0    0    0
          gptid/fb22bbc2-7b89-11e3-8acc-74d02b982992    ONLINE      0    0    0


4. Also, zpool.cache and zpool.cache.saved are both attached.

Sometimes we are learning on our own mistakes. Any chances to get this data back? I will highly appreciate any help or suggestions.
 

Attachments

  • zpool.zip
    1.4 KB · Views: 196
C

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Yep.. if you wiped that stripe you are done. There is no recovering from that error. You aren't the first one to get it wrong, but you aren't the last either I'm sure.

I'll never understand why someone would pull a disk out and zero it before verifying their pool still functions. I don't generally wipe disks except after they've sat in a box for months and I plan to sell them or something.
 
ZFS Noob

ZFS Noob

Contributor
Joined
Nov 27, 2013
Messages
129
What cyberjock said. A pool is a stripe of vdevs, and you ended up with a stripe across a mirror and a single drive. When you removed that single drive and diddled the bits on it, the zpool sees a broken stripe now.
 
C

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Hopefully you have backups.
 
C

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Pretty sure that won't work, but you are welcome to try it.
 
Yatti420

Yatti420

Wizard
Joined
Aug 12, 2012
Messages
1,437
So the third drive which was added became a stripe of the mirror? Wiping or removing striped drive (in any way) would then break "stone" pool entirely?

If they system was left online and you realized what you did would you be able to do anything or would you have to move data and kill the whole pool no matter what?
 
korund

korund

Cadet
Joined
Jan 12, 2014
Messages
4
Yatti420 said:
So the third drive which was added became a stripe of the mirror? Wiping or removing striped drive (in any way) would then break "stone" pool entirely?

If they system was left online and you realized what you did would you be able to do anything or would you have to move data and kill the whole pool no matter what?
Click to expand...

For the moment being I'm trying to recover data from pool.
 
korund

korund

Cadet
Joined
Jan 12, 2014
Messages
4
Update:

I'm able to see history on the drive using
Code:
zdb -lu /dev/da0
however some labels are failed to read.

Code:
--------------------------------------------
LABEL 0
--------------------------------------------
failed to unpack label 0
--------------------------------------------
LABEL 1
--------------------------------------------
failed to unpack label 1
--------------------------------------------
LABEL 2
--------------------------------------------
failed to unpack label
failed to unpack label 2
Uberblock[64]
        magic = 0000000000bab10c
        version = 5000
        txg = 605920
        guid_sum = 11456902719942460913
        timestamp = 1389526699 UTC = Sun Jan 12 13:38:19 2014
Uberblock[68]
        magic = 0000000000bab10c
        version = 5000
        txg = 605921
        guid_sum = 11456902719942460913
        timestamp = 1389526699 UTC = Sun Jan 12 13:38:19 2014
Uberblock[72]
        magic = 0000000000bab10c
        version = 5000
        txg = 605922
        guid_sum = 11456902719942460913
        timestamp = 1389526699 UTC = Sun Jan 12 13:38:19 2014
Uberblock[76]
        magic = 0000000000bab10c
        version = 5000
        txg = 605923
        guid_sum = 11456902719942460913
        timestamp = 1389526704 UTC = Sun Jan 12 13:38:24 2014
Uberblock[80]
        magic = 0000000000bab10c
        version = 5000
        txg = 604740
        guid_sum = 11456902719942460913
        timestamp = 1389480513 UTC = Sun Jan 12 00:48:33 2014
Uberblock[84]
        magic = 0000000000bab10c
        version = 5000
        txg = 604741
        guid_sum = 11456902719942460913
        timestamp = 1389480515 UTC = Sun Jan 12 00:48:35 2014
Uberblock[88]
        magic = 0000000000bab10c
        version = 5000
        txg = 604998
        guid_sum = 11456902719942460913
        timestamp = 1389520060 UTC = Sun Jan 12 11:47:40 2014
Uberblock[92]
        magic = 0000000000bab10c
        version = 5000
        txg = 604903
        guid_sum = 11456902719942460913
        timestamp = 1389481245 UTC = Sun Jan 12 01:00:45 2014
Uberblock[96]
        magic = 0000000000bab10c
        version = 5000
        txg = 606952
        guid_sum = 1155078391826158133
        timestamp = 1389531831 UTC = Sun Jan 12 15:03:51 2014
Uberblock[100]
        magic = 0000000000bab10c
        version = 5000
        txg = 606953
        guid_sum = 1155078391826158133
        timestamp = 1389531832 UTC = Sun Jan 12 15:03:52 2014
Uberblock[104]
        magic = 0000000000bab10c
        version = 5000
        txg = 606954
        guid_sum = 1155078391826158133
        timestamp = 1389531832 UTC = Sun Jan 12 15:03:52 2014
Uberblock[108]
        magic = 0000000000bab10c
        version = 5000
        txg = 606955
        guid_sum = 1155078391826158133
        timestamp = 1389531837 UTC = Sun Jan 12 15:03:57 2014
Uberblock[112]
        magic = 0000000000bab10c
        version = 5000
        txg = 604908
        guid_sum = 11456902719942460913
        timestamp = 1389519611 UTC = Sun Jan 12 11:40:11 2014
Uberblock[116]
        magic = 0000000000bab10c
        version = 5000
        txg = 604909
        guid_sum = 11456902719942460913
        timestamp = 1389519616 UTC = Sun Jan 12 11:40:16 2014
Uberblock[120]
        magic = 0000000000bab10c
        version = 5000
        txg = 604750
        guid_sum = 11456902719942460913
        timestamp = 1389480526 UTC = Sun Jan 12 00:48:46 2014
Uberblock[124]
        magic = 0000000000bab10c
        version = 5000
        txg = 604751
        guid_sum = 11456902719942460913
        timestamp = 1389480527 UTC = Sun Jan 12 00:48:47 2014
--------------------------------------------
LABEL 3
--------------------------------------------
failed to unpack label 3
Uberblock[64]
        magic = 0000000000bab10c
        version = 5000
        txg = 605920
        guid_sum = 11456902719942460913
        timestamp = 1389526699 UTC = Sun Jan 12 13:38:19 2014
Uberblock[68]
        magic = 0000000000bab10c
        version = 5000
        txg = 605921
        guid_sum = 11456902719942460913
        timestamp = 1389526699 UTC = Sun Jan 12 13:38:19 2014
Uberblock[72]
        magic = 0000000000bab10c
        version = 5000
        txg = 605922
        guid_sum = 11456902719942460913
        timestamp = 1389526699 UTC = Sun Jan 12 13:38:19 2014
Uberblock[76]
        magic = 0000000000bab10c
        version = 5000
        txg = 605923
        guid_sum = 11456902719942460913
        timestamp = 1389526704 UTC = Sun Jan 12 13:38:24 2014
Uberblock[80]
        magic = 0000000000bab10c
        version = 5000
        txg = 604740
        guid_sum = 11456902719942460913
        timestamp = 1389480513 UTC = Sun Jan 12 00:48:33 2014
Uberblock[84]
        magic = 0000000000bab10c
        version = 5000
        txg = 604741
        guid_sum = 11456902719942460913
        timestamp = 1389480515 UTC = Sun Jan 12 00:48:35 2014
Uberblock[88]
        magic = 0000000000bab10c
        version = 5000
        txg = 604998
        guid_sum = 11456902719942460913
        timestamp = 1389520060 UTC = Sun Jan 12 11:47:40 2014
Uberblock[92]
        magic = 0000000000bab10c
        version = 5000
        txg = 604903
        guid_sum = 11456902719942460913
        timestamp = 1389481245 UTC = Sun Jan 12 01:00:45 2014
Uberblock[96]
        magic = 0000000000bab10c
        version = 5000
        txg = 606952
        guid_sum = 1155078391826158133
        timestamp = 1389531831 UTC = Sun Jan 12 15:03:51 2014
Uberblock[100]
        magic = 0000000000bab10c
        version = 5000
        txg = 606953
        guid_sum = 1155078391826158133
        timestamp = 1389531832 UTC = Sun Jan 12 15:03:52 2014
Uberblock[104]
        magic = 0000000000bab10c
        version = 5000
        txg = 606954
        guid_sum = 1155078391826158133
        timestamp = 1389531832 UTC = Sun Jan 12 15:03:52 2014
Uberblock[108]
        magic = 0000000000bab10c
        version = 5000
        txg = 606955
        guid_sum = 1155078391826158133
        timestamp = 1389531837 UTC = Sun Jan 12 15:03:57 2014
Uberblock[112]
        magic = 0000000000bab10c
        version = 5000
        txg = 604908
        guid_sum = 11456902719942460913
        timestamp = 1389519611 UTC = Sun Jan 12 11:40:11 2014
Uberblock[116]
        magic = 0000000000bab10c
        version = 5000
        txg = 604909
        guid_sum = 11456902719942460913
        timestamp = 1389519616 UTC = Sun Jan 12 11:40:16 2014
Uberblock[120]
        magic = 0000000000bab10c
        version = 5000
        txg = 604750
        guid_sum = 11456902719942460913
        timestamp = 1389480526 UTC = Sun Jan 12 00:48:46 2014
Uberblock[124]
        magic = 0000000000bab10c
        version = 5000
        txg = 604751
        guid_sum = 11456902719942460913
        timestamp = 1389480527 UTC = Sun Jan 12 00:48:47 2014
 
Status
Not open for further replies.

Similar threads

B
  • Locked
SOLVED Volume status DEGRADED, can't replace with spare
Replies
2
Views
1K
beria
B
T
  • Locked
l2arc removed but still being accessed
Replies
4
Views
922
Teva
T
F
Pool shows offline on GUI but its gone on CLI
Replies
3
Views
1K
sretalla
sretalla
mysticpete
SOLVED Hard failure of drive, cannot mark as 'offline' per manual
Replies
8
Views
2K
Demonlinx
D
wtfuar
  • Locked
auto decrypt after ZFS Pool extention ?
Replies
7
Views
2K
Dusan
Dusan
Top