Expert help required to rebuild ZFS pool

[Dogsbody]

When migrating from one FreeNAS8 box to another FreeNAS8 box our customer accidentally typed the wrong command and really broke the ZFS pool...

• Original configuration: FreeNAS server with zfs pool ‘filestore’ configured as a raid1 mirror. Both disks operational and pool healthy.
• Shutdown server
• Removed Disk A and placed in new chassis, imported ZFS pool successfully, pool showing degraded but data accessible.
• Placed Disk B in new chassis, zfs did not recognize this disk as part of the pool, issued ‘zpool add -f filestore /dev/ada0’ command. I now realize this was not what I wanted, I wanted to issue a zpool attach command.
• This resulted in Disk B being added to the pool as a striped disk, the pool size increased from 2.7TB to 5.2TB, pool showing as degraded but data still accessible.
• Server is rebooted
• Pool is missing

zpool status is now showing "no pools available" and no combinations of zpool import will restore things. I've just been talking to a storage guy who really helped to try and fix the pool but we haven't managed to get anywhere. We now need a FreeNAS / ZFS on FreeBSD expert.

Can anyone help please? I don't know if I'm allowed to offer in this forum but I am more than willing to pay anyone for their time!

Thank you all in advance.

 

[cyberjock]

Can you provide a list of commands that your storage guy did?

I might be willing to help... handled quite a few of these this year... just wondering how much damage has been done :P

 

[Dogsbody]

What haven't we tried! I'll try and give the edited highlights. It's too long for here so please see this pastebin

Thank you

 

[cyberjock]

PM sent...

 

[Ricardo Cardoso]

Almost the same here.

• Freenas Server ZFS raid-1 "RAIDZ" online and Healthy.
• Using the GUI set one disk offline for upgrade disk size
• shutdown
• changed the physical disk
• boot
• unable to import pool
• zpool status: no pool available
• zpool import:

Code:

  pool: RAIDZ
    id: 3634811780714734649
  state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
    devices and try again.
  see: http://www.sun.com/msg/ZFS-8000-3C
config:
 
    RAIDZ                                          UNAVAIL  insufficient replicas
      raidz1-0                                      UNAVAIL  insufficient replicas
        gptid/618319ef-915b-11e1-8fc4-386077d5d4c9  ONLINE
        13020385326369302278                        OFFLINE
        gptid/6287ea8f-915b-11e1-8fc4-386077d5d4c9  ONLINE
        9895012226997409189                        UNAVAIL  cannot open

• shutdown
• put back the off lined disk
• still unable to import, same error, same replies to zpool status and zpool import
• zpool online RAIDZ ada0: cannot open 'RAIDZ': no such pool

So, put a disk off line, replaced, other disk get UNAVAIL, put the offlined disk back but how to get it online if the pool cannot be imported?

Thanks for you all for any help!

 

[cyberjock]

It sounds like you didn't do a scrub to ensure complete redundancy before you started. Not sure how to help you.. plenty busy with the OP for the next few days though. If you can't get it figured out by Tuesday or Wednesday send me a PM and I'll see what I can do. Be careful what commands you attempt to run.. many are permanent and unrecoverable if the command breaks things. :P

 

[papageorgi]

i know it's been a while but if i was not able to get my data back i was prepared to make a dd copy of each disk for some future possiblities.... here we are in the future and if you've made a copy or perhaps still have those drives you may be interested in looking at my thread http://forums.freenas.org/threads/zfs-pool-not-available-after-upgrade-from-8-3-1-to-9-1-beta.13734/

 

[cyberjock]

i know it's been a while but if i was not able to get my data back i was prepared to make a dd copy of each disk for some future possiblities.... here we are in the future and if you've made a copy or perhaps still have those drives you may be interested in looking at my thread http://forums.freenas.org/threads/zfs-pool-not-available-after-upgrade-from-8-3-1-to-9-1-beta.13734/

If you look at his info, his issue has nothing to do with upgrading FreeNAS. His has to do with 2 devices being missing from a RAIDZ1.

Most of your pain is because you did a zpool destroy. Why you'd think that doing a command with the word "destroy" would be a smart decision is completely beyond me. You even chose to go doing stuff several days after I had told you explicitly NOT to do anything that wasn't recommended by the forum. Lucky for you delphij chose to help you get your data back with some custom files he made for you. You owe him bigtime for him helping you, and you can start by not diving into a bunch of different threads(many from quite some time ago) and injecting your comments that are completely unrelated to the thread. I've seen your name pop up a bunch of times tonight, sometimes responding to threads with information that is not related to the thread(like this one) and others comments on threads that are more than 2 months old and those topics are long since done.

 

[cyberjock]

Did you read his post? Do you see any comment about him updating FreeNAS(which started your problem)? Notice where he talks about offlining a disk.. do you see anywhere that you mentioned offlining a disk in your issues? The 2 are not related... at all.

 

[papageorgi]

yes in fact i did have an offline disk, and it's one of the parts that brought the pool back. are you trying to be a "cyberjock"