ATA Status error... and GPT error

[mrEllis]

I had a power blink yesterday and when I got home my freenas box was not sharing my media. I plugged in a monitor and keyboard and rebooted and I see this:

so i think i have a couple things going on here.. ada0 is not happy (a bad drive possibly?) and ada1 shows a corrupt GPT.

how might i go about recovering my data, is it even possible?

Thanks.

 

[danb35]

Looks like your boot device is hosed. Do a clean install to a new device, upload your saved config file, and reboot. You should be set.

 

[mrEllis]

danb35, thanks for the response!

the boot device seems to work fine as freenas does boot and i can get into the webui. ada0, ada1, and ada2 are my data drives (geoms?) according to the webui. im a little weak on the terminology. if i unplug the faulty hard drive (ada0) the only error i get when freenas boots is the GPT error. once booted freenas webui shows that my mediastorage volume is in an unknown state.

i opened the terminal and tried "zpool import" and it gave me an error about not having enough replicas to mount. i am hoping that i can repair the damaged GPT and be able to mount it again.

i unplugged all the drives except for the one with the busted GPT, then booted the system from a ubuntu live usb drive. i ran gdisk on that drive with these results:

i have not tried to correct those errors because i dont know enough about it to not destroy my data...

any suggestions would be greatly appreciated.

 

[mrEllis]

as an update:

i plugged back in my boot drive and the 2 physically functioning data drives and booted the system. once the system was up dmesg shows that both of the remaining data drives have corrupt or invalid GPTs.

i think im going to stop messing with it until i get some direction :/

 

[danb35]

What's the output of 'zpool import'?

 

[mrEllis]

nothing.

 

[fta]

Code:

gpart list ada1

Code:

gpart recover /dev/ada1

 

[mrEllis]

Hi, fta. output below:

ada2 is my boot device, gpart list ada2 works as it should for it.

 

[cyberjock]

Yeah, the gpart recover won't work because the GPT table is corrupt. Can you provide a debug file on your system please? System -> Advanced -> Save Debug

 

[mrEllis]

cyberjock,
debug attached.

 

[cyberjock]

Well, your disks are definitely not partitioned anymore. What version of FreeNAS were you using when you created the zpool? If you aren't 100% sure, then make sure you mention this because it's important to know. ;)

 

[mrEllis]

I *think* I built the system all at once from scratch with a 9.3 installer... HOWEVER, since it is 5TB worth of data that I dont want to lose, if there is a way to make 100% sure, I am willing to do a little more research :)

 

[mrEllis]

i know they were partitioned with 2 partitions each, a 2GB swap, and the rest was a ZFS partition.

 

[titan_rw]

Yea, it looks like your data disks don't have a partition table anymore.

Can you boot up the system with only the boot drive and the 3rd (possibly faulty) disk connected to a different sata port than it was originally? And check it for partitions? gpart list and glabel list should be enough I'd think.

Depending on why the disks don't appear to have partitions, you might be able to simply recreate the GPT manually, and hope the zfs partition is still there.

 

[mrEllis]

how would i manually recreate the partition? what tools would i need to use?

gpart (on ubuntu live usb) can see the partitions but complains about the corrupt GPT. It says it can fix them, however it reports that there are issues with the size of the partition, it extends beyond the end of the disk so it cant write the backup GPT to the end of the disk... or something like that.

i acquired a couple loaner 3TB drives that im dd'ing my data drives to so i can tinker with them.

 

[titan_rw]

If possible, yes, I would mirror the drives to other drives so that you can experiment on the copies.

I was referring to using gpart from within freenas to recreate the partition table. I don't remember the commands right off the top of my head. And yes, you'd have to know for sure what the partition tables were. By default p1 would be 2G, and the rest would be for zfs as p2.

Again, double check that the 3rd 'bad' drive is actually bad. Boot with only it connected, see what it gets assigned for device (adaX), and try gpart list adaX, glabel list adaX, and then smartctl -a /dev/adaX.

 

[mrEllis]

ill give that a go. likely it will be sometime tomrrow, as the drives take quite a while to dd. I really appreciate the suggestions. Ill update as soon as i have more info.

 

[mrEllis]

OK, i have access to my data :)

My setup was 1x 3TB Seagate Barracuda and 2x 3TB WD Red drives... My system failed when the Seagate drive physically failed, the power blinked. When freenas rebooted one of the WD Red's GPT had corrupted. After some tinkering and a few reboots freenas reported that both WD Reds had bad GPTs.

I dd'ed my 2 original WD Reds over 2 new 3TB Seagate drives. I removed my original drives and booted freenas with the freshly copied Seagate drives... and freenas reported the secondary GPT was bad... but it went ahead and mounted the drives! I issued "gpart recover ada1" and "gpart recover ada2" and everything looks good now.

Code:

[root@freenas] ~# gpart show
=>       34  976773101  ada0  GPT  (465G)
         34       1024     1  bios-boot  (512k)
       1058          6        - free -  (3.0k)
       1064  976772064     2  freebsd-zfs  (465G)
  976773128          7        - free -  (3.5k)

=>        34  5860533101  ada1  GPT  (2.7T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5k)

=>        34  5860533101  ada2  GPT  (2.7T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5k)

Thank you guys for your help!