Data Recovery - RAID 10 with 2 failing drives

[Fraser]

Hello folks!

This is my first post and, yes, I am a total noob when it comes to FreeNAS. I am a data recovery specialist and I have recovered data from LOTS of RAID arrays over the years but this is the first time anyone has brought in an array from a FreeNAS.

So a client of mine was upgrading the power supply in his FreeNAS and he mistakenly installed the wrong power cable to 2 of the drives. Of course he did not realize what he had done until he powered it up. 2 of the drives had power pushed in through the ground pins because the cable he plugged in was from his old power supply and was not pinned out the same as his new one....OOPS! The magic smoke has escaped! Since all 4 drives are the same model I was able to migrate the ROM data over to one of the other PCBs only to find out that the power serge also took out the drive's PreAmp chip. In order to get these drives working again I would have to take them into the lab and replace the head stack assemblies on both of the failing drives.

The FreeNAS was configured as a RAID 10. I have analyzed the two remaining drives using a hex editor and it's pretty clear that the data on those 2 drives is different at every sector that I checked so it appears that I have a complete striped pair. I attempted to import just that pair but it was not successful.

My question for the FreeNAS community,...is there a way to mount the striped pair of drives without the 2 failing drives? Would it be possible to put 2 new drives into the array and rebuild the data set using the 2 good drives?

I am very new to FreeNAS and I figure it's probably a good idea to ask the experts before throwing a bunch of commands around that I do not fully understand.

Thank you in advance. Any help would be appreciated.

Fraser Corracne
Progressive Tech

ps. I am working with clones of the good drives so that I don't do anything to the originals that I can't undo.

 

[Heracles]

Hi,

That incident reminded me when I once plugged a 110V power supply in a 220 power bar... Nice little fireworks...

The 4 drives being configured in a equivalent of Raid-10, you have 67% chances of having the required data to recover. The setup will always survive the first lost and 2 out of the 3 remaining drives are from the second mirror, so you can loose any of these two and will survive.

In FreeNAS, the logical is :
--Hard drives are grouped in a vDev, with or without redundancy
--vDevs are grouped in a pool, always without any redundancy
--zfs is the filesystem addressing that pool

So in your case, you would have to deal with 2 vDevs with a single redundant drive each and one pool.

But there was also the pool from which FreeNAS boot. How is that one ? Can the server boot with its existing resources ?

So first test would be : try to boot up FreeNAS with only the boot devices plugged in, no hard drives. Does it boot ? If it does, backup the config just to be safe. If it does not, you will need to make yourself a new FreeNAS boot disk.

Once you confirmed that the server can boot from your boot device (old or new), power it down and replug the 2 good drives. If booting from the still working old boot disk and indeed you have one drive from each vDev, FreeNAS should be able to load the pool in a degraded state all by itself. If it is the case, mark the damaged disks as Offline and power down the server. Replace the drives with new good ones of the same size or bigger and reboot. Once back up, again FreeNAS should load the pool as Degraded. Go in the disk management to activate the new drives as replacement for the old ones. FreeNAS will then start its re-silvering process and once done, the pool should turn back to Healthy.

If you can not boot from the old FreeNAS boot device, boot from a new one with only the 2 good drives plugged in. Once the boot is done, go in the Storage menu / Pool and try to import the pool. FreeNAS will automatically scan the drives for an existing pool and should be able to rebuild the previous pool in its degraded state. Once done, do the disk replacement as explained above.

Should neither the original boot nor the new one be able to import the degraded pool, you may have been unlucky and loss 2 drives of the same vDev. In that case, your pool and data are lost, unless you can revive at least one of the damaged drive with your own hardware expertise.

As for the data being the same or not, FreeNAS save some space on each drive for swap and that swap is encrypted. That may be why you see different data despite the 2 drives being part of the same vDev : you were looking at that encrypted swap space.

Good luck in your data recovery,

 

[Heracles]

The one who asked the question first was indeed a data recovery specialist... The question was about what to do with software before using the heavy artillery and hardware procedures.

 

[danb35]

Something about that post struck me as spammy (the fact that s/he necro'd a three-year-dead thread being part of it), so reported to the mods.

 

[Heracles]

You were right @danb35 : he is just a spammer....

 

[danb35]

...and already reported again.