Disaster Instructions

[bujums]

Hi all, I was running FreeNAS 8 as a very simple way to share information between windows computers. It was easy to set up and ran great. I did have it on a UPS Battery back up but it was not set to shutdown on use of battery (my fault). Any way the power went out one day... OS was fine and the Data turned out to be fine. But we found out that trying to access those drives (raid 1 set of two drives) was nearly impossible I could not with any of of my Linux rescue live discs. the only way to even get the drive mounted was with FreeNAS via command line (FreeNAS could not boot without the messed up drives) BSD is not like most linux and we really had a hard time getting documentation online. We finally got the data after about 15 hours with one person working on it and another person remotely logged in, and I will set up another NAS with another version of Linux. I just wanted to say I really like how easy it was but if there could be some documentation for after a disaster, command line help for BSD for forcing drives to mount. Or even better a rescue disk that can be downloaded. I realize that I am not very experienced but the person remotely logged in has a lot of experience we are lucky we got the data. Too Bad FreeNAS did not switch to Debian for more access to documentation.

Sorry for the rant! FreeNAS would be practically perfect with some disaster instructions though.
Thanks!

 

[xbmcg]

Hi,

There are a lot of desaster recovery tools out there on the market, that can analyze and retrieve data from broken RAID configs, I have done it on a RAID5 with NTFS file system due to an enclosure failure (SATA port multiplyer - don't use such things on valuable data!!!). I needed 2 times the space on a single disk to recover the data - one to raw read all sectors from the disks and the second part to analyze / resort all stuff back to linear files / volumes. It took on a RAID set with 2 TB about a week of computing - but finally all data was retrieved - except 2 damaged files.

I think similar tools exist for ext2/3/4 and FAT32. But I doubt there is something sophisticated enough out there to post-mortem retrieve zfs raidz stuff, hovever raid1 shold not be an issue, since you can mount it as discrete drive and read all sectors.

This is also the reason I will never ever use any striped config for large data. RAID1 and no spanning volumes over RAID1 sets is the safest way to have a chance to keep as much data as possible after a crash disaster.

 

[bujums]

I did find some recovery disc that say they work on UFS files but I am just a little gun shy now. I set my new server up with Ubuntu server 10.04, installed Webmin
Raid 1 (hardware, ext4, backup external hard drive FAT32, and it is just about in action, I am sad though cause I really liked have the OS on a USB stick and having that stick cloned so I could just pop a new on if I had trouble. The data is the most important thing though, not the OS and At least I am learning tons of stuff when I run into a problem!! I think I will look around for some good recovery tools and have them handy for the next issue.

 

[xbmcg]

Good idea. However my problem was not the OS, but a hardware issue in the enclosure, what screwed up the RAID5 configuration. The new controller was unable to mount the disks due to data corruption, so only a disk surface analysis (raw read of the sectors, scan for partition tables, find / guess the right order of the disks and unstripe the content) helped me to recover the files. This could happen even on a OS on a thumb drive. I do not know, if it would work on encrypted, compressed filesystems and variable sector lengths or deduplicated data - where one sector can be referenced / used by many file pointers, also snapshots with multiple data chain trees etc.

I prefer a not so sophisticated, cleaner design, at least to limit the damage down to a single disk and not lose the whole data volume - so RAID1 is my first choice. I still can mount the one single drive and retieve all data on it, the other pairs stay unaffected.

I wonder, if there are tools to data-mine zfs pool disks in a reasonable way, because there seems not to be a fixed, reliable structure one can guess, recover from the remaining spindles, especially if one adds later more disks, so the structure gets mixed up over time and the files gets randomly distributed.

 

[bujums]

I don't know but I did not see anything for AFS when I was looking. I did find some stuff for UFS and that is what I was dealing with. I am using the Raid 1 has hardware only plus an external backup I figure I should be able to get the data pretty easily one way or another! Not too worried about the OS itself. I am getting faster and faster at install and setting up a server.