How much of the disk does a GUI-based wipe actually wipe?

[rico]

OK, I was careless and "wiped" a disk that I shouldn't have. I also wrote a 0 Byte file to the new volume as well. Is there any hope of data recovery?

Can someone tell me how much of the disk actually gets wiped when you use the GUI to create a new ZFS volume and it prompts you to wipe the disk? (and you stupidly tell it to go through with it)?

I'm hoping it's just partial (maybe the front and back of the disk). I read that someone was able to do ZFS data recovery under a given scenario (which may not be as bad as my situation): http://mbruning.blogspot.com/2009/12/zfs-data-recovery.html

I tried to find where the wipe is done in code by googling "freenas wipe", and it did pull up the changeset for when this feature was originally introduced (http://support.freenas.org/changeset/11454/freenas). Looks like one or more dd commands were used (I don't know Python, so I may have made a mess of these):

Code:

"dd if=/dev/zero of=/dev/%s bs=1m oseek=%s" % (devname, size*1024 - 4)
"dd if=/dev/zero of=/dev/%s bs=1m oseek=%s" % (devname, size / 1024 - 4, )
"dd if=/dev/zero of=/dev/%s bs=1m count=1" % (devname, )

That came out of trunk/gui/freeadmin/static/lib/js/freeadmin.js, but the latest freeadmin.js doesn't have any dd commands in it.

For reference, I used FreeNAS-8.3.0-BETA1-x64 to create the new volume on the disk I shouldn't have.

The disk that was wiped contained data written with FreeNAS-8.0.2-RELEASE-amd64 (8288).

I also saw the dd command referenced in the FAQ: http://protosd.blogspot.com/2011/12/protosds-unofficial-freenas-8-faq.html

Is that what is used by the GUI in FreeNAS-8.3.0-BETA1-x64, i.e.

Code:

dd if=/dev/zero of=/dev/ada1 bs=1m count=1
dd if=/dev/zero of=/dev/ada1 bs=1m oseek=`diskinfo ada1 | awk '{print int($3 / (1024*1024)) - 4;}'`

??

 

[ProtoSD]

If you let the wipe complete, there is NO hope.

 

[rico]

So when you use the GUI to create a new ZFS volume and it prompts you to wipe the disk, it actually wipes the entire disk? And not just the partition and/or or other file system meta-data?

It seemed like it got through that rather quickly for a 1.5 TB disk...

 

[paul]

it wipes the disk out try and pull the disk out and try and see if you can use a zfs recovery on the internet don't know i backup you should do the same BACKUP YOUR SERVER Even now it's backing up your computer?

 

[rico]

Even now it's backing up your computer?

No, I've got no backup going on right now. However, if I do ship the disk off for recovery, I'll be sure to take a backup image of it first.

 

[rico]

OK, I'm still looking for the answer to my original question, "How much of the disk does a GUI-based wipe actually wipe?"

Here is what I've done since my last post:

svn co https://freenas.svn.sourceforge.net/svnroot/freenas/tags/8.3.0-BETA1 (this is the version I did the wipe with).

Did some digging around down there looking for strings like "wipe" and "dd if".

I don't know Python, but as best as I can tell, the GUI is using a wizard which in turn uses this "disk_wipe" definition that starts here:

Code:

8.3.0-BETA1/gui/storage/views.py:928:def disk_wipe(request, devname):

Which in turn uses this "notifier" class operation "disk_wipe":

Code:

8.3.0-BETA1/gui/storage/views.py:953:                notifier().disk_wipe(devname, form.cleaned_data['method'])

Which is defined in 8.3.0-BETA1/gui/middleware/notifier.py:

Code:

  3692	    def disk_wipe(self, devname, mode='quick'):
  3693	        if mode == 'quick':
  3694	            self.__gpt_unlabeldisk(devname)
  3695	            pipe = self.__pipeopen("dd if=/dev/zero of=/dev/%s bs=1m count=1" % (devname, ))
  3696	            err = pipe.communicate()[1]
  3697	            if pipe.returncode != 0:
  3698	                raise MiddlewareError(
  3699	                    "Failed to wipe %s: %s" % (devname, err)
  3700	                    )
  3701	            try:
  3702	                p1 = self.__pipeopen("diskinfo %s" % (devname, ))
  3703	                size = int(re.sub(r'\s+', ' ', p1.communicate()[0]).split()[2]) / (1024)
  3704	            except:
  3705	                log.error("Unable to determine size of %s", devname)
  3706	            else:
  3707	                pipe = self.__pipeopen("dd if=/dev/zero of=/dev/%s bs=1m oseek=%s" % (
  3708	                    devname,
  3709	                    size / 1024 - 4,
  3710	                    ))
  3711	                pipe.communicate()
  3712	
  3713	        elif mode in ('full', 'fullrandom'):
  3714	            libc = ctypes.cdll.LoadLibrary("libc.so.7")
  3715	            omask = (ctypes.c_uint32 * 4)(0, 0, 0, 0)
  3716	            mask = (ctypes.c_uint32 * 4)(0, 0, 0, 0)
  3717	            pmask = ctypes.pointer(mask)
  3718	            pomask = ctypes.pointer(omask)
  3719	            libc.sigprocmask(signal.SIGQUIT, pmask, pomask)
  3720	
  3721	            self.__gpt_unlabeldisk(devname)
  3722	            stderr = open('/var/tmp/disk_wipe_%s.progress' % (devname, ), 'w+')
  3723	            stderr.flush()
  3724	            pipe = subprocess.Popen([
  3725	                "dd",
  3726	                "if=/dev/zero" if mode == 'full' else "if=/dev/random",
  3727	                "of=/dev/%s" % (devname, ),
  3728	                "bs=1m",
  3729	                ],
  3730	                stdout=subprocess.PIPE,
  3731	                stderr=stderr,
  3732	                )
  3733	            with open('/var/tmp/disk_wipe_%s.pid' % (devname, ), 'w') as f:
  3734	                f.write(str(pipe.pid))
  3735	            pipe.communicate()
  3736	            stderr.seek(0)
  3737	            err = stderr.read()
  3738	            print err
  3739	            libc.sigprocmask(signal.SIGQUIT, pomask, None)
  3740	            if pipe.returncode != 0 and err.find("end of device") == -1:
  3741	                raise MiddlewareError(
  3742	                    "Failed to wipe %s: %s" % (devname, err)
  3743	                    )
  3744	        else:
  3745	            raise ValueError("Unknown mode %s" % (mode, ))

So it looks like there are 3 modes (or methods) for the disk wipe: quick (starting on line 3693), full (starting on line 3713), and fullrandom (also starting on line 3713).​

I'm struggling to determine where that mode (or method) is assigned. However, it does looks like "quick" is the default.

I guess I'll have to understand how this "form.cleaned_data['method']" from views.py:953 works.

I sure hope the "quick" method is used by the GUI. That looks like it only wipes the first 1m (1024*1024) Bytes of the disk (dd used with bs=1m count=1) and another 1m Bytes at the end.

 

[cyberjock]

Not to dismiss your attempts to recover your data. But someone else in the forum had a similar issue. The short story was that finding the inodes is difficult(I don't know what an inode is exactly, but I've seen the phrase used alot when discussing disk recovery). Even having all of the other disks in a RAID5 or 6 where you are 1 disk short of accessing the data it is difficult to find the inodes.

Someone also tried calling a few companies to get quotes for data recovery and many companies wouldn't do ZFS. The one that did was in the ball park of $10-15k for their data recovery of several 2TB drives, and there was no guarantee that data could be recovered.

My guess is that this is why protosd said that there is no hope. If you have deep pockets there may be a slim chance, at best. But I definitely wouldn't "count" on seeing your data again.

 

[cyberjock]

To be completely honest, if you DO somehow manage to get your data back, you'd be the first I've seen in the forum to recover data from what a senior moderator/admin would consider an "unrecoverable" situation. If you manage this you should definitely post back and let the forum know what company you used or how you did it. I'm sure there would be a few people that would love to see how you did it.

I'd say that ZFS is designed mostly for high end server environments. Anyone that can afford that probably also values their data and usually will do good backups. So that only leaves ZFS users that are low budget. Those individuals likely can't/won't spend the money to recover a zpool. So who exactly is the "target" customer with regards to recovering zpool data?

 

[William Grzybowski]

Yes, if you have just created a new volume using those disks it will just wipe 1 mb in the beginning and 1mb at the end of the disk..

Although this is enough to do a lot of damage...

 

[rico]

Even having all of the other disks in a RAID5 or 6 where you are 1 disk short of accessing the data it is difficult to find the inodes.

The disk with the data that got "wiped" came from a single disk configuration. Does that make it easier to do recovery?

I had intended for it to be 1 of 2 disks in a RAID0 configuration, but the BIOS only saw one of the disks and I didn't figure that out until after I had been using the 2 disks (really only 1) for some time. So all of the data was on the one disk (otherwise I'd have another copy intact to rebuild from).

Someone also tried calling a few companies to get quotes for data recovery and many companies wouldn't do ZFS. The one that did was in the ball park of $10-15k for their data recovery of several 2TB drives, and there was no guarantee that data could be recovered.

I think I saw something like that as well. However, Max Bruning claims to have done ZFS data recovery in much less time than the $15k/month shops. See his blog post to that affect here: http://mbruning.blogspot.com/2009/12/zfs-data-recovery.html.

Now, he was recovering from data corruption, not partition-table wiping (or whatever is lost by the GUI-based wipe), so perhaps he wouldn't be able to help here.

Peter William Lount had a 10:30 PM comment on that same page that discusses finding the uber blocks. I've done this on my wiped disk and have found uber blocks. What I don't know is if the blocks found are from the original data, or from the new filesystem overwrite.

I mentioned before that I had only written a 0 Byte file to the new volume on the same disk, so I imagine (hope) that all my data is still sitting on the disk waiting for someone with enough knowledge of ZFS internals to do the recovery.

My guess is that this is why protosd said that there is no hope. If you have deep pockets there may be a slim chance, at best. But I definitely wouldn't "count" on seeing your data again.

I do not have deep pockets, but I am not ready to throw in the towel yet, either.

 

[rico]

To be completely honest, if you DO somehow manage to get your data back, you'd be the first I've seen in the forum to recover data from what a senior moderator/admin would consider an "unrecoverable" situation. If you manage this you should definitely post back and let the forum know what company you used or how you did it. I'm sure there would be a few people that would love to see how you did it.

Certainly!

I'd say that ZFS is designed mostly for high end server environments. Anyone that can afford that probably also values their data and usually will do good backups. So that only leaves ZFS users that are low budget. Those individuals likely can't/won't spend the money to recover a zpool. So who exactly is the "target" customer with regards to recovering zpool data?

Sure, the target customer is the customer with money. But hobbyists have made things possible with open source that were previously only open to deep pockets. I'm not suggesting that I will make recovery of my data possible. I'm suggesting that someone else may have already done that and that I just need to discover how it's done.

I have to admit that I am getting a bit discouraged at this point, but we'll see.

 

[rico]

Yes, if you have just created a new volume using those disks it will just wipe 1 mb in the beginning and 1mb at the end of the disk..

Thank you for that confirmation!

 

[cyberjock]

Sure, the target customer is the customer with money. But hobbyists have made things possible with open source that were previously only open to deep pockets. I'm not suggesting that I will make recovery of my data possible. I'm suggesting that someone else may have already done that and that I just need to discover how it's done.

ZFS is very complex. I can't imagine a hobbyist is going to come up with some "quick easy tools" to recover data. From what I've read ZFS isn't something a small group of hobbyists can work on. It seems to be something dealt with by a large group of well funded people.

I'd love to see someone come up with a ZFS version of fsck. I've seen several discussions of the need to have one, but nobody seems to be able to do it. No surprise if the above paragraph is true.

Edit: One of my worries before I started using FreeNAS was with ZFS versus using something like NTFS(Windows) or ext3/4(Linux) is that with those more popular file systems there are COTS products to help with recovery in case you have a bad zpool and your backups fail you. With ZFS there are virtually zero COTS products and few if any data recovery companies that will even work with ZFS. I've never seen one that actually listed ZFS as a supported file system. You are literally at the mercy of your admin to know what he/she is doing or the mistakes can be very costly.