Failed drive on mirrored ZFS

[marc davis]

Hello

I'm admittedly a beginner user. That being said, I setup a system with 9.1. It has two 1 gig mirrored drives formatted with ZFS. It's been up and running for a year and doing what we asked of it. A few days ago I noticed I could not access the NAS as usual. A bit of poking around seems to indicate one of the drives has failed. It shows up on the list of drives but does not even show the serial number. So I unplugged that dive. I was hoping to be able to read the data on the remaining drive but I get the error message below.

Is the remaining drive corrupted?
Do I have to add a new drive to get the system up and running again?

Sorry If these are naive questions. I've done some reading and searching but haven't gotten a clear picture of where I stand and how to proceed.

Thanks
Marc

 

[BigDave]

A bit of poking around seems to indicate one of the drives has failed. It shows up on the list of drives but does not even show the serial number. So I unplugged that drive. I was hoping to be able to read the data on the remaining drive but I get the error message below.

Marc, at this point you should consider your data is gone. There may be a slight chance that if you properly shut down your server now
and plug back in the drive you removed, your data may become accessible again (the volume will still be listed as degraded).
If it does, make a backup (i assume u don't have one), then read the step by step instructions in the manual on how to replace a failed drive.
You need to consider building a pool with a little more redundancy (RAIDz2), but first you need to attempt to recover your data.

In order for forum members to give quality help and instruction, please read and then follow the Forum Rules,
the link is in red at the top of each page. Thanks, and good luck.

 

[marc davis]

Thanks for the reply.

Can you help me understand how the failure of one drive caused the corruption of the data on the other? I thought the point of merrored drives was to avoid that.

 

[BigDave]

Who said the data on the other drive was corrupted?

 

[BigDave]

Have you put the drive back into your server?

 

[marc davis]

Who said the data on the other drive was corrupted?

"Marc, at this point you should consider your data is gone."

I'm sorry, I thought that is what you were saying.

 

[marc davis]

Ok, With both drives plugged in you can see that one can't even get the serial number. Also during the NAS boot there are reams of error messages.

Thanks again for you help.

 

[SweetAndLow]

So you should be able to access your data now. Can you please back it up.

Also what is the output of smartctl -a /dev/ada0 and smartctl -a /dev/ada1, please provide the output in code tags. Do you run smart tests it scrubs on a schedule?

 

[BigDave]

Marc, SweetAndLo is asking you to access the smart output for both your drives
so you can figure out which one to replace.
Not showing a serial number for a paticular drive does not mean that
that drive is the *bad* drive. But as has been strongly suggested, back up
your data before doing anything else. At this point, if you were to lose the
second drive (the mirror), your data will be gone.

 

[marc davis]

CGiT and SAL

I ran the smart commands in the console: (the two drives are same make and model)
ada1 reports "unable to detect devices type"
ada0 reports "SMART overall health" .... FAILED! (seems bad )
The vendor specific error is Raw_read_error_rate (very high value)
and Reallocated Sector Count = 2042

So it seems like both the drives failed. I have most of the data backed other places so no major loss.

Thanks for your help.

Sorry I don't know how to get a full dump of the command off the machine to poast here.

 

[BigDave]

Sorry I don't know how to get a full dump of the command off the machine to poast here.

SSH is the method most used for copy/paste (pasted within code [] tags to preserve formatting) to the forum posts.
The manual has instruction regarding the SSH service built into FreeNAS and you would need client software
installed on the computer used to access your server (I use Bitvise now, after switching from PuTTY).
Search google to learn more about this, but to get help here in the forum, being able to post CLI output from SSH is an absolute must.

So it seems like both the drives failed. I have most of the data backed other places so no major loss.

Here is where I recommend you START, so that you understand basic best practices regarding the use of the ZFS file system
and can be better prepared for events like you've recently experienced.
Slideshow explaining VDev, zpool, ZIL and L2ARC and other newbie mistakes!
Don't let the date on the original post fool you, it has been updated many times since cyberjock first posted the thread.

In order for forum members to give quality help and instruction, please read and then follow the Forum Rules,
the link is in red at the top of each page. Thanks, and good luck.

You seemed to have forgotten to post your system's hardware specs, please do this before asking for further help.
Thanks, Dave