How to find out which device caused an error

[imikejackson]

In my nightly email I got something like the following:

Checking status of zfs pools:
pool: Storage
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://www.sun.com/msg/ZFS-8000-9P
scrub: resilver completed after 0h19m with 0 errors on Tue Jul 3 03:20:52 2012
config:

NAME STATE READ WRITE CKSUM
Storage ONLINE 0 0 0
raidz2 ONLINE 0 0 0
ada0p2 ONLINE 0 0 0
ada1p2 ONLINE 0 0 0
ada2p2 ONLINE 0 0 0
ada3p2 ONLINE 6.07K 30.6K 0 17.2G resilvered
ada4p2 ONLINE 0 0 0
ada5p2 ONLINE 0 0 0 11.5G resilvered

So how do I find out which drive was the culprit? I had to hard reset the machine as the WebGUI would not respond and there doesn't seem to be a way to login as root or admin? No sudo. No su. Then I finally got back to work to use the console and select 'reboot' a million errors messages about vm_swap were output and just kept going and going. I let this go on for about 10 minutes before I finally just reset the thing.

In case I hosed the admin/root passwords is there a way to reset them from the console possibly?

Thanks for any help/clarifications. This was on a FreeNAS-8.0.4-RELEASE-p2-x64 (11367) server.

Mike J.

 

[paleoN]

It looks like you need to replace ada3 at the least and possibly ada5. I would check out the SMART data for those drives in particular, actually I would look at all the drives. Run the following against each drive when the NAS isn't being used, i.e. overnight:

Code:

smartctl -t long ada0
smartctl -t long ada1
smartctl -t long ada2
etc...

When finished check the status of each drive with:

Code:

smartctl -a /dev/adX

See this [thread=7699]thread[/thread] for the main things to look out for.

 

[survive]

Hi imikejackson,

The serial number of the drive will be in the output of "smartctl -a /dev/adX" so you can use that to match the device to the drive.

While you are in there replacing the drive it would be a good time to get a label maker & tag whichever "end" of the drive you can see with the last 4 digits of the serial number to make future swaps easier.

-Will