Offline uncorrectable errors

[Magnus33]

Now there are more then a few posts on this but this one a tad different i think.

Recently moved to a new house and after moving one of the nas drives is showing offline correctable errors.
The number remained the same and not increased in what appears the month since the move.

Data backed up so no worries there but i like to see if i can't get the drive to mark the sectors as bad and remove the warning.

Now here the minor issue the ac presently being worked on so the heat causing my ms to flare so iam having trouble focusing.
Iam sitting here looking at freenas knowing i know how to do this but for the life of me i can't remember how lol.

So whats the simple non destructive way of making freenas do the job.

Thanks :)

 

[dlavigne]

one of the nas drives is showing offline correctable errors.

Where? /var/log/messages, zpool status, someplace else?

i like to see if i can't get the drive to mark the sectors as bad and remove the warning.

Which warning?

Also, post the system build (from System -> Information) and hardware specs as per the forum rules.

 

[Magnus33]

Errors ( hasn't changed in anyway likely do to the idiot movers )

Code:

Jul 25 00:00:00 freenas newsyslog[99893]: logfile turned over due to size>100K
Jul 25 00:00:00 freenas syslog-ng[1435]: Configuration reload request received, reloading configuration;
Jul 25 00:05:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 00:05:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 00:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 00:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 01:05:20 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 01:05:20 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 01:35:14 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 01:35:14 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 02:05:03 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 02:05:03 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 02:35:09 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 02:35:09 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 03:04:24 freenas kernel: pid 17544 (Plex Media Scanner), uid 972: exited on signal 11
Jul 25 03:05:06 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 03:05:06 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 03:35:09 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 03:35:09 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 04:05:09 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 04:05:09 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 04:35:12 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 04:35:12 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 05:05:06 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 05:05:06 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 05:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 05:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 06:05:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 06:05:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 06:35:20 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 06:35:20 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 07:05:20 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 07:05:20 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 07:35:20 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 07:35:20 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 08:05:20 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 08:05:20 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 08:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 08:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 09:00:18 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs snapshot "Fusion@auto-20160725.0900-2w"
Jul 25 09:00:19 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs snapshot "MEDIA@auto-20160725.0900-2w"
Jul 25 09:00:19 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs snapshot "Backup@auto-20160725.0900-2w"
Jul 25 09:00:19 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs snapshot "Repository@auto-20160725.0900-2w"
Jul 25 09:00:20 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs snapshot "LORE@auto-20160725.0900-2w"
Jul 25 09:00:20 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs snapshot "Alpha@auto-20160725.0900-2w"
Jul 25 09:00:20 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs destroy -r -d "Fusion@auto-20160711.0900-2w"
Jul 25 09:00:20 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs destroy -r -d "Repository@auto-20160711.0900-2w"
Jul 25 09:00:20 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs destroy -r -d "Alpha@auto-20160711.0900-2w"
Jul 25 09:00:20 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs destroy -r -d "LORE@auto-20160711.0900-2w"
Jul 25 09:00:21 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs destroy -r -d "Backup@auto-20160711.0900-2w"
Jul 25 09:00:21 freenas autosnap.py: [tools.autosnap:61] Popen()ing: /sbin/zfs destroy -r -d "MEDIA@auto-20160711.0900-2w"
Jul 25 09:05:15 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 09:05:15 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 09:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 09:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 10:05:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 10:05:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 10:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 10:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 11:05:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 11:05:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 11:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 11:35:21 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 12:05:20 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 12:05:20 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Jul 25 12:35:20 freenas smartd[2476]: Device: /dev/ada0, 24 Currently unreadable (pending) sectors
Jul 25 12:35:20 freenas smartd[2476]: Device: /dev/ada0, 24 Offline uncorrectable sectors
Stop refresh

Zfs status is normal

What error?
I only posted about the one error so how you got confused there i don't know.

System specs:
Build
FreeNAS-9.10-STABLE-201606270534 (dd17351)
Platform AMD FX-8370E Eight-Core Processor
Memory 16255MB
System Time Mon Jul 25 13:04:38 EDT 2016
Uptime 1:04PM up 2 days, 2:01, 0 users
Load Average 0.34, 0.54, 0.59


I do apologize about not getting all the info but when you having a ms attack or in this case a pseudo-exacerbation it gets really hard to focus.

 

[m0nkey_]

Please provide output of smartctl

Code:

smartctl -x /dev/ada0

 

[Magnus33]

Shell

Code:

0xbe-0xbf GPL VS 65535 Device vendor specific log
0xc0 GPL,SL VS 1 Device vendor specific log
0xe0 GPL,SL R/W 1 SCT Command/Status
0xe1 GPL,SL R/W 1 SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed: read failure 40% 13649 4074902832
# 2 Short offline Completed without error 00% 13628 -
# 3 Short offline Completed without error 00% 5296 -

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version: 3
SCT Version (vendor specific): 522 (0x020a)
SCT Support Level: 1
Device State: Active (0)
Current Temperature: 38 Celsius
Power Cycle Min/Max Temperature: 36/42 Celsius
Lifetime Min/Max Temperature: 17/51 Celsius
Under/Over Temperature Limit Count: 0/0

SCT Data Table command not supported

SCT Error Recovery Control command not supported

Device Statistics (GP/SMART Log 0x04) not supported

SATA Phy Event Counters (GP Log 0x11)
ID Size Value Description
0x000a 2 6 Device-to-host register FISes sent due to a COMRESET
0x0001 2 0 Command failed due to ICRC error
0x0003 2 0 R_ERR response for device-to-host data FIS
0x0004 2 0 R_ERR response for host-to-device data FIS
0x0006 2 0 R_ERR response for device-to-host non-data FIS
0x0007 2 0 R_ERR response for host-to-device non-data FIS

 

[cyberjock]

First, you have 24 bad sectors. This isn't a case of a single bad sector.

This disk is not in a condition where you had a single bad sector, you've got 2 dozen (and likely more as time goes on). It's really time to look at replacing the disk.

 

[Magnus33]

Yes apparently its been this way since the move and i just got around to checking.

Gotten no worse so i figure this was more likely due to the movers rather then a hardware fault.

Already have a wd red for replacement and data already backed up so no worries there.

Was afraid i have to take it out and run it through full 18 write to see if it maps out the sectors or gets worse.

Bummer :( least the heats let off today and the ac working.

 

[cyberjock]

If you're wanting to do it just to do it, you could try doing a dd write to the entire disk or using some disk wiping software that will write to every sector. That should clear those 24 bad sectors. BUT, if experience is any indication of how things will go, you'll wipe the disk and have hundreds of bad sectors, things going downhill, etc. :P

 

[Robert Trevellyan]

My approach would be to run badblocks on it. If it remaps successfully, without additional bad sectors appearing, keep it as a cold emergency spare. If not, recycle it.

when you having a ms attack or in this case a pseudo-exacerbation it gets really hard to focus.

Might not be the best time to take chances with your pool. Perhaps start with a scrub?

What is the pool structure (RAIDZ1, RAIDZ2 or ...)?

What does zpool status report?

 

[Magnus33]

Zpool fine and scrubs are set to run on their own although iam going to run it now just to be safe.

Iam going to do a full scan on it and see what turns up.

I had one drive fail and another wd green scan/replace the blocks and run fine ever since.

Its a crap shoot it could be isolated or a sign of failure.

This one straight storage without spare since its backed up to a large system.

 

[Robert Trevellyan]

What does zpool status report?

Zpool fine

Let me be more explicit. Please post the output of zpool status.

 

[Magnus33]

Let me do so as well. The drive is out of the system and been replaced with a wd red.

Posting the pool results when they were perfectly normal would have been pointless and requesting it again after i mentioned i was replacing the drive would sever no purpose.
The whole system in question as i mentioned is backed up so data was never in any danger of being lost.
I assume that got missed with everything that got posted.

The drive in question is being scanned to see if this is a isolated problem or a drive failure

Issue been dealt with and i thank everyone for their help. :)

On a side note i probably should have waited to post when the heat wasn't making me less then sharp.

 

[Robert Trevellyan]

I'm glad you have everything under control.

Members generally have a good reason for requesting the information they ask for. Taking zpool status as an example, beyond showing whether the pool is healthy or degraded, and whether there are any errors present, it also reveals the structure of the pool (number of drives, mirror vs RAIDZ1, RAIDZ2 etc). This can help members to give appropriate guidance on how to deal with problems. For example pulling a drive from a mirror or RAIDZ1 vdev would be much more risky than pulling a drive from a RAIDZ2 or RAIDZ3 vdev. It can also reveal issues that the poster is unaware of, e.g. a disk that has been accidentally striped into an otherwise healthy structure.

I'm a bit puzzled why you would ignore explicit requests for specific information from people trying to help you.

 

[Magnus33]

I wasn't exactly ignoring them as i was more channeling embarrassment and a trip to see a neurologist at the hospital.

Seems the MS has done enough damage to the myelin over the nerves in my melon that changes in heat by a few degrees start blocking the signal or slowing them.
Its a lot like being drunk without any of the enjoyment getting there.

Ever find your self staring at a keyboard and not remembering how to use it and then coming back after the ac working and looking at your own posts going wtf.

I was aware of everything that needed to be done and how to check it but at the time wasn't capable of processing it.

Seems life has a sense of the ironic as the over smart guy now becomes quite slow if over heated.

 

[Robert Trevellyan]

Sounds very challenging.

 

[Magnus33]

Its certainly a life changing.

Have to avoid the heat now and i suddenly enjoy winter a lot more.
No one ever said life was fair though and you just got to life at it as you deal with the hand your dealt.

Upside is apparently now my better half says iam a cheap date much to her amusement.