Currently unreadable (pending) sectors

[Jasonachmed]

I know this is a common question but how do I know if my drive need replaced?

I have 10x3tb drives in raidz 2 and i am getting the below alert. When I run a SMART test it comes back with SUCCESS.
Do I need to replace this drive or do I need to run something to repair bad sectors?

Code:

CRITICAL
Device: /dev/ada5, 6 Currently unreadable (pending) sectors.
2021-03-15 14:14:53

Code:

@truenas[~]# zpool status
  pool: Main
 state: ONLINE
  scan: scrub repaired 0B in 07:24:18 with 0 errors on Sun Mar  7 07:24:19 2021
config:

        NAME                                            STATE     READ WRITE CKSUM
        Main                                            ONLINE       0     0 0
          raidz2-0                                      ONLINE       0     0 0
            gptid/b512616f-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/b5e351d1-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/b6a85d1a-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/b777622e-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/b8512d2e-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/b91a62e1-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/b9e20c9e-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/bab16311-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/bb7a63ba-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0
            gptid/bc48e59f-c796-11e5-922c-305a3a48b6cd  ONLINE       0     0 0

errors: No known data errors

  pool: boot-pool
 state: ONLINE
  scan: scrub repaired 0B in 00:01:03 with 0 errors on Wed Mar 10 03:46:03 2021
config:

        NAME        STATE     READ WRITE CKSUM
        boot-pool   ONLINE       0     0     0
          da0p2     ONLINE       0     0     0

errors: No known data errors

Code:

@truenas[~]# smartctl -a /dev/ada5
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N4AETD4V
LU WWN Device Id: 5 0014ee 20c916c30
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Mar 15 17:40:13 2021 SAST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (38880) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 390) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       43
  3 Spin_Up_Time            0x0027   188   180   021    Pre-fail  Always       -       5575
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       420
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   040   040   000    Old_age   Always       -       44439
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       418
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       333
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       497
194 Temperature_Celsius     0x0022   116   109   000    Old_age   Always       -       34
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       6
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       6

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     44184         -
# 2  Short offline       Completed without error       00%     43995         -
# 3  Extended offline    Aborted by host               90%     43995         -
# 4  Short offline       Completed without error       00%     43995         -
# 5  Short offline       Completed without error       00%     43990         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

 

[sretalla]

Yes, it's on its way out. Organize the replacement.

If you really want to cause yourself more trouble in the name of "saving money" (while risking your data), go ahead and look at finding the bad blocks and writing zeros to them (or as suggested in the link below, just do it for the whole disk).

Here's a link to a suggestion on how to do that:

repair bad block

how do I repair bad sectors in FreeBSD? I use SATAII

forums.FreeBSD.org

 

[Hellione]

you can´t repair bad sectors. thats all bullshit. if there are more than zero reallocated or pending sectors, insert a new drive, and everything is fine.

 

[sretalla]

you can´t repair bad sectors. thats all bull****. if there are more than zero reallocated or pending sectors, insert a new drive, and everything is fine.

Maybe it wasn't clear that I was also making this point. (But I was... Perhaps with too much sarcasm).

 

[joeschmuck]

You have over 5 years of runtime on this drive so it's not surprising it's failing and as others have said, it's probably time to replace that hard drive and to do it when you want vice being forced to do it once it fails. Also if all your drives are of similar age then I'd check all of them one at a time.

you can´t repair bad sectors. thats all bull****.

That is true, you cannot repair bad sectors but you can map bad sectors out which is what @sretalla is saying, he did not say it would repair the bad sectors and he also recommended replacing the drive an this option will just stay off a little time. At this point in time you have 6 pending sectors and I would expect that after running badblocks or just writing to the entire drive a few times, those sectors would be mapped out (in ID 5) or maybe they would be fine and ID 197 would drop to zero. However I would honestly expect the count to go up as there is very possibly a surface defect with a drive with so many hours.

Of course you don't need to write to the drive or run badblocks as this will happen over time anyway, it's just a faster way to figure out how bad your drive might be.

 

[Jasonachmed]

Thanks for the replies I was just curious if it was worth trying to fix.
I have already organized a new drive and will swap it out this weekend. I should be ok till then?

Is the rule to always replace a drive when it says "Currently unreadable (pending) sectors" or do situations exist where it is fixable?

 

[Jasonachmed]

You have over 5 years of runtime on this drive so it's not surprising it's failing and as others have said, it's probably time to replace that hard drive and to do it when you want vice being forced to do it once it fails. Also if all your drives are of similar age then I'd check all of them one at a time.

That is true, you cannot repair bad sectors but you can map bad sectors out which is what @sretalla is saying, he did not say it would repair the bad sectors and he also recommended replacing the drive an this option will just stay off a little time. At this point in time you have 6 pending sectors and I would expect that after running badblocks or just writing to the entire drive a few times, those sectors would be mapped out (in ID 5) or maybe they would be fine and ID 197 would drop to zero. However I would honestly expect the count to go up as there is very possibly a surface defect with a drive with so many hours.

Of course you don't need to write to the drive or run badblocks as this will happen over time anyway, it's just a faster way to figure out how bad your drive might be.

I got all the drives at the same time. That's an expensive thought. How would I "check all of them one at a time"?

Would i run smartctl -a /dev/x for each drive? and what would i be looking for in the results? Even the drive im going to replace says it passes if im reading the results correctly?

 

[sretalla]

Would i run smartctl -a /dev/x for each drive?

Exactly.

what would i be looking for in the results?

ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 43
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 6
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 6

1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 43
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 6
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 6

Basically for non-zero values in:
1 Raw_Read_Error_Rate
5 Reallocated_Sector_Ct
7 Seek_Error_Rate
196 Reallocated_Event_Count
197 Current_Pending_Sector
198 Offline_Uncorrectable
199 UDMA_CRC_Error_Count
200 Multi_Zone_Error_Rate

A Successful SMART test doesn't mean a healthy disk.

 

[joeschmuck]

Even the drive im going to replace says it passes if im reading the results correctly?

So your currently failing drive is fine technically but SMART was originally designed to try and provide the user a 24 hour warning period in a ideal situation. Of course that doesn't always happen but in your case you have notification of pending doom and you can plan around it. If you give it enough time I would expect the value in ID5 to start incrementing meaning a specific sector could not be used and was removed from the sector map, and a SMART Extended/Long test would not complete.

Check your other drives and ensure you are running SMART Long and Short tests like you are doing for the failing drive and if they show any errors then I would recommend replacing those as well. There is a link in my signature for hard drive troubleshooting, you might find it helpful but I would not try to make the drive better, it's time for the old drive to die with a little respect.

 

[Jasonachmed]

Check your other drives and ensure you are running SMART Long and Short tests like you are doing for the failing drive and if they show any errors then I would recommend replacing those as well. There is a link in my signature for hard drive troubleshooting

Thanks for the link I have added a task for short and long test.

All other drives seemed fine until I ran a short test now ada7 is showing "Completed: read failure" under SMART Self-test but all the SMART Attributes seem to be 0. Do I need to replace this drive?

If i need to replace the drive I have 1 drive that I was going to use to replace ada5 and have ordered a 2nd one that should take a few days to arrive. Should I replace ada5 or ada7 first?

I am running a long test on all drives now to see if anything else turns up.

Code:

@truenas[~]#  smartctl -a /dev/ada7
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N7SCAYJ7
LU WWN Device Id: 5 0014ee 20c929596
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Thu Mar 18 20:08:45 2021 SAST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      ( 118) The previous self-test completed having
                                        the read element of the test failed.
Total time to complete Offline
data collection:                (40380) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 405) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   188   179   021    Pre-fail  Always       -       5575
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       418
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   040   040   000    Old_age   Always       -       44514
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       418
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       333
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       476
194 Temperature_Celsius     0x0022   114   105   000    Old_age   Always       -       36
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   100   253   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed: read failure       60%     44513         1565563352
# 2  Short offline       Completed: read failure       60%     44505         1565563352

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

 

[Jasonachmed]

With the long test done ada7 now shows a Multi_Zone_Error_Rate

ada5
- 5 Multi_Zone_Error_Rate
- 6 Current_Pending_Sector
- Self Test "Completed without error"

ada7
- 1 Multi_Zone_Error_Rate
- Self Test "Completed: read failure"

ID 200 MultiZone Error Rate can be the cause of a drive failure although a value in this location doesn't always mean it's the fault. It is notable if there are no other failing indications.

Does ada7 needs to be replaced?

If ada7 needs replaced Im running Z2 so its probably risky to replace 2 drives at the same time.
Which drive is in worse condition and should be replace first ada5 or ada7?

Code:

@truenas[~]# smartctl -a /dev/ada5
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N4AETD4V
LU WWN Device Id: 5 0014ee 20c916c30
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Mar 20 00:11:39 2021 SAST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (38880) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 390) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       43
  3 Spin_Up_Time            0x0027   188   180   021    Pre-fail  Always       -       5583
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       421
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   040   040   000    Old_age   Always       -       44528
10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       419
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       334
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       497
194 Temperature_Celsius     0x0022   118   109   000    Old_age   Always       -       32
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       6
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       5

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     44525         -
# 2  Short offline       Completed without error       00%     44513         -
# 3  Short offline       Completed without error       00%     44505         -
# 4  Extended offline    Completed without error       00%     44184         -
# 5  Short offline       Completed without error       00%     43995         -
# 6  Extended offline    Aborted by host               90%     43995         -
# 7  Short offline       Completed without error       00%     43995         -
# 8  Short offline       Completed without error       00%     43990         -

SMART Selective self-test log data structure revision number 1
SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Code:

@truenas[~]# smartctl -a /dev/ada7
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N7SCAYJ7
LU WWN Device Id: 5 0014ee 20c929596
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Mar 20 00:07:24 2021 SAST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      ( 113) The previous self-test completed having
                                        the read element of the test failed.
Total time to complete Offline
data collection:                (40380) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 405) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   188   179   021    Pre-fail  Always       -       5575
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       419
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   040   040   000    Old_age   Always       -       44528
10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       419
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       334
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       476
194 Temperature_Celsius     0x0022   115   105   000    Old_age   Always       -       35
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       1

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed: read failure       10%     44525         1565563352
# 2  Short offline       Completed: read failure       60%     44513         1565563352
# 3  Short offline       Completed: read failure       60%     44505         1565563352

SMART Selective self-test log data structure revision number 1
SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

 

[danb35]

Is the rule to always replace a drive when it says "Currently unreadable (pending) sectors" or do situations exist where it is fixable?

I'm going to be a slightly contrary voice here and say that, if there are no other evident problems (which there are in your case), a single-digit number of pending sectors that is stable isn't cause, for me, for immediate replacement. Failing SMART tests are cause for immediate replacement. Increasing number of pending or uncorrectable sectors call for immediate replacement. Raw read error rate > 0 for a non-Seagate disk probably calls for immediate replacement. But a half-dozen pending sectors, if that number is stable, wouldn't result in me replacing the disk--though I'd be keeping an eye on it. It isn't fixable, in any meaningful sense, but it won't necessarily harm your data in the short term.

And you should have the SMART service enabled, and regular tests scheduled, so you'll catch these issues sooner.

 

[Etorix]

If ada7 needs replaced Im running Z2 so its probably risky to replace 2 drives at the same time.
Which drive is in worse condition and should be replace first ada5 or ada7?

That depends whether you physically replace the drive first, and then resilver, or whether you plug in new drive(s) and then replace drives through the GUI, the "old" drives still being present. In the first case, do it one at time, ada7 appearing as the most urgent. In the second case, there is no loss of redundancy and you can do any number of drives at the same time.

 

[joeschmuck]

My friendly advice for your two different problems is basically the same given the amount of hours you have on those drives. They are showing signs of age and are telling you they are going to fail. Heed the warning and replace the drives one at a time as you can afford them but I wouldn't wait 6 months to start the process. If it's one drive every month or so then that is moving in the right direction. If you prefer to roll the dice then at least ensure you have a backup of all your data in a safe place.

And I agree, ada7 is the most critical drive to replace based on the data you posted becasue it cannot read the entire drive but the way the read test works is it starts at 0 and reads the data, it stops when it can't read the data and does not try to skip over and continue reading, so it hits a hard stopping point for the drive test. Ada5 can read the drive hand has a few currently temporary errors, they become real when ID5 increments.