Degraded pool but no alerts on login? Thoughts?

[godsfshrmn]

Received email alert about pool being degraded

Code:

New alerts:
* Pool freenas state is DEGRADED: One or more devices has been removed by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state.
The following devices are not healthy:
Disk ATA ST3000VN000-1H41 S300PN3J is REMOVED


Current alerts:
* Pool freenas state is DEGRADED: One or more devices has been removed by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state.
The following devices are not healthy:
Disk ATA ST3000VN000-1H41 S300PN3J is REMOVED

Less than a minute later:

Code:

The following alert has been cleared:
* Pool freenas state is DEGRADED: One or more devices has been removed by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state.
The following devices are not healthy:
Disk ATA ST3000VN000-1H41 S300PN3J is REMOVED

I log in to replace with a spare drive, but there are no alerts or a cleared alert that was from this email.

zpool status

Code:

pool: freenas
 state: ONLINE
  scan: resilvered 7.55M in 00:00:03 with 0 errors on Tue Feb  8 08:21:26 2022
config:

        NAME                                            STATE     READ WRITE CKSUM
        freenas                                         ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            gptid/1693403a-f961-11e3-892d-0cc47a052893  ONLINE       0     0     0
            gptid/16e784cc-f961-11e3-892d-0cc47a052893  ONLINE       0     0     0
            gptid/5e9e9f4d-6f1e-11e7-ba8c-0cc47a052893  ONLINE       0     0     0
            gptid/1791d445-f961-11e3-892d-0cc47a052893  ONLINE       0     0     0
            gptid/17ed23a4-f961-11e3-892d-0cc47a052893  ONLINE       0     0     0
            gptid/18400dec-f961-11e3-892d-0cc47a052893  ONLINE       0     0     0

Bingo correct? Resilvered a few sectors?

Last smartctl however does not seem to suggest a problem? Maybe I am not reading correctly?
Note Power on hours--- drives in this pool are now over 7 years old. Given the error and age, I'm going to replace the drive anyways. Thoughts on these reports?

Running a Long test currently. I think the RAW_READ_ERROR_RATE is way too high and that alone indicates need for replacement? Trying to learn a bit here and be proactive. Thanks

Code:

=== START OF INFORMATION SECTION ===
Model Family:     Seagate NAS HDD
Device Model:     ST3000VN000-1H4167
Serial Number:    S300PN3J
LU WWN Device Id: 5 000c50 074af9fdd
Firmware Version: SC44
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5900 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Feb  8 09:09:53 2022 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   114   099   006    Pre-fail  Always       -       77746752
  3 Spin_Up_Time            0x0003   092   091   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       151
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   084   060   030    Pre-fail  Always       -       9216369236
  9 Power_On_Hours          0x0032   027   027   000    Old_age   Always       -       64696
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       151
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   099   000    Old_age   Always       -       8590065666
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   078   059   045    Old_age   Always       -       22 (Min/Max 20/27)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       86
193 Load_Cycle_Count        0x0032   093   093   000    Old_age   Always       -       14083
194 Temperature_Celsius     0x0022   022   041   000    Old_age   Always       -       22 (0 15 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     64695         -
# 2  Extended offline    Completed without error       00%     64287         -
# 3  Extended offline    Completed without error       00%     63951         -
# 4  Short offline       Completed without error       00%     63634         -
# 5  Extended offline    Completed without error       00%     63545         -
# 6  Short offline       Completed without error       00%     63468         -

 

[Alecmascot]

Running a Long test currently. I think the RAW_READ_ERROR_RATE is way too high and that alone indicates need for replacement? Trying to learn a bit here and be proactive.

It is a Seagate drive and that count is not what it seems. It is not a simple error count:

3.1 Attribute ID 1: Raw Error Rate
Normalized Raw Error Rate = 10 * log10(NumberOfSectorsTransferredToOrFromHost * 512 * 8 /
(Number of sectors requiring retries))
Where the factor of 512*8 is to convert from sectors to bits. The attribute value is only
computed when the number of bits in the "transferred bits" count is in the range 1010 to 1012.
The counts are cleared when Number Of Bits Transferred To Or From Host > 1012.
“Number of sectors requiring retries” does not count free retry or hidden retry.
Normalized Raw Error Rate is evaluated to a number between 1 and 166.
Raw Usage
Raw [3 – 0] = Number of sector reads
Raw [6 - 4] = Number of read errors

Maybe you are more of a mathematician than I am ?