My main pool is currently in a degraded state due to an "unrecoverable error" in one of the drives.
Data1 is 4 disks of 4TB drives in Raidz1. WD Red Pro's. I am running AMD-8350x with 16 GB non-ECC ram. Running TrueNAS-SCALE-22.02.2.1.
`zpool status -v` shows 2 checksum errors in the drive
Truenas tells me that the device is /dev/sdc so `smartctl -a /dev/sdc` shows:
Reading the suggested openzfs docs post (https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P/) it seems like this is a minor issue, but I just wanted to check that I'm ok to `zpool clear` the drive and not replace the drive? I should obviously check the cables and whatnot as well.
I have already:
- Scrubbed the pool
- Performed a long SMART test on the drive
- Restarted the machine
The only weird thing is that before I restarted the number of checksum errors was different. Before it was reading I/O errors on the device but now it is reading checksum errors on the device. This error did occur after I had left the machine off for 3 days and whilst I was writing data to an SMB share.
Thanks for any help.
Data1 is 4 disks of 4TB drives in Raidz1. WD Red Pro's. I am running AMD-8350x with 16 GB non-ECC ram. Running TrueNAS-SCALE-22.02.2.1.
`zpool status -v` shows 2 checksum errors in the drive
Code:
root@truenas[~]# zpool status -v
pool: Data1
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
scan: resilvered 25.4G in 00:07:17 with 0 errors on Mon Aug 8 21:16:07 2022
config:
NAME STATE READ WRITE CKSUM
Data1 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
f514485f-c6e0-4317-984c-25a1ecaf4645 ONLINE 0 0 0
21e09f5f-a64f-479c-8fe5-19c5be56c8b3 ONLINE 0 0 0
0a5a59a2-a8a0-4339-a00b-d53644064a13 ONLINE 0 0 0
8849c89a-a52f-4b2e-852b-37e5139a1744 ONLINE 0 0 2
errors: No known data errorsTruenas tells me that the device is /dev/sdc so `smartctl -a /dev/sdc` shows:
Code:
root@truenas[~]# smartctl -a /dev/sdc
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.10.120+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Red
Device Model: WDC WD40EFRX-68N32N0
Serial Number: WD-WCC7K6RJ1ANH
LU WWN Device Id: 5 0014ee 2bbfc825a
Firmware Version: 82.00A82
User Capacity: 4,000,787,030,016 bytes [4.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Form Factor: 3.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-3 T13/2161-D revision 5
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Tue Aug 9 08:08:58 2022 EEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (43080) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 457) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x303d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 180 161 021 Pre-fail Always - 6000
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 766
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 078 078 000 Old_age Always - 16122
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 713
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 8
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 766
194 Temperature_Celsius 0x0022 112 107 000 Old_age Always - 38
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 16121 -
# 2 Extended offline Completed without error 00% 16097 -
# 3 Short offline Completed without error 00% 16081 -
# 4 Short offline Completed without error 00% 16057 -
# 5 Short offline Completed without error 00% 16033 -
# 6 Short offline Completed without error 00% 16009 -
# 7 Short offline Completed without error 00% 15985 -
# 8 Short offline Completed without error 00% 15962 -
# 9 Short offline Completed without error 00% 15938 -
#10 Short offline Completed without error 00% 15555 -
#11 Extended offline Completed without error 00% 15465 -
#12 Short offline Completed without error 00% 15347 -
#13 Short offline Completed without error 00% 14627 -
#14 Short offline Completed without error 00% 13906 -
#15 Short offline Completed without error 00% 13162 -
#16 Short offline Completed without error 00% 12443 -
#17 Short offline Completed without error 00% 11701 -
#18 Short offline Completed without error 00% 10981 -
#19 Short offline Completed without error 00% 10238 -
#20 Short offline Completed without error 00% 9567 -
#21 Short offline Completed without error 00% 8824 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.Reading the suggested openzfs docs post (https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P/) it seems like this is a minor issue, but I just wanted to check that I'm ok to `zpool clear` the drive and not replace the drive? I should obviously check the cables and whatnot as well.
I have already:
- Scrubbed the pool
- Performed a long SMART test on the drive
- Restarted the machine
The only weird thing is that before I restarted the number of checksum errors was different. Before it was reading I/O errors on the device but now it is reading checksum errors on the device. This error did occur after I had left the machine off for 3 days and whilst I was writing data to an SMB share.
Thanks for any help.
