blueether
Patron
- Joined
- Aug 6, 2018
- Messages
- 259
I started doing a disk replace with a old-ish HDD that I had spare as one was degraded with ~1.5k CKSUM
Now that the resilver has started the error count is climbing and the old-ish drive is throwing unreadable (pending) sectors
I have backups of important data on another box, just not the jails
Now that the resilver has started the error count is climbing and the old-ish drive is throwing unreadable (pending) sectors
I have backups of important data on another box, just not the jails
Code:
Oct 12 07:46:55 freenas ada2 at ahcich2 bus 0 scbus2 target 0 lun 0 Oct 12 07:46:55 freenas ada2: <ST2000DM001-1E6164 CC43> ATA8-ACS SATA 3.x device Oct 12 07:46:55 freenas ada2: Serial Number Z1E2K9TC Oct 12 07:46:55 freenas ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) Oct 12 07:46:55 freenas ada2: 1907729MB (3907029168 512 byte sectors) Oct 12 07:46:55 freenas ada2: quirks=0x1<4K> Oct 12 07:49:33 freenas ZFS: vdev state changed, pool_guid=14900235701281331664 vdev_guid=15589422846635699418 Oct 12 09:29:47 freenas smartd[36233]: Device: /dev/ada2, 8 Currently unreadable (pending) sectors Oct 12 09:29:47 freenas smartd[36233]: Device: /dev/ada2, 8 Offline uncorrectable sectors undefinedOct 12 09:34:45 freenas smartd[36233]: Device: /dev/ada2, 8 Currently unreadable (pending) sectors Oct 12 09:34:45 freenas smartd[36233]: Device: /dev/ada2, 8 Offline uncorrectable sectors Oct 12 09:39:46 freenas smartd[36233]: Device: /dev/ada2, 8 Currently unreadable (pending) sectors Oct 12 09:39:46 freenas smartd[36233]: Device: /dev/ada2, 8 Offline uncorrectable sectors Oct 12 09:44:46 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors (changed +8) Oct 12 09:44:46 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors (changed +8) Oct 12 09:49:46 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 09:49:46 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 09:54:46 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 09:54:46 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 09:59:47 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 09:59:47 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 10:04:47 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 10:04:47 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 10:09:46 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 10:09:46 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 10:14:45 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 10:14:45 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 10:19:47 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 10:19:47 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 10:24:46 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 10:24:46 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 10:29:46 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 10:29:46 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 10:34:46 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 10:34:46 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors Oct 12 10:39:46 freenas smartd[36233]: Device: /dev/ada2, 16 Currently unreadable (pending) sectors Oct 12 10:39:46 freenas smartd[36233]: Device: /dev/ada2, 16 Offline uncorrectable sectors
Code:
root@freenas:~ # zpool status -v IronWolf
pool: IronWolf
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Sat Oct 12 07:49:33 2019
586G scanned at 60.6M/s, 133G issued at 13.8M/s, 1.56T total
133G resilvered, 8.34% done, 1 days 06:14:24 to go
config:
NAME STATE READ WRITE CKSUM
IronWolf DEGRADED 0 0 1.67K
mirror-0 DEGRADED 0 0 6.66K
gptid/79d283da-979d-11e8-a256-94de80b75a94 ONLINE 0 0 6.66K
replacing-1 DEGRADED 0 0 4.82K
gptid/7a8be22e-979d-11e8-a256-94de80b75a94 DEGRADED 0 0 6.66K too many errors
gptid/db372b8d-ec57-11e9-9162-001b21981f60 ONLINE 0 0 4.82K
errors: Permanent errors have been detected in the following files:
IronWolf/Media:<0x0>
Code:
root@freenas:~ # smartctl -a /dev/ada1
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Seagate IronWolf
Device Model: ST2000VN004-2E4164
Serial Number: Z5248MM7
LU WWN Device Id: 5 000c50 0afef4c46
Firmware Version: SC60
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5900 rpm
Form Factor: 3.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sat Oct 12 11:07:06 2019 NZDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 128) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 277) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x10bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 112 099 006 Pre-fail Always - 42188216
3 Spin_Up_Time 0x0003 097 095 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 099 099 020 Old_age Always - 1464
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 080 060 030 Pre-fail Always - 107899659
9 Power_On_Hours 0x0032 089 089 000 Old_age Always - 10109
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 120
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
188 Command_Timeout 0x0032 100 099 000 Old_age Always - 1
189 High_Fly_Writes 0x003a 001 001 000 Old_age Always - 112
190 Airflow_Temperature_Cel 0x0022 069 048 045 Old_age Always - 31 (Min/Max 27/37)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 53
193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 1474
194 Temperature_Celsius 0x0022 031 052 000 Old_age Always - 31 (0 15 0 0 0)
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 10109 -
# 2 Short offline Completed without error 00% 10108 -
# 3 Short offline Completed without error 00% 10107 -
# 4 Short offline Completed without error 00% 10106 -
# 5 Short offline Completed without error 00% 10106 -
# 6 Short offline Completed without error 00% 10105 -
# 7 Short offline Completed without error 00% 10104 -
# 8 Short offline Completed without error 00% 10103 -
# 9 Short offline Completed without error 00% 10102 -
#10 Short offline Completed without error 00% 10101 -
#11 Short offline Completed without error 00% 10100 -
#12 Short offline Completed without error 00% 10099 -
#13 Short offline Completed without error 00% 10098 -
#14 Short offline Completed without error 00% 10097 -
#15 Short offline Completed without error 00% 10096 -
#16 Short offline Completed without error 00% 10095 -
#17 Short offline Completed without error 00% 10095 -
#18 Short offline Completed without error 00% 10094 -
#19 Short offline Completed without error 00% 10094 -
#20 Short offline Completed without error 00% 10093 -
#21 Short offline Completed without error 00% 10092 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Code:
root@freenas:~ # smartctl -a /dev/ada2
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda 7200.14 (AF)
Device Model: ST2000DM001-1E6164
Serial Number: Z1E2K9TC
LU WWN Device Id: 5 000c50 04f5afad3
Firmware Version: CC43
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 7200 rpm
Form Factor: 3.5 inches
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA8-ACS T13/1699-D revision 4
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sat Oct 12 11:08:24 2019 NZDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 249) Self-test routine in progress...
90% of test remaining.
Total time to complete Offline
data collection: ( 592) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x3081) SCT Status supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 112 099 006 Pre-fail Always - 132726480
3 Spin_Up_Time 0x0003 095 095 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 083 083 020 Old_age Always - 17982
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 067 060 030 Pre-fail Always - 21502710673
9 Power_On_Hours 0x0032 086 086 000 Old_age Always - 12688
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 083 083 020 Old_age Always - 17898
183 Runtime_Bad_Block 0x0032 099 099 000 Old_age Always - 1
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
188 Command_Timeout 0x0032 100 099 000 Old_age Always - 2 3 23
189 High_Fly_Writes 0x003a 099 099 000 Old_age Always - 1
190 Airflow_Temperature_Cel 0x0022 061 033 045 Old_age Always In_the_past 39 (5 49 39 20 0)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 092 092 000 Old_age Always - 16211
193 Load_Cycle_Count 0x0032 083 083 000 Old_age Always - 34183
194 Temperature_Celsius 0x0022 039 067 000 Old_age Always - 39 (0 9 0 0 0)
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 40
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 40
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 1520h+44m+14.992s
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 7568979139
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 10795411086
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Self-test routine in progress 90% 12688 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
