Greetings,
My server configuration
Type: SuperMicro SYS-6029U-E1CR25M
Processor: Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz
Memory: 12 x 32GB Samsung M393A4K40DB2-CVF
Storage:
2 x 256GB NVMe SSD, INTEL SSDPEKKA256GB
12 x 16TB HDD, Seagate ST16000NM001G-2KK103
TrueNAS-12.0-U3.1
- Boot: 1 x 2-way mirror (SSD)
- Pool: 2 x 6-wide RAIDZ2 data VDEVs
zpool status shows degraded zpool with one faulted disk
glabel status
smartctl output
I was wondering if those are transient errors and thus ran twice, the first run cleared WRITE value, the second run cleared READ value
the zpool status become
Is this a real hardware failure? Any chances to bring it ONLINE without disk replacement?
Thanks in advance.
My server configuration
Type: SuperMicro SYS-6029U-E1CR25M
Processor: Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz
Memory: 12 x 32GB Samsung M393A4K40DB2-CVF
Storage:
2 x 256GB NVMe SSD, INTEL SSDPEKKA256GB
12 x 16TB HDD, Seagate ST16000NM001G-2KK103
TrueNAS-12.0-U3.1
- Boot: 1 x 2-way mirror (SSD)
- Pool: 2 x 6-wide RAIDZ2 data VDEVs
zpool status shows degraded zpool with one faulted disk
Code:
root@nas01-chq[~]# zpool status pool: boot-pool state: ONLINE scan: scrub repaired 0B in 00:00:02 with 0 errors on Wed Nov 10 03:45:02 2021 config: NAME STATE READ WRITE CKSUM boot-pool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 nvd0p2 ONLINE 0 0 0 nvd1p2 ONLINE 0 0 0 errors: No known data errors pool: truenaspool1 state: DEGRADED status: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the faulted device, or use 'zpool clear' to mark the device repaired. scan: scrub repaired 0B in 04:20:02 with 0 errors on Sun Nov 7 03:20:03 2021 config: NAME STATE READ WRITE CKSUM truenaspool1 DEGRADED 0 0 0 raidz2-0 ONLINE 0 0 0 gptid/b619a936-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/ba837a06-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/b9a27bb1-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/ba913a95-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/be9799ed-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/be888bc7-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 raidz2-1 DEGRADED 0 0 0 gptid/bd074641-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/bd13d975-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/c013c1f7-cf7f-11eb-a7dd-f8f21e518c34 FAULTED 3 581 0 too many errors gptid/c124c26b-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/c1af3982-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/c2c29171-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 errors: No known data errors
glabel status
Code:
root@nas01-chq[~]# glabel status Name Status Components gptid/3ff4ae9d-c3ed-11eb-ac8d-40a6b7221038 N/A nvd0p1 gptid/3fff8166-c3ed-11eb-ac8d-40a6b7221038 N/A nvd1p1 gptid/b619a936-cf7f-11eb-a7dd-f8f21e518c34 N/A da0p2 gptid/be9799ed-cf7f-11eb-a7dd-f8f21e518c34 N/A da1p2 gptid/be888bc7-cf7f-11eb-a7dd-f8f21e518c34 N/A da2p2 gptid/ba913a95-cf7f-11eb-a7dd-f8f21e518c34 N/A da3p2 gptid/ba837a06-cf7f-11eb-a7dd-f8f21e518c34 N/A da4p2 gptid/b9a27bb1-cf7f-11eb-a7dd-f8f21e518c34 N/A da5p2 gptid/c1af3982-cf7f-11eb-a7dd-f8f21e518c34 N/A da6p2 gptid/c2c29171-cf7f-11eb-a7dd-f8f21e518c34 N/A da7p2 gptid/c124c26b-cf7f-11eb-a7dd-f8f21e518c34 N/A da8p2 gptid/bd13d975-cf7f-11eb-a7dd-f8f21e518c34 N/A da9p2 gptid/bd074641-cf7f-11eb-a7dd-f8f21e518c34 N/A da10p2 gptid/c013c1f7-cf7f-11eb-a7dd-f8f21e518c34 N/A da11p2 gptid/400294bc-c3ed-11eb-ac8d-40a6b7221038 N/A nvd1p3 gptid/3ff824e9-c3ed-11eb-ac8d-40a6b7221038 N/A nvd0p3
smartctl output
Code:
root@nas01-chq[~]# smartctl -a /dev/da11 smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p6 amd64] (local build) Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Seagate Exos X16 Device Model: ST16000NM001G-2KK103 Serial Number: ZL28A8YN LU WWN Device Id: 5 000c50 0c6ff3657 Firmware Version: SN03 User Capacity: 16,000,900,661,248 bytes [16.0 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 7200 rpm Form Factor: 3.5 inches Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-4 (minor revision not indicated) SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Wed Nov 10 11:59:11 2021 EST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 567) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: (1428) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x70bd) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 083 064 044 Pre-fail Always - 205752312 3 Spin_Up_Time 0x0003 090 090 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 21 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 078 060 045 Pre-fail Always - 57985720 9 Power_On_Hours 0x0032 091 091 000 Old_age Always - 8544 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 21 18 Head_Health 0x000b 100 100 050 Pre-fail Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 073 049 040 Old_age Always - 27 (Min/Max 22/34) 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 9 193 Load_Cycle_Count 0x0032 096 096 000 Old_age Always - 8197 194 Temperature_Celsius 0x0022 027 040 000 Old_age Always - 27 (0 19 0 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 200 Pressure_Limit 0x0023 100 100 001 Pre-fail Always - 0 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 3485h+48m+15.111s 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 12913677634 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 22478272832 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.
I was wondering if those are transient errors and thus ran twice, the first run cleared WRITE value, the second run cleared READ value
Code:
zpool clear truenaspool1 gptid/c013c1f7-cf7f-11eb-a7dd-f8f21e518c34
the zpool status become
Code:
root@nas01-chq[~]# zpool status -xv pool: truenaspool1 state: DEGRADED status: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the faulted device, or use 'zpool clear' to mark the device repaired. scan: scrub repaired 0B in 04:20:02 with 0 errors on Sun Nov 7 03:20:03 2021 config: NAME STATE READ WRITE CKSUM truenaspool1 DEGRADED 0 0 0 raidz2-0 ONLINE 0 0 0 gptid/b619a936-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/ba837a06-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/b9a27bb1-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/ba913a95-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/be9799ed-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/be888bc7-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 raidz2-1 DEGRADED 0 0 0 gptid/bd074641-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/bd13d975-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/c013c1f7-cf7f-11eb-a7dd-f8f21e518c34 FAULTED 0 0 0 too many errors gptid/c124c26b-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/c1af3982-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 gptid/c2c29171-cf7f-11eb-a7dd-f8f21e518c34 ONLINE 0 0 0 errors: No known data errors
Is this a real hardware failure? Any chances to bring it ONLINE without disk replacement?
Thanks in advance.
Last edited: