System Appears to be Dying

[Cosmas]

I just discovered tonight that one of my 10TB WDs is not registering with TrueNAS, not even at the BIOS level. I've removed it and tested it on my Windows box and it doesnt even show up there.

Beyond that though, I had a power outage as a few others have and I'm having alerts in the webGUI but .. its not for the missing 10TB, this appears to be for the TrueNAS OS SSD drive.

Am I right here? Should I buy a new SSD and try to clone it over? Or does the uploading of the config file still work?

Last login: Sat Sep 23 21:41:39 on pts/1
FreeBSD 12.2-RELEASE-p14 325282c09a5(HEAD) TRUENAS

TrueNAS (c) 2009-2022, iXsystems, Inc.
All rights reserved.
TrueNAS code is released under the modified BSD license with some
files copyrighted by (c) iXsystems, Inc.

For more information, documentation, help or support, go here:
http://truenas.com
Welcome to FreeNAS

Warning: settings changed through the CLI are not written to
the configuration database and will be reset on reboot.

root@freenas[~]# zpool status
pool: Volume1
state: DEGRADED
status: One or more devices has been removed by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using zpool online' or replace the device with
'zpool replace'.
scan: scrub repaired 0B in 02:54:18 with 0 errors on Sun Aug 20 02:54:18 2023
config:

NAME STATE READ WRITE CKSUM
Volume1 DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
gptid/674932a4-00a0-11e9-bf3a-e069950db0ca ONLINE 0 0 0
gptid/67dabecb-00a0-11e9-bf3a-e069950db0ca REMOVED 0 0 0
gptid/6873e128-00a0-11e9-bf3a-e069950db0ca ONLINE 0 0 0
gptid/69128620-00a0-11e9-bf3a-e069950db0ca ONLINE 0 0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
status: One or more devices are configured to use a non-native block size.
Expect reduced performance.
action: Replace affected devices with devices that support the
configured block size, or migrate data to a properly configured
pool.
scan: scrub repaired 0B in 00:00:08 with 0 errors on Wed Sep 20 03:45:08 2023
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
ada0p2 ONLINE 0 0 0 block size: 512B configured, 4096B native

errors: No known data errors
root@freenas[~]#
root@freenas[~]# smartctl -a /dev/ada0
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p14 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Silicon Motion based SSDs
Device Model: ADATA SU800
Serial Number: 2I4220075639
LU WWN Device Id: 5 707c18 1006f7a9c
Firmware Version: R0427ANR
User Capacity: 256,060,514,304 bytes [256 GB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
TRIM Command: Available, deterministic, zeroed
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-3 T13/2161-D revision 5
SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sat Sep 23 21:58:36 2023 CDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Status command failed
Please get assistance from https://www.smartmontools.org/
Register values returned from SMART Status command are:
ERR=0x00, SC=0x7c, LL=0x16, LM=0xf2, LH=0xaa, DEV=...., STS=....
SMART Status not supported: Invalid ATA output register values
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.

General SMART Values:
Offline data collection status: (0x03) Offline data collection activity
is in progress.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 248) Self-test routine in progress...
80% of test remaining.
Total time to complete Offline
data collection: ( 1103) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0002) Does not save SMART data before
entering power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 30) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x0035) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x0000 100 100 000 Old_age Offline - 0
5 Reallocated_Sector_Ct 0x0000 100 100 000 Old_age Offline - 0
9 Power_On_Hours 0x0000 100 100 000 Old_age Offline - 46525
12 Power_Cycle_Count 0x0000 100 100 000 Old_age Offline - 134
148 Total_SLC_Erase_Ct 0x0000 100 100 000 Old_age Offline - 156
149 Max_SLC_Erase_Ct 0x0000 100 100 000 Old_age Offline - 24
150 Min_SLC_Erase_Ct 0x0000 100 100 000 Old_age Offline - 2
151 Average_SLC_Erase_Ct 0x0000 100 100 000 Old_age Offline - 10
159 DRAM_1_Bit_Error_Count 0x0000 100 100 000 Old_age Offline - 0
160 Uncorrectable_Error_Cnt 0x0000 100 100 000 Old_age Offline - 0
161 Valid_Spare_Block_Cnt 0x0000 100 100 000 Old_age Offline - 50
163 Initial_Bad_Block_Count 0x0000 100 100 000 Old_age Offline - 9
164 Total_Erase_Count 0x0000 100 100 000 Old_age Offline - 1638
165 Max_Erase_Count 0x0000 100 100 000 Old_age Offline - 7
166 Min_Erase_Count 0x0000 100 100 000 Old_age Offline - 2
167 Average_Erase_Count 0x0000 100 100 000 Old_age Offline - 3
168 Max_Erase_Count_of_Spec 0x0000 100 100 000 Old_age Offline - 1500
169 Remaining_Lifetime_Perc 0x0000 100 100 000 Old_age Offline - 100
177 Wear_Leveling_Count 0x0000 100 100 050 Old_age Offline - 0
181 Program_Fail_Cnt_Total 0x0000 100 100 000 Old_age Offline - 0
182 Erase_Fail_Count_Total 0x0000 100 100 000 Old_age Offline - 0
192 Power-Off_Retract_Count 0x0000 100 100 000 Old_age Offline - 122
194 Temperature_Celsius 0x0000 100 100 000 Old_age Offline - 52
195 Hardware_ECC_Recovered 0x0000 100 100 000 Old_age Offline - 0
196 Reallocated_Event_Count 0x0000 100 100 016 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0000 100 100 050 Old_age Offline - 0
232 Available_Reservd_Space 0x0000 100 100 000 Old_age Offline - 100
241 Host_Writes_32MiB 0x0000 100 100 000 Old_age Offline - 6612
242 Host_Reads_32MiB 0x0000 100 100 000 Old_age Offline - 25417
245 TLC_Writes_32MiB 0x0000 100 100 000 Old_age Offline - 29484

SMART Error Log Version: 1
Warning: ATA error count 0 inconsistent with error log pointer 1

ATA Error Count: 0
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 0 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was in an unknownstate.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
00 ec 00 00 00 00 00

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
ec 00 00 00 00 00 00 00 00:00:00.000 IDENTIFY DEVICE
c8 00 00 00 00 00 00 00 00:00:00.000 READ DMA

SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Self_test_in_progress [80% left] (0-65535)
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing

 

[Jailer]

What exactly makes you think that your boot drive needs replacing?

 

[Cosmas]

What exactly makes you think that your boot drive needs replacing?

The error? To be fair I'm not exactly sure what I'm seeing. I'm open to any suggestions here.