Did you try rebooting and trying a second time?
Yes. I am still concerned about the process it is attempting either way, because it is suggesting it is trying to
wipe the drives.
I'm going to be honest here, while I don't have much experience adding and removing VDEVs, I'm also not sure of the configuration of the entire pool.
I'm probably being stupid here but why didn't you just resilver in the 20TB drives to replace the 4TB drives? You can replace one drive in mirror1, resilver it, once done replace the second drive in mirror-1, let it resilver. Then just add a third 20tb drive (extend) to mirror-1 and now you have a 3-way mirror for mirror-1. Make sure the drives removed have been wiped for reuse.
I would also remove the spare drives and use those to bolster the other 4TB drives, add on to mirror and one to mirror-3. Next you have two more drives left over from mirror-1 and add one to mirror4 and ad the last to mirror-5. This leaves you hanging with mirror6 having a 2 way mirror of 4TB drives.
As you purchase more 20TB drives, replace the mirrored drives one at a time, not as one group.
And you can remove a mirror vdev of course via the GUI. I just created a VM of Scale and added and removed virtual drives in mirrors to test it out. Since you are not understanding what is being told to you, I highly recommend you run up a virtual machine of TrueNAS Scale so you can play with it and understand what you need to do. It's not complicated and if you are concerned about data loss, backup your data first. We always tell people this because while things generally work fine, it's also easy to click the wrong button and make a huge mistake.
This reference in the user guide should help you a lot and the VM of TrueNAS.
So you're saying, replace 1 of the 4tb drives with a 20tb, let it resilver, then replace the other 4tb drive, then I will have replaced one of the mirrors with two 20tb drives. Then add add the 3rd onto the mirror.
And then do the same with another one of the mirrors?
And yes I did plan to remove the spare drives and add them onto the leftover 2 way 4tb mirrors to turn them into 3 way mirrors.
I guess that makes sense. I've done drive replacements so I guess I am more confident in that method.
The only reason I did it the way I did was because that was how the process was suggested. However, it does not seem to be working as intended, and even if I do it in a VM I don't know if I trust it on my live data here. I don't see any other way of doing it than what I already did for that method.
Also this is on TrueNAS Core, so
maybe it is a bug on Core, and it works on Scale. Idk.vBut I will just hold and replace the drives individually. It feels a bit safer.
Unfortunately I can't really use my current 20tb drives. But it's fine, they will give me more storage.
One of them is already saying pre-failing, but also old_age? lmao so maybe I'll have to RMA that? The results are really weird though..?
Rerunning a scrub to see if anything changes.
Code:
Device: /dev/da17 [SAT], failed to read SMART Attribute Data.
Code:
# smartctl -a /dev/da17
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p7 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Device Model: ST20000NM007D-3DJ103
Serial Number: ZVT5JR8S
LU WWN Device Id: 5 000c50 0e5cb06f8
Firmware Version: SN01
User Capacity: 20,000,588,955,648 bytes [20.0 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 7200 rpm
Form Factor: 3.5 inches
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ACS-4 (minor revision not indicated)
SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Wed Sep 20 14:49:38 2023 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 567) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: (1714) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x70bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 084 064 044 Pre-fail Always - 244122256
3 Spin_Up_Time 0x0003 094 094 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 5
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 071 060 045 Pre-fail Always - 11347170
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 591
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 5
18 Unknown_Attribute 0x000b 100 100 050 Pre-fail Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 063 048 000 Old_age Always - 37 (Min/Max 28/43)
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 4
193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 1059
194 Temperature_Celsius 0x0022 037 047 000 Old_age Always - 37 (0 24 0 0 0)
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0023 100 100 001 Pre-fail Always - 0
240 Head_Flying_Hours 0x0000 100 100 000 Old_age Offline - 311 (62 151 0)
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 2094009876
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 549859027646
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 577 -
# 2 Short offline Completed without error 00% 553 -
# 3 Short offline Completed without error 00% 529 -
# 4 Short offline Completed without error 00% 505 -
# 5 Short offline Completed without error 00% 481 -
# 6 Short offline Completed without error 00% 457 -
# 7 Short offline Completed without error 00% 433 -
# 8 Short offline Completed without error 00% 409 -
# 9 Short offline Completed without error 00% 385 -
#10 Short offline Completed without error 00% 361 -
#11 Short offline Completed without error 00% 338 -
#12 Short offline Completed without error 00% 316 -
#13 Short offline Completed without error 00% 292 -
#14 Short offline Completed without error 00% 268 -
#15 Short offline Completed without error 00% 244 -
#16 Short offline Completed without error 00% 220 -
#17 Short offline Completed without error 00% 196 -
#18 Short offline Completed without error 00% 172 -
#19 Short offline Completed without error 00% 148 -
#20 Short offline Completed without error 00% 124 -
#21 Short offline Completed without error 00% 100 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing