SOLVED 8 Drive vdev Upgraded HDD size from 8tb drives to 12tb drives, 8th drive will not show member drive to use for replacement

[BitBug]

Over the last few days I have been replacing 8 x 8tb Seagate Iron Wolf Hard Drives with 8 x 12tb Seagate Iron Wolf Hard Drives. The first 7 went smoothly and resilvered without issue. The 8th drive will not show a member drive to replace it. After a lot of tries, I assumed the drive was DOA and returned. However, the replacement drive is having the same problem and the odds of two dead drives in a row seem slim.

The pool is Media and you can see below only 7 of 8 drives show up, yet I can see the drive in the UI offline. After I replace the drive with the new, larger drive and then select "replace", the Replace disk.... shows up, but when I click on Member Disk it does not bring up the new, larger hard drive to use for replacement. There is not disk listed at all to use for the replacement.

I am at a loss to what is going on and would appreciate any help.

Here is zpool status:

FreeBSD 12.2-RELEASE-p14 325282c09a5(HEAD) TRUENAS

TrueNAS (c) 2009-2022, iXsystems, Inc.
All rights reserved.
TrueNAS code is released under the modified BSD license with some
files copyrighted by (c) iXsystems, Inc.

For more information, documentation, help or support, go here:
http://truenas.com
Welcome to FreeNAS

Warning: settings changed through the CLI are not written to
the configuration database and will be reset on reboot.

root@RiceClanNAS:~ # zpool status
pool: Backups
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: resilvered 37.8G in 00:15:13 with 0 errors on Sat Apr 30 10:36:48 2022
config:

NAME STATE READ WRITE CKSUM
Backups ONLINE 0 0 0
raidz3-0 ONLINE 0 0 0
gptid/6274432a-fd94-11e7-a155-00259063ab8c ONLINE 0 0 0
gptid/63b8baa6-fd94-11e7-a155-00259063ab8c ONLINE 0 0 0
gptid/64ed71b0-fd94-11e7-a155-00259063ab8c ONLINE 0 0 0
gptid/662d2816-fd94-11e7-a155-00259063ab8c ONLINE 0 0 0
gptid/675ec2fc-fd94-11e7-a155-00259063ab8c ONLINE 0 0 0
gptid/6895f362-fd94-11e7-a155-00259063ab8c ONLINE 0 0 0
gptid/69ce7d02-fd94-11e7-a155-00259063ab8c ONLINE 0 0 0
gptid/6aff5939-fd94-11e7-a155-00259063ab8c ONLINE 0 0 0

errors: No known data errors

pool: ColdStorageTwo
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: scrub repaired 0B in 05:21:12 with 0 errors on Sun Mar 27 05:21:13 2022
config:

NAME STATE READ WRITE CKSUM
ColdStorageTwo ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
gptid/89ffe250-67f4-11eb-a19d-00259063ab8c ONLINE 0 0 0
gptid/6b8da27d-6808-11eb-a19d-00259063ab8c ONLINE 0 0 0
gptid/4cd88d94-8b4e-11eb-91f0-00259063ab8c ONLINE 0 0 0
gptid/cfcac294-8b5a-11eb-91f0-00259063ab8c ONLINE 0 0 0

errors: No known data errors

pool: DesktopBackups
state: ONLINE
scan: scrub repaired 0B in 16:34:15 with 0 errors on Sun Mar 27 16:34:16 2022
config:

NAME STATE READ WRITE CKSUM
DesktopBackups ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/3e41f9d7-4868-11eb-bda6-00259063ab8c ONLINE 0 0 0
gptid/3e6bf36c-4868-11eb-bda6-00259063ab8c ONLINE 0 0 0
gptid/3f1424a1-4868-11eb-bda6-00259063ab8c ONLINE 0 0 0
gptid/3f7d8913-4868-11eb-bda6-00259063ab8c ONLINE 0 0 0

errors: No known data errors

pool: Media
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: resilvered 84K in 00:00:00 with 0 errors on Sat Apr 30 15:07:22 2022
config:

NAME STATE READ WRITE CKSUM
Media DEGRADED 0 0 0
raidz3-0 DEGRADED 0 0 0
gptid/7db9711f-a2ac-11ea-93e1-00259063ab8c OFFLINE 0 0 0
gptid/68c0e80b-bf74-11ec-9241-00259063ab8c ONLINE 0 0 0
gptid/06a2422b-bea6-11ec-9241-00259063ab8c ONLINE 0 0 0
gptid/ce88acae-c2d6-11ec-9241-00259063ab8c ONLINE 0 0 0
gptid/07820434-bff5-11ec-9241-00259063ab8c ONLINE 0 0 0
gptid/4a9b29c3-c390-11ec-9241-00259063ab8c ONLINE 0 0 0
gptid/f3a7b48d-c05d-11ec-9241-00259063ab8c ONLINE 0 0 0
gptid/b6f5eb60-c24e-11ec-9241-00259063ab8c ONLINE 0 0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: scrub repaired 0B in 00:01:08 with 0 errors on Fri Apr 29 03:46:08 2022
config:

NAME STATE READ WRITE CKSUM
freenas-boot

 

[Nick2253]

It could be that whatever port you are plugging the new drive into is bad. Try plugging it in using a different SATA port or cable. With as many disks as you show here, I'm guessing that you're using some kind of hot-swap chassis. You might try plugging in the new drive directly to the motherboard in order to bypass the backplane and rule it out as a cause.

 

[BitBug]

Its a 24 bay system overall . I did swap the drive into different bays with no luck. And I tried drives from my other pools on the system in the original bay I had this 8th drive in and they worked fine, so I think I ruled out it being a bad cable or drive bay.

 

[BitBug]

Thanks Nick2253. Not sure what to do for next steps here. I've added my system details in my signature in case that helps. Also, being new to this board, is your style of listing your systems in your signature common for the community, or unique to you? I don't want to be stealing your style if not appropriate. The other odd thing is I can't wipe the drive in TureNas. Doesn't give me the option. Is there another way I can wipe any partitions on it in case that is the problem? Probably from the shell, but I'm not strong with the commands and so worry about making a mistake in my syntax. From what I've read around the web it seems like a pre-existing partitions can be an issue. Thank again for you assistance.

 

[sretalla]

I've seen cases where I needed to do a forced refresh of the browser (clearing the cache) for the GUI. (Shift + Refresh in many browsers)

After that, the inserted disk appeared as is usually the case without it.

Short of that, reboot.

The other odd thing is I can't wipe the drive in TureNas. Doesn't give me the option.

That does indeed seem odd... Wipe should be available to clear partitions, so it won't be stopped by them (at least not usually).

It also sounds odd if those drives are factory new... should be no funky LVM partitioning or CEPH hangover to get rid of either.

Is there another way I can wipe any partitions on it in case that is the problem? Probably from the shell

gpart destroy -F /dev/da0 or whatever disk label ... be careful.

 

[BitBug]

Just thought I'd report back. While hard to believe every indication was the replacement drive was also bad. Finally got annoyed enough the other day that I RMA'd the replacement disk and received the third drive tonight. Popped it in, yada yada yada, and it is resilvering as I write this. That's a new one on me. Never had two bad disks in a row since my 5 1/4" floppy disk days . Call this one resolved and thanks to everyone who helped out. Much Appreciated!