ZFS Pool degraded looking for advice

[fonze98]

On my home lab I have a RAID Z1 pool set up with three VDEVs. Two of the VDEVs have three 8TB drives each and the third VDEV has three 12TB drives. (I am slowly upgrading each VDEV to 12TB drives) I also have a 12TB hot spare drive.

One of the 8TB drives had a cable go bad and started erroring so the spare took over. I have replaced the cable and performed multiple long smart tests on the drive that was erroring out and it seems to be fine now.

The pool shows up as degraded and in the POOL STATUS page it show a drop down for the SPARE that shows both the 12TB spare and the 8TB drive that was having issues. It shows the 8TB as FAULTED and the 12TB as ONLINE.

This is the first time I have had any drive failure\pool problems and I can not seem to find how best to handle this situation. I am fine with putting the 8TB back into the pool (all the data is also on two other systems) and keeping the 12TB as the hot spare but not really sure what steps I need to take to accomplish this.

Any advice would be welcome.

 

[morganL]

It would be better to update to 22.02.4..... there were some issues with disk management in early SCALE versions.

 

[fonze98]

Sorry I had not updated my sig I am on the latest version

I am basically just looking to see what the process is for once the hot spare has taken over how do I tell the system I want to use the old drive again

Although I just bought 3 12TB EXOS drives on a black friday deal so may just upgrade that whole vdev. I would still like to know the process for future reference.

 

[morganL]

Read this part of user guide: https://www.truenas.com/docs/scale/scaletutorials/storage/disks/replacingdisks/

Its unusual to fail a disk, resilver into a spare and then re-install the same disk with data... I'd recommend wiping and re-installing it to reduce confusion and risk. But only after pool is healthy again.

 

[fonze98]

Ok according to that guide the first step I should take is to offline the drive that was having issues. when I try that absolutely nothing happens. I am attaching a screenshot of my current state of the pool.

 

[morganL]

So sdw doesn't provide the confirmation dialog for offline? Do you get all the disk options?

 

[sretalla]

The replacing disks process makes no allowance for spares being involved.

You have a spare in action in the pool, so you'll need to handle it accordingly.

You must decide to either return the spare to be an available spare (after replacing the sdw disk more-or-less according to the process ... Hint: FAULTED=already offlined ... then detach sdq to return it to spare. Otherwise, you can keep the spare and detach sdw, then add another spare later.

 

[fonze98]

I think that was the piece I was missing

FAULTED=already offlined

 

[morganL]

I think that was the piece I was missing

If its not clear in the documentation, please make a comment on the relevant page....
User feedback is really needed to make docs better.