ZFS - device shows removed cancelled but the condition doesn't seem to go away

[Big Al]

I recently found an odd (seemingly) issue with ZFS in release TrueNAS-12.0-U8.1.

I replaced 2 drives that were dying but not dead. I replaced them one at a time (each in a different mirror set), waiting for resilvering to be completed before replacing the second drive. After replacing the second drive, the first drive failed for some unknown reason. I had issues in removing the drive, but finally got it removed. I subsequently added another drive to that mirror and the resilvering was successful. I also added back the "failed" drive successfully as a spare.

Now I have a remnant "remove: Removal of mirror canceled" message in the zpool status header. Is there anyway to clear it?

Here is the pool status:

Code:

root@nas001:~ # zpool status vol1
  pool: vol1
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
    continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Mon Jul 18 13:19:03 2022
    767G scanned at 38.7M/s, 693G issued at 35.0M/s, 1.13T total
    342G resilvered, 59.90% done, 03:46:35 to go
remove: Removal of mirror canceled on Sat Jul 16 20:55:23 2022
config:

    NAME                                            STATE     READ WRITE CKSUM
    vol1                                            ONLINE       0     0     0
      mirror-0                                      ONLINE       0     0     0
        gptid/ad4ca83b-5503-11ea-937d-d43d7e933a20  ONLINE       0     0     0
        gptid/6afdb997-050a-11ed-8943-d43d7e933a20  ONLINE       0     0     0
      mirror-1                                      ONLINE       0     0     0
        gptid/f782dc70-634b-11ea-9577-d43d7e933a20  ONLINE       0     0     0
        gptid/2fab66bd-06c5-11ed-8479-d43d7e933a20  ONLINE       0     0     0  (resilvering)
    spares
      gptid/eb36b8e3-0494-11ed-85ce-d43d7e933a20    AVAIL   

errors: No known data errors

 

[sretalla]

status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete.

Seems waiting for the next action to complete is all you need... then the remove will disappear and the last completed action will become the resilver.

 

[Big Al]

Seems waiting for the next action to complete is all you need... then the remove will disappear and the last completed action will become the resilver.

Thank you for your reply. All actions are successfully completed yet the "remove: " message is still there. I've also performed a

Code:

zpool clear

to clear any legacy errors.

Code:

root@nas001:~ # zpool status vol1
  pool: vol1
 state: ONLINE
  scan: resilvered 629G in 11:03:27 with 0 errors on Tue Jul 19 00:22:30 2022
remove: Removal of mirror canceled on Sat Jul 16 20:55:23 2022
config:

    NAME                                            STATE     READ WRITE CKSUM
    vol1                                            ONLINE       0     0     0
      mirror-0                                      ONLINE       0     0     0
        gptid/ad4ca83b-5503-11ea-937d-d43d7e933a20  ONLINE       0     0     0
        gptid/6afdb997-050a-11ed-8943-d43d7e933a20  ONLINE       0     0     0
      mirror-1                                      ONLINE       0     0     0
        gptid/f782dc70-634b-11ea-9577-d43d7e933a20  ONLINE       0     0     0
        gptid/2fab66bd-06c5-11ed-8479-d43d7e933a20  ONLINE       0     0     0
    spares
      gptid/eb36b8e3-0494-11ed-85ce-d43d7e933a20    AVAIL   

errors: No known data errors

 

[sretalla]

Have a try with zpool clear vol1 that may do it.

 

[Big Al]

Have a try with zpool clear vol1 that may do it.

Thanks, but no banana.

 

[Samuel Tai]

I don't believe there's any way to clear messages in the zpool status header, short of destroying and recreating the pool.

 

[Big Al]

I don't believe there's any way to clear messages in the zpool status header, short of destroying and recreating the pool.

Thank you. I guess I'll have to just live with it!