replacement disk stuck in replacing, unable to detach

[mazdajai]

I am running FreeNAS-11.2-U8 and trying to detach a failed disk that is stuck on replacing status for couple days.

The failed disk was replacing via GUI. (Server shutdown, swap disk, wipe new disk and "Replace" option under pool. Is there anything else I can do such as checking replacing /resilvering status before I have to redo the pool?

Code:

  pool: vol1
 state: DEGRADED
status: One or more devices has experienced an error resulting in data
    corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
    entire pool from backup.
   see: http://illumos.org/msg/ZFS-8000-8A
  scan: resilvered 676G in 0 days 03:09:16 with 16348 errors on Sat Dec 19 18:22:20 2020
config:

    NAME                                              STATE     READ WRITE CKSUM
    vol1                                              DEGRADED     0     0 19.8K
      raidz1-0                                        DEGRADED     0     0 47.6K
        gptid/158a0c38-1c89-11eb-896f-001e674758cd    ONLINE       0     0     0  block size: 512B configured, 4096B native
        gptid/53fc6bb4-e3b4-11e2-9dcd-003018a56e64    DEGRADED     0     0     0  too many errors
        gptid/547d76c5-e3b4-11e2-9dcd-003018a56e64    DEGRADED     0     0     0  too many errors
        gptid/5502f4d4-e3b4-11e2-9dcd-003018a56e64    DEGRADED     0     0     0  too many errors
        replacing-4                                   UNAVAIL      0     0     0
          17085095126061593047                        UNAVAIL      0     0     0  was /dev/ada4p2
          gptid/d62a52fa-4235-11eb-8374-001e674758cd  ONLINE       0     0     0

errors: 16081 data errors, use '-v' for a list

 

[joeschmuck]

Based on your message above, your pool is in bad shape. Have you backed up all your critical information? If not, do it right now, do not wait for the pool to fix itself.

Per our forum rules (you have been a member for a long time) please post your system configuration build info. Also after backing up your data, I'd recommend that you run some SMART data checking (see the Hard Drive Troubleshooting Guide link in my signature) and once you have verified the integrity of your hard drives, take the next step. Right now I either suspect you have some failing hard drives or you have shutdown your machine improperly or its rebooting on it's own. Difficult to know without more info.

Good Luck, you might really need it.

 

[mazdajai]

Thank you! Curious, is there a zpool command to check replacing status?

Yes indeed I am a long term member! (I am using the same hardware sine 2012) How do I add the systems config likes yours in the signature?

 

[joeschmuck]

Your status above indicates the resilvering is complete. You have a lot of errors of your data across multiple hard drives, also indicated in your original posting. This is why I suggested that you backup your data. Once your data is backed up then I would run some SMART Long Tests on all your drives and examine the data. If your drives look good then I would destroy your pool and recreate it. Again, backup all your data, including stuff you might have in a jail that you want to retain.

 

[sretalla]

Because those errors are all checksum, I would think about checking your connections and cabling before doing too much else than a backup.

is there a zpool command to check replacing status?

zpool status -v
which will show you the corrupt file(s), which are preventing the resilver from really being finished. You will need to replace, restore or delete those files before the replacing disk will go away.