Failed Drive replaced with Hot Spare sill showing Pool DEGRADED

[dfriestedt]

Just had a data drive fail. The Hot Spare kicked in. The good drive was resilvered without errors. However, the pool is still showing as DEGRADED. What additional steps do I need to take?

I did try to put /dev/gptid/a7296f03-9690-11e9-ac10-3cfdfecf5af8 OFFLINE. It's currently OFFLINE now as show before. Do I need to power down and remove the drive? Or is there something else I need to do?

Code:

root@freenas-backup[~]# zpool status -v
  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:00:01 with 0 errors on Tue Aug 20 03:45:01 2019
config:

        NAME        STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          nvd0p2    ONLINE       0     0     0

errors: No known data errors

  pool: gmdz2
 state: DEGRADED
status: One or more devices has been taken offline by the administrator.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Online the device using 'zpool online' or replace the device with
        'zpool replace'.
  scan: resilvered 2.09T in 0 days 15:00:23 with 0 errors on Fri Aug 23 15:31:31 2019
config:

        NAME                                              STATE     READ WRITE CKSUM
        gmdz2                                             DEGRADED     0     0   0
          raidz2-0                                        DEGRADED     0     0   0
            gptid/8f47c078-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            gptid/934ad67a-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            gptid/969d804b-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            gptid/3c5e9f98-61e4-11e9-bfa2-3cfdfecf5af8    ONLINE       0     0   0
            gptid/9e0d0fad-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            gptid/a1742971-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            spare-6                                       DEGRADED     0     0   0
              2022795340040031925                         OFFLINE      0     0   0  was /dev/gptid/a7296f03-9690-11e9-ac10-3cfdfecf5af8
              gptid/6dbfef0e-c547-11e9-9b1f-3cfdfecf5af8  ONLINE       0     0   0
          raidz2-1                                        ONLINE       0     0   0
            gptid/c9b834be-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            gptid/cdc312dd-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            gptid/d12be3bf-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            gptid/d49a7908-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
            gptid/7db30d85-c282-11e9-9b1f-3cfdfecf5af8    ONLINE       0     0   0
            gptid/bfcbc96b-c2e2-11e9-9b1f-3cfdfecf5af8    ONLINE       0     0   0
            gptid/e02a32c9-5ed1-11e9-bf67-3cfdfecf5af8    ONLINE       0     0   0
        logs
          gptid/c29dab3b-60f0-11e9-bfa2-3cfdfecf5af8      ONLINE       0     0   0
        spares
          16188425594630421685                            INUSE     was /dev/gptid/6dbfef0e-c547-11e9-9b1f-3cfdfecf5af8

errors: No known data errors

 

[dfriestedt]

Guys - I power off the server, removed the degrated drive, and restated. The output above has not changed. Thoughts on how to proceed?

 

[dfriestedt]

I'm now trying to replace the degraded drive. I powered off the server, physically replaced the damaged drive, and the replaced the removed drive with the new drive in the freenas GUI. The new drive is reslivering, but the spare drive is still associated with the failed drive. I hope I'm not digging myself a hole here.

 

[Johnny Fartpants]

hot-spares are temporary, once they have kicked in you need to replace the *FAILED* drive with a good one and once its resilvered the hot-spare will automatically go back into the pool as a hot-spare ready to save the day again at some point in the future.

 

[dfriestedt]

Thank you for that. So just to confirm, when a drive fails, the spare kicks in, resilvers and sits there (in a DEGRADED state) until a replacement is inserted and replaces the original failed drive. Once the replacement drive is up and running (and resilvered) the spare will revert back to a spare and wait for another drive to fail. Am I understanding that correctly?

thx for the intel.

 

[Johnny Fartpants]

Exactly