SOLVED Hard failure of drive, cannot mark as 'offline' per manual

[mysticpete]

Hi all

Just trying to replace a failed drive but not getting Drive Offline when trying to do this, checked other posts but could not see anything similar, can anyone assist me with this please, thanks in advance.

root@freenas:/ # zpool status
pool: freenas-boot
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: scrub repaired 0B in 00:02:17 with 0 errors on Thu Apr 7 03:47:17 2022
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
ada0p2 ONLINE 0 0 0

errors: No known data errors

pool: media
state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
repaired.
scan: scrub repaired 672K in 11:38:23 with 0 errors on Sun Apr 3 11:38:30 2022
config:

NAME STATE READ WRITE CKSUM
media DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
gptid/c8198192-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/c8f8d91d-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/c98173f8-42f6-11e3-b481-c8cbb8c52f20 FAULTED 74 0 0 too many errors
gptid/ca025a49-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/304c50d2-ff29-11e6-87ad-c8cbb8c52f20 ONLINE 0 0 0

errors: No known data errors
root@freenas:/ #

System:
Version: TrueNAS-12.0-U8

 

[joeschmuck]

Just trying to replace a failed drive but not getting Drive Offline when trying to do this

What drive?

So you need to do a few more things before you start replacing drives. First, post your system configuration per the forum rules. We have no idea what version of FreeNAS you are running and it does matter. This helps us understand your configuration a little better and allows us to provide you the correct help information.

Since we do not have that extra important data, here is a few steps I'd do, that maybe you already have done, who knows.

Next, did you have any other failure indications or was it just this one indication? I ask because this alone doesn't indicate a drive failure, it indicate data corruption which can be caused by a few factors. I'm looking for if you have any SMART test failures? If not then I'd run a SMART long/extended test on all your drives, make sure they pass, or maybe one fails. Tell us the results. If you have no SMART drive failures then run a Scrub on the "media" pool. When it is complete check the scrub status for a failure indication by looking at the "scan:" line to see if it repaired anything.

Now for the tricky part... If you had no SMART failures AND you had no new Scrub failures (looking for "scrub repaired 0b", you can try to reset the scrub error message by entering zpool clear media and then check your zpool status again. If it looks good, just for good measure, run another scrub for peace of mind and check it once done.

If you find a drive failure, then work on replacing the failed drive.

Lastly, if you have some crazy pool setup or have "valuable data" you want to keep, back that data up before pulling out any hard drives. It's the smart thing to do.

EDIT: To answer the question you asked though... If you cannot force the drive to Offline, shut down the system and swap out the drive, when you power back on you "should" be able to GUI now to complete the replacement. It is documented a few times in the archived forums.

 

[mysticpete]

What drive?

So you need to do a few more things before you start replacing drives. First, post your system configuration per the forum rules. We have no idea what version of FreeNAS you are running and it does matter. This helps us understand your configuration a little better and allows us to provide you the correct help information.

Since we do not have that extra important data, here is a few steps I'd do, that maybe you already have done, who knows.

Next, did you have any other failure indications or was it just this one indication? I ask because this alone doesn't indicate a drive failure, it indicate data corruption which can be caused by a few factors. I'm looking for if you have any SMART test failures? If not then I'd run a SMART long/extended test on all your drives, make sure they pass, or maybe one fails. Tell us the results. If you have no SMART drive failures then run a Scrub on the "media" pool. When it is complete check the scrub status for a failure indication by looking at the "scan:" line to see if it repaired anything.

Now for the tricky part... If you had no SMART failures AND you had no new Scrub failures (looking for "scrub repaired 0b", you can try to reset the scrub error message by entering zpool clear media and then check your zpool status again. If it looks good, just for good measure, run another scrub for peace of mind and check it once done.

If you find a drive failure, then work on replacing the failed drive.

Lastly, if you have some crazy pool setup or have "valuable data" you want to keep, back that data up before pulling out any hard drives. It's the smart thing to do.

Apologies Old Man, completely forgot to post the system details, I have updated the post with this now along with other relevant info requested.

 

[mysticpete]

Actually just noticed that the S.M.A.R.T test shows Extended Offline but the drive status still shows Faulty and not Offline, maybe not the same thing?

 

[sretalla]

Just trying to replace a failed drive but not getting Drive Offline when trying to do this, checked other posts but could not see anything similar, can anyone assist me with this please

Faulted is equivalent to Offline, so you can skip that step and just do the replace.

 

[danb35]

the S.M.A.R.T test shows Extended Offline

"Extended Offline" is a type of SMART self-test, not a result or status code--the result of that test is that the drive failed. It's also the only SMART test in the drive's 30k-hour service life, so once you do replace the disk, you'll need to take a look at your SMART test settings to make sure those tests are running regularly (on all your disks).

But to your question, the FAULTED status for the disk implies it being OFFLINE. You can proceed with the rest of the replacement per the manual.

 

[joeschmuck]

Look at all that help, I love this place!

 

[mysticpete]

Thank you all you Wizened Sages

Much appreciated :)

 

[Demonlinx]

This has helped me so much! I hadn't been able to find that FAULTED == OFFLINE. Is this somewhere in the documentation? The instructions tell you to offline the drive but when you try to offline the faulted drive it pops up the status and spins and then goes back and doesn't change FAULTED to OFFLINE. If this is intended behavior it would be nice to document that somewhere.