2 failed drives with RAIDZ2

[Ronald Pritchard]

First off I had 3 older drives sitting around so I used them for my array I have 6 x 3tb drives I had one fail so I replaced that drive the next time I looked it replace the one drive, then I had 2 more fail I am running ZFS2 so I could withstand 2 drive failures. Now all 6 drives are NAS drives so hopefully I won't loose any more they are 6 months old at the oldest. So I replace my 2 drives they are online but my array stills shows degraded, the system still shows the drives that I replaced plus the ones failed that I replace the new drives with I suspect that is why my array is degraded. How do I get rid of the failed drives that aren't there but show up as there I tried remove drives it failed any help with this issue would be greatly appreciated.

 

[dlavigne]

Does zpool status show that the resilver completed? If so, does rebooting fix it?

 

[Bidule0hm]

It is probably still resilvering the new drives, it'll take more time than the first time because you have 2 failed drives instead of one.

 

[Ronald Pritchard]

Does zpool status show that the resilver completed? If so, does rebooting fix it?

Yes it shows resilvering complete and I rebooted it it shows all the drives online except the 2 that were bad that I have replaced and it still show degraded status (scratching head)

 

[Bidule0hm]

Can you post the output of zpool status between code tags please?

 

[cyberjock]

You need to detach the removed/offlined drives. :P

 

[Ronald Pritchard]

You need to detach the removed/offlined drives. :p

I've tried to detach the bad drives they won't detach

 

[Ronald Pritchard]

Can you post the output of zpool status between code tags please?

I would post the output of zpool status but I am not home and won't be any time soon but when I am I will post my zpool status

 

[Ronald Pritchard]

I tried to detach the bad drive that isn't there and this is what I get, the array is resilver again

 

[jgreco]

Please note there's no such thing as "ZFS2". I assume from the enclosed snapshots you mean RAIDZ2.

 

[cyberjock]

That screenshot isn't as good as a zpool status. Try again with the output from the command. ;)

 

[Ronald Pritchard]

That screenshot isn't as good as a zpool status. Try again with the output from the command. ;)

Code:

[root@freenas ~]# zpool status
pool: Media
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Sun Oct 25 15:50:31 2015
132G scanned out of 6.32T at 429M/s, 4h12m to go
69.0M resilvered, 2.04% done
config:

NAME STATE READ WRITE CKSUM
Media DEGRADED 0 0 71
raidz2-0 DEGRADED 0 0 256
replacing-0 DEGRADED 0 0 0
9447652936656904740 UNAVAIL 0 0 0 was /dev/gptid/b0ff7b6d-4f6b-11e5-851e-fcaa149
03549
gptid/2f7b96ad-6f4c-11e5-8a76-fcaa14903549 ONLINE 0 0 0 (resilvering)
gptid/6fb2f52d-6ca9-11e5-9000-fcaa14903549 ONLINE 0 0 0 (resilvering)
replacing-2 DEGRADED 0 0 0
2800920352173413274 UNAVAIL 0 0 0 was /dev/gptid/b1c36bd5-4f6b-11e5-851e-fcaa149
03549
gptid/ee5b51aa-6f4c-11e5-8a76-fcaa14903549 ONLINE 0 0 0 (resilvering)
gptid/b20d684e-4f6b-11e5-851e-fcaa14903549 ONLINE 0 0 0
gptid/b26907e6-4f6b-11e5-851e-fcaa14903549 ONLINE 0 0 0
gptid/b2b33e17-4f6b-11e5-851e-fcaa14903549 ONLINE 0 0 0

errors: 24071762 data errors, use '-v' for a list

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Wed Oct 7 03:45:53 2015
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da0p2 ONLINE 0 0 0

errors: No known data errors

 

[Bidule0hm]

Please use the code tags to post scripts outputs because it's unreadable otherwise, thanks.

 

[jgreco]

Please use the code tags to post scripts outputs because it's unreadable otherwise, thanks.

... he types as a moderator is editing the post...

 

[Bidule0hm]

Yeah but the formatting has been lost, it's not useful to add the tags after as you can see... you need to re-paste from the CLI :)

 

[jgreco]

It's nice to see it in a fixed font.

 

[Ronald Pritchard]

Please use the code tags to post scripts outputs because it's unreadable otherwise, thanks.

Code:

 [root@freenas ~]# zpool status

pool: Media
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Sun Oct 25 15:50:31 2015
132G scanned out of 6.32T at 429M/s, 4h12m to go
69.0M resilvered, 2.04% done
config:

NAME STATE READ WRITE CKSUM
Media DEGRADED 0 0 71
raidz2-0 DEGRADED 0 0 256
replacing-0 DEGRADED 0 0 0
9447652936656904740 UNAVAIL 0 0 0 was /dev/gptid/b0ff7b6d-4f6b-11e5-851e-fcaa149
03549
gptid/2f7b96ad-6f4c-11e5-8a76-fcaa14903549 ONLINE 0 0 0 (resilvering)
gptid/6fb2f52d-6ca9-11e5-9000-fcaa14903549 ONLINE 0 0 0 (resilvering)
replacing-2 DEGRADED 0 0 0
2800920352173413274 UNAVAIL 0 0 0 was /dev/gptid/b1c36bd5-4f6b-11e5-851e-fcaa149
03549
gptid/ee5b51aa-6f4c-11e5-8a76-fcaa14903549 ONLINE 0 0 0 (resilvering)
gptid/b20d684e-4f6b-11e5-851e-fcaa14903549 ONLINE 0 0 0
gptid/b26907e6-4f6b-11e5-851e-fcaa14903549 ONLINE 0 0 0
gptid/b2b33e17-4f6b-11e5-851e-fcaa14903549 ONLINE 0 0 0

errors: 24071762 data errors, use '-v' for a list

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Wed Oct 7 03:45:53 2015
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da0p2 ONLINE 0 0 0

errors: No known data errors.

 

[Bidule0hm]

By code tags I mean code tags, not list tags inside code tags. And you need to copy from the terminal as the formatting has been lost.

 

[danb35]

...or just post a debug file. I'm not a fan of digging through them to find the needed information, but it's all there, and formatting isn't an issue.

 

[Ronald Pritchard]

By code tags I mean code tags, not list tags inside code tags. And you need to copy from the terminal as the formatting has been lost.

Code:

pool: Media                                                                                                                    
state: DEGRADED                                                                                                                  
status: One or more devices has experienced an error resulting in data                                                            
        corruption.  Applications may be affected.                                                                                
action: Restore the file in question if possible.  Otherwise restore the                                                          
        entire pool from backup.                                                                                                  
   see: http://illumos.org/msg/ZFS-8000-8A                                                                                        
  scan: resilvered 2.14G in 21h22m with 18858809 errors on Thu Oct 29 17:37:30 2015                                              
config:                                                                                                                          
                                                                                                                                 
        NAME                                              STATE     READ WRITE CKSUM                                              
        Media                                             DEGRADED     0     0 18.1M                                              
          raidz2-0                                        DEGRADED     0     0 36.4M                                              
            replacing-0                                   DEGRADED     0     0     0                                              
              9447652936656904740                         UNAVAIL      0     0     0  was /dev/gptid/b0ff7b6d-4f6b-11e5-851e-fcaa14903549                                                                                                                            
              gptid/2f7b96ad-6f4c-11e5-8a76-fcaa14903549  ONLINE       0     0     0                                              
            gptid/6fb2f52d-6ca9-11e5-9000-fcaa14903549    ONLINE       0     0     0                                              
            replacing-2                                   DEGRADED     0     0     0                                              
              2800920352173413274                         UNAVAIL      0     0     0  was /dev/gptid/b1c36bd5-4f6b-11e5-851e-fcaa14903549                                                                                                                            
              gptid/ee5b51aa-6f4c-11e5-8a76-fcaa14903549  ONLINE       0     0     0                                              
            gptid/b20d684e-4f6b-11e5-851e-fcaa14903549    ONLINE       0     0     0                                              
            gptid/b26907e6-4f6b-11e5-851e-fcaa14903549    ONLINE       0     0     0                                              
            gptid/b2b33e17-4f6b-11e5-851e-fcaa14903549    ONLINE       0     0     0

Hopefully this is what you need to look at?