DEGRADED RAISZ2 REPLACE DISKS

[aaronpraha]

Sorry, I've read through a ton of threads but have not been able to figure this out.

I had 2 drives fail on my encrypted RAIDZ2 Array.
I offline'd the problem disks, shutdown, replaced the drives, and rebuilt. The re-slivering took 4 days but completed. However, I still have "REPLACING" on the new drives with multiple (because I reslivered/replaced them multiple times). Now I have the disk online but 2 /dev/gptid/xxxxxxxxxxxxxxxxx.eli's for each drive.
I don't know what to do.

'zpool status' output:

FreeBSD 11.3-RELEASE-p11 (FreeNAS.amd64) #0 r325575+fb17f3e15b8(HEAD): Tue Jul 2 8 11:09:10 EDT 2020

FreeNAS (c) 2009-2020, The FreeNAS Development Team
All rights reserved.
FreeNAS is released under the modified BSD license.

For more information, documentation, help or support, go here:
http://freenas.org
Welcome to FreeNAS

Warning: settings changed through the CLI are not written to
the configuration database and will be reset on reboot.

root@tardis[~]# zpool status
pool: Tardis
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: resilvered 7.89T in 3 days 13:24:19 with 584 errors on Thu Nov 5 22:44:26 2020
config:

NAME STATE READ WRITE CKSUM
Tardis DEGRADED 0 0 31.8K
raidz2-0 DEGRADED 0 0 63.7K
gptid/8b6a0c20-3241-11e9-afe3-ac1f6b7274dc.eli DEGRADED 0 0 0 too many errors
gptid/8d61beee-3241-11e9-afe3-ac1f6b7274dc.eli DEGRADED 0 0 0 too many errors
gptid/5f6684fa-eebd-11ea-87ca-ac1f6b7274dc.eli DEGRADED 0 0 0 too many errors
replacing-3 DEGRADED 0 0 0
13827796178520726740 OFFLINE 0 0 0 was /dev/gptid/b940118c-c971-11ea-ac2a-ac1f6b7274dc.eli
5347387314038862202 OFFLINE 0 0 0 was /dev/gptid/c1b92e3c-0e96-11eb-a3f0-ac1f6b7274dc.eli
gptid/7c223cfe-1570-11eb-8d10-ac1f6b7274dc.eli ONLINE 0 0 0
gptid/93c04eab-3241-11e9-afe3-ac1f6b7274dc.eli DEGRADED 0 0 0 too many errors
replacing-5 OFFLINE 0 0 0
9147511623165312555 OFFLINE 0 0 0 was /dev/gptid/95a620bf-3241-11e9-afe3-ac1f6b7274dc.eli
7576730276490940677 OFFLINE 0 0 0 was /dev/gptid/757902ed-156f-11eb-8d10-ac1f6b7274dc.eli
gptid/d249ef34-1d1e-11eb-8d10-ac1f6b7274dc.eli ONLINE 0 0 0

errors: 1168 data errors, use '-v' for a list

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0 days 00:09:38 with 0 errors on Tue Nov 10 03:54:38 2020
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
da1p2 ONLINE 0 0 0
da0p2 ONLINE 0 0 0

errors: No known data errors
root@tardis[~]#

 

[aaronpraha]

Is this a difficult problem or is no one weighing in because I'm a fool and am missing something very obvious; asked and answered before? I have done a decent amount of looking on these forums and googling but I'm still at a loss as what to do.

thank you

 

[joeschmuck]

My first recommendation would be to backup your data. I figured that if you are encrypting it, it must be important.

Please use CODE tags when posting output data, it generally retains the format of the data which helps us out.

Once all your data is safe elsewhere then you can do stuff to your pool in an attempt to make it proper again. Worst case is you destroy the pool and recreate it. I would also ensure you have done a SMART Long test on all your hard drives before continuing to ensure they are all in good shape, and check those SMART values too.

 

[aaronpraha]

I tend to encrypt everything just cuz. Yes, this has some sensitive date on it, but it is also mainly my media server. Would you recommend not encrypting for easier maintenance? Yes SMART long test are clean.

I'm in the process of backing up the data to other locations (day 1.5 and going. thank goodness I have enough extra storage).

 

[joeschmuck]

Would you recommend not encrypting for easier maintenance?

I would recommend not encrypting unless you need it, but if you desire it then it's just another thing consider. In the earlier days it was a real pain to work with. So keep it if you like.

If you would rather not destroy your pool then you could search the forums for help in correcting your drive issue. I find it easier to use something like Google and enter a search on something like "freenas drive replacing offline degraded" and I'm certain it will pull up a lot of hits. Many people have had issues just like yours, but most were not encrypted. You could add "encrypted" into that search as well and see what comes up. Again, worst case is you destroy the pool. Of course if you have any jails/plugins then those may go away unless you have those on a separate drive.