Can't replace drive

[DanielKlein]

I had a Mirror Volume, with 2x6TB disks, one of them started presenting errors, and I wanted to upgrade the storage, so I bought 2 10TB ones, I swapped the first, but when I went to replace the second one I get the error

errors: Permanent errors have been detected in the following files: <0x29a>:<0xd09d0>

Is there a way to find which file is this and then delete it so i can complete replacing the Hard Drive? (I prefer not having to recreate the pool from a backup if possible)

 

[PhiloEpisteme]

I had a Mirror Volume, with 2x6TB disks, one of them started presenting errors, and I wanted to upgrade the storage, so I bought 2 10TB ones, I swapped the first, but when I went to replace the second one I get the error

When you say you swapped the first, do you mean you resilvered the first drive? What is the output of zpool status for that pool?

 

[DanielKlein]

Yes, the first one resilvered.

Warning: settings changed through the CLI are not written to
the configuration database and will be reset on reboot.

root@server:~ # zpool status
pool: Vol1
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://illumos.org/msg/ZFS-8000-9P
scan: resilvered 34.7M in 0 days 00:00:34 with 0 errors on Tue Apr 21 20:32:19 2020
config:

NAME STATE READ WRITE CKSUM
Vol1 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
gptid/704ee0d5-9eb7-11e6-b9f6-f46d04502f34 ONLINE 0 0 0
gptid/710c2f10-9eb7-11e6-b9f6-f46d04502f34 ONLINE 0 0 0
gptid/71ad0c84-9eb7-11e6-b9f6-f46d04502f34 ONLINE 0 030

errors: No known data errors

pool: Vol2
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: resilvered 8.12T in 0 days 11:49:42 with 1 errors on Wed Apr 22 08:21:35 2020
config:

NAME STATE READ WRITE CKSUM
Vol2 ONLINE 0 0 1
mirror-0 ONLINE 0 0 2
replacing-0 ONLINE 0 0 2
gptid/e4a14ab4-5497-11e8-8471-f46d04502f34 ONLINE 0 0 26
gptid/63d54c1d-833e-11ea-a679-f46d04502f34 ONLINE 0 0 2
ada0 ONLINE 0 0 2

errors: 1 data errors, use '-v' for a list

pool: freenas-boot
state: ONLINE
scan: resilvered 1.04G in 0 days 00:00:13 with 0 errors on Fri Apr 17 14:33:58 2020
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
mirror-0 ONLINE 0 0 0

The pool I'm trying to replace is the Vol2

 

[Basil Hendroff]

I swapped the first,

I hope you replaced the faulty drive first.

 

[DanielKlein]

I hope you replaced the faulty drive first.

The other drive stopped showing up at all :/

 

[PhiloEpisteme]

Which drive was the "other" drive. In general, if the state of the pool changes the output of zpool status is super useful; and it is nice if you leave the original formatting if possible. Those tabs and spaces make it easier to quickly read.

Also, did you by chance do the resilver via the command line? Generally with FreeNAS that is ill-advised. FreeNAS uses GPT partition tables on disks and includes a swap partition on each disk by default and refers to the storage partition via its GPT partition rawuuid. In order to allow FreeNAS to manage disks the way it expects to it is best to do as much as possible via the GUI.

 

[DanielKlein]

Which drive was the "other" drive. In general, if the state of the pool changes the output of zpool status is super useful; and it is nice if you leave the original formatting if possible. Those tabs and spaces make it easier to quickly read.

Also, did you by chance do the resilver via the command line? Generally with FreeNAS that is ill-advised. FreeNAS uses GPT partition tables on disks and includes a swap partition on each disk by default and refers to the storage partition via its GPT partition rawuuid. In order to allow FreeNAS to manage disks the way it expects to it is best to do as much as possible via the GUI.

Yes, when I swapped the faulty driver I got an error, so I resilvered the new one via command line. Now looking back to it I should've asked here in the forums about the error before doing that :/

 

[PhiloEpisteme]

Just so I have this right.

You have a pool with two disks, call them disk1 and disk2.

1. disk1 experienced a failure, so you tried to resilver it via the GUI with disk1_2; that failed
2. You then tried to resilver disk 1 via the command line using disk1_2
3. Then, you resilvered disk2 with disk2_2 via the GUI

If that is correct, did the resilver in step 2 complete before you started on step 3?

 

[DanielKlein]

Yes, you are exactly right. And yes, I waited for the resilver to complete. But now the resilver is done with 1 error, and the replacement won't finish

pool: Vol2
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: resilvered 8.12T in 0 days 11:49:42 with 1 errors on Wed Apr 22 08:21:35 2020
config:

NAME STATE READ WRITE CKSUM
Vol2 ONLINE 0 0 1
mirror-0 ONLINE 0 0 2
replacing-0 ONLINE 0 0 2
gptid/e4a14ab4-5497-11e8-8471-f46d04502f34 ONLINE 0 0 26
gptid/63d54c1d-833e-11ea-a679-f46d04502f34 ONLINE 0 0 2
ada0 ONLINE 0 0 2

errors: Permanent errors have been detected in the following files:

<0x29a>:<0xd09d0>

ADA6 is the old Disk2 and ADA1 de new Disk2_2

I don't think deleting the file would be a problem for me even don't knowing exactly what it is, I have a cloudbackup of the whole pool, and would be way more expensive to restore the pool from a cloudbackup