encrypted zpool disk replacement questions...

[SnakeByte]

I've got a 4 disk encrypted zpool where two disks are redundant. Recently I started getting errors on my /dev/ada1 drive. I removed the drive in the second slot of the iXSystem's freenas mini, only to discover that it was /dev/ada6. :(

Storage -> Volumes -> trunk -> Volume Status showed that drive as removed even when I plugged the drive back in again. The only choice given to me from the web gui is "Replace" when I click on the drive - no option to just bring the drive back online, so I went ahead and "replaced" the drive with itself.

Now, according to the documentation, when dealing with encryption, I have to essentially re-do the encryption steps: Assign a passphrase, download the key, and create a recovery key.

When I attempted to assign a passphrase however, I got this message:

Code:

Feb  3 11:26:35 freenas manage.py: [middleware.exceptions:37] [MiddlewareError: Unable to set passphrase on gptid/52093a8a-3484-11e5-80f0-d050991b6355: geli: Cannot open gptid/52093a8a-3484-11e5-80f0-d050991b6355: No such file or directory.

Of course, during all of this, the original drive I was getting errors from (/dev/ada1) decided to detach during the resilvering process.

So today, when looking at the Volume Status, the drive I had incorrectly removed shows "online" and the "bad" drive shows "removed."

My questions:

1. How can I determine if the passphrase I changed "stuck" and I'll be able to access this volume after a reboot?
2. If I do pull the wrong drive again, is there a way to bring the drive back to an online status without "replacing" it through the long resilvering process + passphrase + rekey process?

 

[SnakeByte]

More questions:

zpool status shows this:

Code:

        NAME                                                STATE     READ WRITE CKSUM
        trunk                                               DEGRADED     0     0     0
          raidz2-0                                          DEGRADED     0     0     0
            gptid/50bb91eb-3484-11e5-80f0-d050991b6355.eli  ONLINE       0     0     0
            3543114081011200308                             REMOVED      0     0     0  was /dev/gptid/5127d49a-3484-11e5-80f0-d050991b6355.eli
            gptid/518fa2f3-3484-11e5-80f0-d050991b6355.eli  ONLINE       0     0     0
            gptid/410b6535-caab-11e5-bae6-d05099792f2d.eli  ONLINE       0     0     0

I went ahead and re-tried to set a passphrase, and create a recover key.

This is what appeared in my /var/log/messages file when I attempted to assign the passphrase again:

Code:

Feb  5 09:13:03 freenas manage.py: [middleware.exceptions:37] [MiddlewareError: Unable to set passphrase on gptid/5127d49a-3484-11e5-80f0-d050991b6355: geli: Cannot read passphrase: Inappropriate ioctl for device.
]

I assume this failure is normal when a drive is "removed" and that a subsequent reboot will allow me to gain access to the encrypted zpool.

However, this happened when I attempted to create a recovery key:

Code:

Feb  5 09:18:21 freenas manage.py: [middleware.exceptions:37] [MiddlewareError: Unable to set passphrase on gptid/5127d49a-3484-11e5-80f0-d050991b6355: geli: Cannot read passphrase: Inappropriate ioctl for device.
]
Feb  5 09:18:21 freenas manage.py: [middleware.exceptions:37] [MiddlewareError: Unable to set passphrase on gptid/52093a8a-3484-11e5-80f0-d050991b6355: geli: Cannot open gptid/52093a8a-3484-11e5-80f0-d050991b6355: No such file or directory.
]
Feb  5 09:18:21 freenas manage.py: [middleware.exceptions:37] [MiddlewareError: Unable to set recovery key for 2 devices: [MiddlewareError: Unable to set passphrase on gptid/5127d49a-3484-11e5-80f0-d050991b6355: geli: Cannot read passphrase: Inappropriate ioctl for device.
], [MiddlewareError: Unable to set passphrase on gptid/52093a8a-3484-11e5-80f0-d050991b6355: geli: Cannot open gptid/52093a8a-3484-11e5-80f0-d050991b6355: No such file or directory.

What concerns me is the reference to: gptid/52093a8a-3484-11e5-80f0-d050991b6355

That's not in my list if devices that are returned when I did the zpool status. I have a feeling it was the gptid of my /dev/ada6 disk before I had to "replace" it, but don't understand why it's still hanging around now, or if it's presence is going to monkey wrench anything.

 

[dlavigne]

Were you able to resolve this or is it still giving rekey errors?

 

[SnakeByte]

The only way I could resolve this was to hand edit the sqlite db per the steps given in a previous thread here. Once I removed the stale drive entries, everything played nicely again and I was able to generate a new recovery key from the GUI.