Today I ran into a problem and found the solution from this post: Can't Re-Key Encryption After Replace/Re-Silver.
Quoting the OP from the above thread:
The difference for my particular scenario vs. the aforementioned post was that the drive was offlined by FreeNAS automatically according to the GUI and this problem persisted.
Let me explain further.
Quoting the OP from the above thread:
My pool is set up as follows:
RAID-Z2-0
(6)3TB drives
RAID-Z2-1
(5)3TB Drives
(1)4TB Drive(This is the new disk that replaced a 3TB and just finished re-silvering)
RAID-Z2
(6)4TB Drives
After re-silvering takes place I' try to follow the steps from the freeNAS manual here:
http://doc.freenas.org/9.3/freenas_storage.html#replacing-an-encrypted-drive\
However when attempting to re-key the pool I get the following error:
"Error: Unable to set key: [MiddlewareError: Unable to set passphrase on gptid/d1a6e94d-9f9f-11e4-b99f-000c2934e0ad: geli: Cannot open gptid/d1a6e94d-9f9f-11e4-b99f-000c2934e0ad: No such file or directory. ]"
Part of the blame lies with me I suspect b/c I removed the old 3TB drive with out off lining it first, but I figured it would just simulate a HDD dying. I've replaced like 3 other hard drives that have failed (this one didn't fail but is being replaced one at a time to extend the vdev) and never had this problem despite not being able to mark them offline.
Also how the hell do you lose access to your whole pool if you reboot before re-key (warning according to freeNAS man.)
Any help is greatly appreciated.
Found this Thread:
https://forums.freenas.org/index.ph...ive-unable-to-set-key-geli-cannot-open.14554/
Seems he found a way to remove info for old drive, but I cant make heads or tales of the commands he was using.
[root@freenas] /data# sqlite3 /data/freenas-v1.db "select * from storage_disk;"
for me trying that get a y/n/e/a/prompt and replying yes gets permissions denied.
Edit #2: Finally got it figured out.
FIRST USE:Code:sqlite3 /data/freenas-v1.db "select * from storage_encrypteddisk;"
Then Identify the one giving the error and use
Code:sqlite3 /data/freenas-v1.db "delete from storage_encrypteddisk where id=3;"
and replace the '3' in "where id=3" with the correct number of the offending entry.
The difference for my particular scenario vs. the aforementioned post was that the drive was offlined by FreeNAS automatically according to the GUI and this problem persisted.
Let me explain further.
- I have a bunch of WD RED NAS 6TB drives connected to LSI HBA's on P20 firmware that are fully supported by FreeNAS.
- I am running the latest FreeNAS stable build at the time of this post. (9.10-STABLE-201606270534)
- One WD RED NAS 6TB drive fails on a RAID Z3 volume.
- Failed drive was automatically offlined by FreeNAS (there was no option in the GUI to offline it). This is expected behavior and I've replaced other failed drives like this in the past. When there is no offline option, FreeNAS has already done this for you, or so it seems. No big deal.
- I pop a replacement drive in and run through the re-silver. All is well.
- Re-sliver completes. I attempt to re-key the volume, and get the errors listed in the above referenced thread regarding ""Error: Unable to set key:"
- Solution given in edit #2 of above referenced thread allows me to remove the bad drive from the freenas-v1.db file, re-key the volume, and finally create and download a new geli.key and geli_recovery.key.
- Code:
sqlite3 /data/freenas-v1.db "select * from storage_encrypteddisk;" sqlite3 /data/freenas-v1.db "delete from storage_encrypteddisk where id=x;"
- I reboot and test both my passphrase and recovery key and everything works as expected.