Recover encryption key

Status
Not open for further replies.

klayman

Dabbler
Joined
Nov 6, 2013
Messages
14
Hello,

little big problem: I had to replace my USB boot stick as it was not booting properly anymore (showing F1: ######## indefinately). No things are back up again and I was able to restore my configuration db from the old stick (by mounting it to a temp folder and copying the db file itself). Unfortunately this doesn't seem to work for the encryption keys of the ZFS volume. When trying to unlock the volume it fails with

NAS manage.py: [middleware.notifier:3030] Importing zpool [7425633488718000201] failed with: cannot import '7425633488718000201': no such pool available
NAS manage.py: [middleware.exceptions:38] [MiddlewareError: Volume could not be imported: 5 devices failed to decrypt]

Any ideas? The old stick is still available and I can copy files from it...

thanks,
Klayman
 

Yatti420

Wizard
Joined
Aug 12, 2012
Messages
1,437

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
It "should" be working if you have the right key. It sounds like something is wrong if its not working for you.
 

klayman

Dabbler
Joined
Nov 6, 2013
Messages
14
can you tell me where I can find the key itself on the original USB stick?

thanks,
Klayman
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
You don't. You download the key via the GUI when you setup the pool. You should have setup the password and key per the manual, then stored those "in a safe place" as losing the key means losing your pool. There is no way to brute force the encryption if you lose the key.

If you didn't setup the password you might find the key on your old stick in /data. But that's not guaranteed.
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
The recovery key is not on the stick. The "main" key is on the fourth partition in the geli directory. If possible, you should copy the content of that directory into /data/geli in the new installation.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
The recovery key is not on the stick. The "main" key is on the fourth partition in the geli directory. If possible, you should copy the content of that directory into /data/geli in the new installation.

Can you elaborate on this? When I talked with one of the developers about keys the key was stored in /data somewhere until you specified a password(I forget the exact location.. just woke up). Once you've done that there is no key on the USB sticks at all anywhere.
 

klayman

Dabbler
Joined
Nov 6, 2013
Messages
14
well, as I can shutdown or reboot the server there must be some key stored somewhere. Of course I need to enter the passphrase (which I still know) but as far as I know that only "unlocks" the key itself.
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
I don't see the keys being deleted. The key is there so that you do not need to reupload it every time you reboot the box (you only need to enter the passphrase). Without setting a passphrase, the key in /data/geli enables automout of encrypted pools.
You can check this yourself. SSH into your FreeNAS box and check /data/geli. Compare (byte-by-byte) the .key file there with the key you downloaded via GUI.
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
well, as I can shutdown or reboot the server there must be some key stored somewhere. Of course I need to enter the passphrase (which I still know) but as far as I know that only "unlocks" the key itself.
Actually the master AES key is stored encrypted in the geli metadata on the disk and you are not able to access it via the GUI (you can retrieve it by running geli backup <provider> <file>). To unlock/decrypt that key you need the key file you can download (and is stored in /data/geli) and the passpharese (if set).
 

klayman

Dabbler
Joined
Nov 6, 2013
Messages
14
Got it. I followed Dusan's suggestion and copied the original geli directory into /data. After clicking on unlock and entering the passphrase everything is fine again. Although I learned my lesson and backed up encryption key and config db ;-)

thanks again!!
Klayman
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
I'm not available to check this out myself right now, but are we saying that the keys are always available from the USB stick?
 

klayman

Dabbler
Joined
Nov 6, 2013
Messages
14
yes and no. the key itself is there but it is only useful with the correct passphrase. If you haven't set one then yes, everything to decrypt is on the thumbdrive.
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
Yes, the key file you get when you click the Download Key button is always stored in /data/geli (4th partition on the USB stick). This means you do not need to provide the file when you reboot the server and want to unlock your pool -- the passphrase is enough. You do need to upload the key file when you change/overwrite the USB stick.

More details on how this works:
Every geli provider ("disk drive") has its own AES master key that can never be changed -- you would have to rewrite (reencrypt) all data on the drive to change it. The master key is stored in the geli metadata on the drive itself. This key is, of course, encrypted. There are two "slots" for the encrypted master key in the geli metadata, which allows you to encrypt the master key with two user keys (0 and 1). Any of the user keys (if set) can be used to decrypt the master key and thus access the data. You can change the user keys as often as you want provided that you have at least one of them -- you decrypt the master key with the old user key and encrypt it with the new one. An user key can be created from two components -- a passphrase and/or a key file -- you can use one, the other, or both.

FreeNAS uses the user keys as follows:
  • user key 0 (the "main" key) always has the key file component which is stored in /data/geli. It optionally can also have the passphrase component. If the passphrase is set then both the key file and the passphrase are needed to unlock the pool (decrypt the master key).
  • user key 1 (the "recovery" key) is optional and can only have the key file component (the middleware actually supports a passphrase here as well, but the GUI doesn't use that functionality).
This is how all this maps to FreeNAS operations and GUI buttons (when I say "sets a key" it means that the master key is encrypted with the key and stored in the respective user key slot in the geli metadata):
  • When you create an encrypted pool the master keys are initialized on all drives, a random key file is generated, stored in /data/geli and set as the key file component of user key 0. There is no passphrase component and the pool will automount after reboot.
  • The Create passsphrase button adds the passphrase component to user key 0. The pool will no longer automount after reboot as the key file stored in /data/geli is not enough anymore (it is now just one component of user key 0). You now need to provide the passphrase via the Unlock button.
  • The Change passphrase button changes the passphrase component of user key 0.
  • The Download key button gives you the key file component of user key 0 (it just lets you download the file from /data/geli).
  • The Encryption Re-key button generates a new key file, stores it in /data/geli and sets it as the key file component of user key 0.
  • The Add recovery key button generates a new key file, sets it as the user key 1 (overwriting anything that was there before) and allows you to download it. This key file is not kept stored anywhere in FreeNAS.
  • The Remove recovery key button removes the user key 1 from the geli metadata.
 

panz

Guru
Joined
May 24, 2013
Messages
556
This is the clearest explanation I have ever read. Thank you!
 

DoubleE

Cadet
Joined
May 18, 2013
Messages
1
I agree with panz that this is the best explanation I have seen. Thanks!

I'd like to suggest that this be added to the FreeNAS user manual, maybe in an appendix. All that's needed is a cut and paste - nothing fancier needed!
 

denisov

Cadet
Joined
Apr 17, 2015
Messages
1
hi.
when i try "geli setkey -p -k /data/geli/0949dc47-dfcb-4b07-8b2f-f42055bb36b3.key -n 1 ada0p2" it says "Operation not permitted".
why and how to set secondary key (slot 1) as passphrase without keyfile. thks.
 

Craig2

Cadet
Joined
Aug 22, 2015
Messages
2
Yes, the key file you get when you click the Download Key button is always stored in /data/geli (4th partition on the USB stick). This means you do not need to provide the file when you reboot the server and want to unlock your pool -- the passphrase is enough. You do need to upload the key file when you change/overwrite the USB stick.

I don't have a "geli" folder in my /data folder. I've looked on the 4th partition as well as the 2nd partition (where most of the other files seem to be located). I think this was version 9.1.3. Not sure as it's failing to boot now. Was the location of the main key moved?
 

troun

Dabbler
Joined
Jul 13, 2013
Messages
33
Yes, the key file you get when you click the Download Key button is always stored in /data/geli (4th partition on the USB stick). This means you do not need to provide the file when you reboot the server and want to unlock your pool -- the passphrase is enough. You do need to upload the key file when you change/overwrite the USB stick.

(...)
  • user key 0 (the "main" key) always has the key file component which is stored in /data/geli. It optionally can also have the passphrase component. If the passphrase is set then both the key file and the passphrase are needed to unlock the pool (decrypt the master key).
(...)

Hey Dusan, I really wanted to thank you, not first time that one of your post/reply helps me. :)
You are always calm, respect and helpful, pedagogic and not judging in your posts.

Just a small precision in case it may help, about keys in /data/geli folder; they need to be renamed with proper ID number. This ID number appears in log/consol if you try to decrypt without any file ("IDnumber.key not found"). (Denisov)
It may also needed to create a geli sub-folder in data folder (Craig2)
 

Juan Manuel Palacios

Contributor
Joined
May 29, 2017
Messages
144
Yes, the key file you get when you click the Download Key button is always stored in /data/geli (4th partition on the USB stick). This means you do not need to provide the file when you reboot the server and want to unlock your pool -- the passphrase is enough. You do need to upload the key file when you change/overwrite the USB stick.

More details on how this works:
Every geli provider ("disk drive") has its own AES master key that can never be changed -- you would have to rewrite (reencrypt) all data on the drive to change it. The master key is stored in the geli metadata on the drive itself. This key is, of course, encrypted. There are two "slots" for the encrypted master key in the geli metadata, which allows you to encrypt the master key with two user keys (0 and 1). Any of the user keys (if set) can be used to decrypt the master key and thus access the data. You can change the user keys as often as you want provided that you have at least one of them -- you decrypt the master key with the old user key and encrypt it with the new one. An user key can be created from two components -- a passphrase and/or a key file -- you can use one, the other, or both.

FreeNAS uses the user keys as follows:
  • user key 0 (the "main" key) always has the key file component which is stored in /data/geli. It optionally can also have the passphrase component. If the passphrase is set then both the key file and the passphrase are needed to unlock the pool (decrypt the master key).
  • user key 1 (the "recovery" key) is optional and can only have the key file component (the middleware actually supports a passphrase here as well, but the GUI doesn't use that functionality).
This is how all this maps to FreeNAS operations and GUI buttons (when I say "sets a key" it means that the master key is encrypted with the key and stored in the respective user key slot in the geli metadata):
  • When you create an encrypted pool the master keys are initialized on all drives, a random key file is generated, stored in /data/geli and set as the key file component of user key 0. There is no passphrase component and the pool will automount after reboot.
  • The Create passsphrase button adds the passphrase component to user key 0. The pool will no longer automount after reboot as the key file stored in /data/geli is not enough any more (it is now just one component of user key 0). You now need to provide the passphrase via the Unlock button.
  • The Change passphrase button changes the passphrase component of user key 0.
  • The Download key button gives you the key file component of user key 0 (it just lets you download the file from /data/geli).
  • The Encryption Re-key button generates a new key file, stores it in /data/geli and sets it as the key file component of user key 0.
  • The Add recovery key button generates a new key file, sets it as the user key 1 (overwriting anything that was there before) and allows you to download it. This key file is not kept stored anywhere in FreeNAS.
  • The Remove recovery key button removes the user key 1 from the geli metadata.

Hi Dusan,

As others, I also want to thank you for this incredibly clear and useful explanation of how FreeBSD GELI encryption works and how FreeNAS uses it.

I just have two questions, one for confirmation and the other one because I can't figure it out by myself:

  1. When FreeBSD decrypts the master key to access the volume, that decrypted copy of the key only ever exists in RAM, correct?
  2. If user keys 0 and 1 are completely independent of each other (they are, right?), why does the FreeNAS guide tell you that you need to *immediately* generate/re-generate a new recovery key when setting/re-setting the passphrase on user key 0 (even to the point of warning you that might lose access to your data if you don't do so)? If they are indeed fully independent, doing whatever you could think of to user key 0 should not affect user key 1's ability to decrypt the volume, if I'm understanding your explanation correctly. Is it that user key 1 is somehow generated off of user key 0 (cf. your "An user key can be created from two components -- a passphrase and/or a key file -- (...)" comment)? Or is there something else that I'm misunderstanding?
Thanks!
 
Status
Not open for further replies.
Top