Reusing Encryption Keys & Passphrases

[PhiloEpisteme]

PhiloEpisteme submitted a new resource:

Reusing Encryption Keys & Passphrases - How to reuse your encryption keys and passphrase when you replace a pool or replace a drive

Let's get the messy bit out of the way. This resource makes no claim about whether you should or should not use encryption. Discussion on the merits or implementation strategy used in FreeNAS is not relevant to this resource. I'm not looking to advocate for or against encryption.

This resource is intended as an instructional guide for folks to follow as an educational tool to learn more about FreeNAS encryption in a specific use-case. Whether you are comfortable reusing the keys and...

Read more about this resource...

 

[PhiloEpisteme]

In the process of writing and testing the above resource I found some interesting warnings regarding encryption in the User Guide. I wanted to go through some of them and validate what the User Guide is saying and offer some proof as to why the User Guide suggests what it does.

To start, I'll test the following phrase from Adding a Spare Drive.

When adding a spare disk to an encrypted pool the passphrase and recovery key are reset. Click Download Recovery Key after adding the spare device. Then, create a new passphrase by clicking  (Encryption Options) ➞ Create Passphrase. Since creating a new passphrase invalidates the recovery key, click  (Encryption Options) ➞ Add Recovery Key to add a new one.

The specific bit I want to validate here is that the passphrase is reset when you add a spare device. I know that the recovery key cannot be set on the new spare drive because FreeNAS never stores the recovery key and thus couldn't set the same recovery key on the new spare disk.

So, to test I've got my pool named tank. tank has two drives in a mirror setup and uses encryption with a passphrase.

Show : zpool status tank prior to adding the spare

Code:

$ zpool status tank
  pool: tank
state: ONLINE
config:

    NAME                                                STATE     READ WRITE CKSUM
    tank                                              ONLINE       0     0     0
      mirror-0                                          ONLINE       0     0     0
        gptid/59bbf181-ef57-11e9-a95d-00074334f630.eli  ONLINE       0     0     0
        gptid/95517156-ef62-11e9-a95d-00074334f630.eli  ONLINE       0     0     0

I'm going to add a spare to this pool and validate whether it is true that the passphrase is reset and the recovery key is invalidated. To do this I'll follow these steps

1. After adding the spare check to see fi the key found in /data/geli is different from the original. If not, this is an indication that the passphrase was removed but the key was unchanged.
2. After adding the spare reboot the machine to see if a passphrase is required. If not, this is an indication that the passphrase was removed but the key was unchanged.
3. Test the recovery key by temporarily moving the key in /data/geli and attempting to unlock the pool after a reboot using the original recovery key.

Show : zpool status tank after adding the spare

Code:

zpool status tank
  pool: tank
state: ONLINE
  scan: none requested
config:

    NAME                                                STATE     READ WRITE CKSUM
    tank                                              ONLINE       0     0     0
      mirror-0                                          ONLINE       0     0     0
        gptid/59bbf181-ef57-11e9-a95d-00074334f630.eli  ONLINE       0     0     0
        gptid/95517156-ef62-11e9-a95d-00074334f630.eli  ONLINE       0     0     0
    spares
      gptid/1947c21c-ef71-11e9-aac5-00074334f630.eli    AVAIL

While adding the spare I am not prompted for a passphrase during this process. Furthermore, I am prompted to download a recovery key, I suspect that this is either a new primary key (User Key 0) altogether or is the same one previously used but the passphrase has been removed. Lets check it.

Code:

$ diff /data/geli/633f5864-7829-462a-a448-394150b6bb4e.key original.key
$

Alright, so the key didn't change, but perhaps the passphrase requirement did. Let's restart the machine and find out!

On reboot my pool is already unlocked and the zpool status is as follows.

Show : zpool status tank after reboot

Code:

$ zpool status tank
  pool: tank
state: ONLINE
  scan: none requested
config:

    NAME                                                STATE     READ WRITE CKSUM
    tank                                              ONLINE       0     0     0
      mirror-0                                          ONLINE       0     0     0
        gptid/59bbf181-ef57-11e9-a95d-00074334f630.eli  ONLINE       0     0     0
        gptid/95517156-ef62-11e9-a95d-00074334f630.eli  ONLINE       0     0     0
    spares
      gptid/ea7797fa-ef74-11e9-aac5-00074334f630.eli    AVAIL

Alright, I've still got my pool and the drives are all there and unlocked. This tells me that the though the key did not change, the passphrase requirement for the key was removed.

What about the recovery file? Lets temporarily move the key out of /data/geli and reboot the server and try to unlock the pool with our original recovery key.

To test the recovery key lets start fresh to make sure none of our prior steps affected the recovery key. You can do this however you like including destroying the pool and restarting from scratch. Once you've done that add the spare again and then move the key found in /data/geli to a temporary location and reboot the machine. This will result in the system being unable to unlock the pool at startup so we can test our original recovery key.

Show : zpool status tank after adding the spare

Code:

$ zpool status tank
  pool: tank
state: ONLINE
  scan: none requested
config:

    NAME                                                STATE     READ WRITE CKSUM
    tank                                              ONLINE       0     0     0
      mirror-0                                          ONLINE       0     0     0
        gptid/19dd59df-ef7c-11e9-9f0d-00074334f630.eli  ONLINE       0     0     0
        gptid/1baca999-ef7c-11e9-9f0d-00074334f630.eli  ONLINE       0     0     0
    spares
      gptid/3ba328c4-ef7d-11e9-9f0d-00074334f630.eli    AVAIL

Show : Identifying The Encryption Key

Use the following commands to easily identify the encryption key for the pool. You can try other methods such as ls -al /data/geli and look for the most recently created key if you've just created the pool etc. The method here will be the most reliable though because it tells you the name of the file that the system itself will look for.

Code:

$ sqlite3 /data/freenas-v1.db
SQLite version 3.23.0 2018-04-02 11:04:16
Enter ".help" for usage hints.
sqlite> .mode colum
sqlite> .header on
sqlite> select vol_encryptkey from storage_volume where vol_name = 'tank';
vol_encryptkey
------------------------------------
f00c2285-f766-4cee-a0cb-7c105d767c52

Code:

$ mv /data/geli/f00c2285-f766-4cee-a0cb-7c105d767c52.key /data/geli/f00c2285-f766-4cee-a0cb-7c105d767c52.key.bak

On reboot my pool is locked, as expected. If I try to unlock with my recovery key it works but zpool status shows that the spare disk did not unlock. This should be expected because FreeNAS doesn't save the recovery key and thus couldn't set it on the new disk.

Show : Identifying The Encryption Key

Code:

baret# zpool status tank
  pool: tank
state: ONLINE
  scan: none requested
config:

    NAME                                                STATE     READ WRITE CKSUM
    tank                                              ONLINE       0     0     0
      mirror-0                                          ONLINE       0     0     0
        gptid/19dd59df-ef7c-11e9-9f0d-00074334f630.eli  ONLINE       0     0     0
        gptid/1baca999-ef7c-11e9-9f0d-00074334f630.eli  ONLINE       0     0     0
    spares
      3464174391559215341                               UNAVAIL   was /dev/gptid/3ba328c4-ef7d-11e9-9f0d-00074334f630.eli

Conclusion
When you add a spare to a pool the spare is given the same same primary key (User Key 0) but the passphrase requirement is removed across all disks in the pool. Furthermore, because FreeNAS does NOT store the Recovery Key anywhere the new spare drive does NOT have the recovery set. The warning in the User Guide is spot on, as expected. When you add a spare you must reset the passphrase and the recovery key in order to apply them to the new spare. Resetting the recovery key is doubly important because when you reset a passphrase on a pool the recovery keys are invalidated so even the data disks can't be unlocked with the original recovery key once the passphrase has been set.