PhiloEpisteme
Guru
- Joined
- Oct 18, 2018
- Messages
- 969
I've been interested in writing a bit of a detailed encryption guide for some time. The idea is to walk folks through how they can play with and manipulate the keys in their system to understand how FreeNAS makes use of encryption so folks could better make an informed decision about whether to use it.
In the process of putting this together I came across some odd behavior. What follows below is a bit of demonstration to show expected behavior and then steps to show the odd behavior.
Disclaimer: If you're reading this in the hopes of using this information to recover data in a lost pool, stop. Before you follow any of these instructions on a pool with data on it post your exact plan to the forums so folks can verify you're not about to lock yourself out of your pool. These are not steps one should generally take in normal operation of FreeNAS.
Other Disclaimer: If you're following along at home and my steps don't perform as expected make sure you're on the same version. While it is possible that I've made a mistake below I tried to be very diligent about testing what I wrote as I wrote it to confirm behavior. If I you think I made a mistake though definitely point it out, it is definitely possible as the steps below are somewhat involved.
The System
FreeNAS uses confusing terminology when it comes to encryption etc.
Setup
Now, for something odd
What about testing that exporting/importing did not invalidate our keys somehow when we import with
Okay, time for my final theory. Perhaps FreeNAS tracks whether a pool should make use of a passphrase at all or not? I figured I could test this by creating a pool without a passphrase, using
Pause for a question. What gives with being unable to change the password using geli above? I did some searching around the forums and found several answers that seemed similar that suggested things related to
It gets a bit more unexpected; after unlocking my pool using the passphrase in the prior steps clicking the lock icon does not show an option to lock the pool. Why doesn't the lock icon appear? This suggests my hypothesis is partially correct, that FreeNAS somehow knows that this pool shouldn't have a passphrase and so does not provide the option to lock it.
I've searched around and played with variations on the above and have been unable to figure this out. I don't see anything in
Edit: Corrected a typo found by @blueether
In the process of putting this together I came across some odd behavior. What follows below is a bit of demonstration to show expected behavior and then steps to show the odd behavior.
Disclaimer: If you're reading this in the hopes of using this information to recover data in a lost pool, stop. Before you follow any of these instructions on a pool with data on it post your exact plan to the forums so folks can verify you're not about to lock yourself out of your pool. These are not steps one should generally take in normal operation of FreeNAS.
Other Disclaimer: If you're following along at home and my steps don't perform as expected make sure you're on the same version. While it is possible that I've made a mistake below I tried to be very diligent about testing what I wrote as I wrote it to confirm behavior. If I you think I made a mistake though definitely point it out, it is definitely possible as the steps below are somewhat involved.
The System
- FreeNAS 11.2-U5
- 1 encrypted pool 1 mirrored vdev of 2 disks
FreeNAS uses confusing terminology when it comes to encryption etc.
0.key
is what I will call the key that you find in /data/geli
and may be referred to as /data/geli/<original>.key
. It is the same key as the one you download when you first create a pool and click Download Recovery Key
. It is also the same key you download when you click the lock and select Download Encrypt Key
. This is the key which is typically associated with geli's slot 0, hence the name.1.key
is what I will call the key that you download when you click the lock icon and select Add Recovery Key
. This is the key typically associated with geli's slot 1, hence the name.Setup
- Create an encrypted pool
- Add a passphrase to the pool
- Download the primary key via
Download Encrypt Key
as0.key
- Create and download a recovery key for the pool via
Add Recovery Key
as0.key1.key
- Lock and unlock the pool using the passphrase and
1.key
to verify everything is working
- Export the pool (don't destroy the data on the pool)
- Import the pool using the primary key (
0.key
) passphrase pair - After successful import lock the pool
- Unlock the pool using the passphrase
- Lock and then unlock the pool using the recovery key,
1.key
Now, for something odd
- Export the pool (don't destroy the data on the pool)
- Import the pool using the recovery key (
1.key
). Note that here we don't use the passphrase because this key does not require a passphrase. - After successful import see that you cannot lock the pool. This is somewhat expected because the key we used to import the pool has no passphrase.
- Move the key found in
/data/geli/<original>.key
to/data/geli/<original>.key_bak
. If you have more than one pool this key can be identified by trial and error or by checking thestorage_volume
in/data/freenas_v1.db
. - Upload the original primary key
0.key
via whatever tool you prefer, I usedscp
and store it in/data/geli/<original>.key
- Restart the machine. As expected the pool will be locked because the key you uploaded in the previous step requires a passphrase.
- Attempt to unlock your pool using the passphrase, it won't work! Puzzling behavior here!
- Attempt to unlock your pool using the recovery key,
1.key
, it works.
User Key 0
and User Key 1
and whichever succeeds is the slot against which FreeNAS uses to check the key found in /data/geli
. If this is true, swapping out the key in /data/geli
shouldn't work because FreeNAS wouldn't check the key against the correct slot. Then, on further reflection I thought this unlikely, when you use geli attach
to unlock a device you don't provide the slot you just provide a key file and if that key file requires a password geli prompts you for it and handles the rest. I tested it nonetheless.- Create a new encrypted pool with a passphrase and download both keys as before
- Export and reimport your pool using
0.key
and the passphrase - Copy
1.key
to/data/geli/<original>.key
- Restart the machine and notice that your pool is automatically unlocked. FreeNAS was able to use
1.key
to unlock the pool without an issue.
/data/geli
.What about testing that exporting/importing did not invalidate our keys somehow when we import with
1.key
- Create a new pool with a passphrase. Download both keys,
0.key
and1.key
. - Export the pool
- Import the pool using
1.key
- Move
/data/geli/<original>.key
to/data/geli/<orignal>.key_bak
- Restart the machine, your pool should start out locked.
- Using
scp
upload both0.key
and1.key
- Using geli, try to unlock your disks with
1.key
withgeli attach -k 1.key /dev/gptid/<provider>
. All of the disks should work. - Lock those same disks again with
geli detach /dev/gptid/<provider>.eli
- Try to unlock them again using
0.key
exactly as before. You should be prompted for a password and it should work.
Okay, time for my final theory. Perhaps FreeNAS tracks whether a pool should make use of a passphrase at all or not? I figured I could test this by creating a pool without a passphrase, using
geli
directly to add a passphrase to 0.key
, restart the machine and see what happens.- Create a new pool without a passphrase. Download both
0.key
and1.key
- Add a passphrase to
0.key
using the following command on each disk in the poolgeli setkey -n 0 -k /data/geli/<original>.key -p -K /data/geli/<original>.key /dev/gptid/<provider>
. The above command givesgeli: Provider <provider> invalid
.More puzzling behavior here!
vfs.zfs.vol.mode
but they were fairly old and the links to bug reports they filed were in the old bug system and I cannot find them in the new system. I was unable to find any provider which would work while the encrypted volume was unlocked and mounted.Move/data/geli/<original>.key
to/data/geli/<original>.key_bak
and restart the machine so the pool and therefore disks are locked.Change the passphrase as above withgeli setkey -n 0 -k /data/geli/<original>.key -p -K /data/geli/<original>.key /dev/gptid/<provider>
for all disks in the pool.ls /dev/gptid
to verify that no providers are attached, ie nothing there is prepended with.eli
Move the key back in placemv /data/geli/<original>.key_bak /data/geli/<original>.key
Attempt to unlock with the passphrase. It succeeds!
geli setkey -n 0 -k /data/geli/<original>.key -p -K /gata/geli/<original>.key /dev/gptid/<provider.
or geli setkey -n 0 -K /gata/geli/<original>.key /dev/gptid/<provider.
. Both seem to succeed so I must have made a mistake the first few times through.- After changing the password on the providers restart the system and try to unlock using the password, it should work.
1.key
then placed 0.key
into /data/geli
the passphrase failed but when I manually set a passphrase on the disks using geli
and then tried to unlock using the passphrase it worked. So it seems that sometimes FreeNAS will function with the passphrase and sometimes it will not.It gets a bit more unexpected; after unlocking my pool using the passphrase in the prior steps clicking the lock icon does not show an option to lock the pool. Why doesn't the lock icon appear? This suggests my hypothesis is partially correct, that FreeNAS somehow knows that this pool shouldn't have a passphrase and so does not provide the option to lock it.
I've searched around and played with variations on the above and have been unable to figure this out. I don't see anything in
/data/freenas_v1.db
that suggests the system is aware of when and how to use a passphrase. I found old threads suggesting that the failure to change the passphrase using geli in my first attempt may be related to vfs.zfs.vol.mode
but the bug referenced does not exist and is for a very old version of FreeNAS.Edit: Corrected a typo found by @blueether
Last edited: