Confused by native ZFS encryption, some observations, many questions

[winnielinnie]

I have been playing around with TrueNAS Core 12.0 in a virtual machine, specifically to get a "hands on" feel for native ZFS encryption.



Prior to this, I held the following assumptions where GELI and native ZFS encryption differ:

1. GELI encrypts partitions / block devices, which are then used as the bottom layer that ZFS resides above (i.e, "all or nothing" encryption).
2. Native ZFS encryption is per pool / per dataset, irrespective of the block devices underneath (i.e, possible to have a "mix" of non-encrypted and encrypted in the same pool).
3. Native ZFS encryption does not encrypt the ZFS metadata (size, usage, properties, etc), nor does it require unlocking all datasets upon importing a pool.

Here's where things get tricky and confusing. I'm going to withhold copy-pasting text from the terminal for now to keep this from becoming too technical and straining on the eyes:

1. I read about native ZFS encryption from multiple places online (Reddit, Oracle docs, wikis, etc), and there seems to be contradictory claims, such as if you encrypt the root dataset you can never create a non-encrypted child dataset. However, I tried this myself and I was in fact able to create a child dataset without encryption by unchecking the "inherit" option. (An icon with an [X] padlock appears next to the child dataset to signify it is non-encrypted.)
2. If the root dataset is encrypted, what is the actual relationship between it and all its child datasets that inherit its encryption properties? Is there one crypto process that handles all read-writes to these child datasets? (UPDATE: Answered in thread.)
3. For fun, I made a bunch of nested datasets that all use different encryption properties (256, 128, etc), some use keyfiles, some use passphrases, some are outright non-encrypted. For every "break" in a nested child dataset that does not inherit its parent's encryption properties, is this another layer of encryption on top of encryption? (i.e, does it require multiple crypto processes for read-writes, for each different cipher / non-inherited encrypted child?) (UPDATE: Answered in thread.)
4. I read about "encryptionroot" as a ZFS property, which seems to indicate that even if you have many, many child datasets, if they all share (inherit) the same "encryptionroot", this is treated as a single crypto process; and they are all immediately accessible upon unlocking their "encryptionroot" dataset? (i.e, you do not need to "unlock child datasets" if they inherit the dataset you are currently unlocking.) (UPDATE: Answered in thread.)
5. Related to question 1, even if there is a non-encrypted child dataset within an encrypted root dataset, can it still be mounted before unlocking the root dataset? (UPDATE: Explored further in thread.)
6. When no passphrase is ever used to protect the master key, where is the path that the .json file is stored? (It contains the 64-character strings used as keys to decrypt the master key.) I know you can backup the entire pool's "key database" and save it somewhere on a USB stick or client PC, but I'm curious to know where it resides in the local TrueNAS system.
7. I noticed that the swap for each pool is encrypted using GELI; 2GB partitions by default. Is there a technical reason or limitation why the swap cannot be encrypted with native ZFS encryption? Why would the NAS desperately need 2GB of swap space before any pool is imported / unlocked?

Block device encryption is easier for me to intuitively grasp: a block device is encrypted and therefore nothing is accessible or known (not even file system metadata) until the block device is unlocked. Otherwise, anything above the block device layer is random garbage. With native ZFS encryption, it's not as straight-forward: you can change options (compression, atime, etc) without unlocking the dataset or even its parent dataset (or encryptionroot). That baffles my small little mind. How is that even possible?

Where does the data scrambling begin?

Where does it end?

How is it possible to use zfs send / recv to only transfer the differences if the dataset is still locked and hence there should be nothing known about the file and folder structure underneath?

Does using separate encryption options for child / nested datasets (different cipher, different keyfile, different passphrase) mean that there are multiple crypto processes, hence double, triple, etc load for the CPU? (UPDATE: Answered in thread.)

Does using separate encryption options for child / nested datasets (different cipher, different keyfile, different passphrase) work like the equivalent of encrypted block devices within block devices? As in, in order to even access the ones buried underneath, you need to first unlock the parents? (Peeling layers of an onion.) (UPDATE: Answered in thread.)

If the above are true, then what is the point of breaking encryption inheritance with child datasets, since it nullifies any benefit of granular control of what gets encrypted and who has access to what: no trusted person can unlock their "assigned" dataset at will without requiring you to first unlock the parent dataset. Even if they know the passphrase for their own dataset.



Let me know if I can clear up my questions. I'm trying to keep these questions and observations "fresh" in my mind. I don't plan on creating crazy and risky dataset structures with all types of different encrypted child datasets willy nilly. The reason I try to figure these things out is because I believe playing around with something to its extremes gives you a better overall understanding of how things really work underneath.

Should I assume there are three different crypto processes? (Since there are three "encryptionroots").

The following all share the same "encryptionroot": mainpool, mainpool/dogs, and mainpool/dogs/puppies

This means that for all three datasets (reads and writes), there is only one crypto process?

However, for mainpool, mainpool/cats/kittens, and mainpool/cats/kittens/runts (reads and writes) there are three separate crypto processes, (since they are all different "encryptionroots")?

So is it really each "encryptionroot" that ultimately dictates access for its child datasets? Would the analogy be that each "encryptionroot" is the equivalent of an encrypted block device? (Ignoring the fact that metadata is always in plaintext.)

 

[AlexGG]

For fun, I made a bunch of nested datasets that all use different encryption properties (256, 128, etc), some use keyfiles, some use passphrases, some are outright non-encrypted. For every "break" in a nested child dataset that does not inherit its parent's encryption properties, is this another layer of encryption on top of encryption? (i.e, does it require multiple crypto processes for read-writes, for each different cipher / non-inherited encrypted child?)

With native ZFS encryption, no. Only one encryption is applied to each dataset, the one specified in the dataset itself. If nothing is specified, the inheritance hierarchy is traversed towards the root to identify the applicable encryption. If there is none, the data is written in plain. No more than one cipher is applied.

If you use GELI and then put encrypted ZFS on top of it, then (at most) two sets of ciphers will be applied, one at GELI level and up to one at ZFS level, doubling CPU usage. This happens because layers are not aware of each other, and each thinks it is responsible for encryption. Also, in this case, two keys are required to unlock the dataset.

 

[winnielinnie]

With native ZFS encryption, no. Only one encryption is applied to each dataset, the one specified in the dataset itself. If nothing is specified, the inheritance hierarchy is traversed towards the root to identify the applicable encryption. If there is none, the data is written in plain. No more than one cipher is applied.

But if there are multiple "encryptionroots", they are, by definition, using separate master keys to encrypt/decrypt each respective encryptionroot.

Multiple master keys = multiple crypto processes, correct?

In the above screenshot, wouldn't the following hold true?

• ONE crypto process for mainpool, mainpool/dogs, and mainpool/dogs/puppies
• plus
• ONE crpyto process for mainpool/kittens
• plus
• ONE crypto process for mainpool/kittens/runts
• equals
• THREE total crypto processes for the server's CPU

Another way to look at it, these are all of the "encryptionroots" in the pool:

1. mainpool
2. mainpool/kittens
3. mainpool/kittens/runts

Since each encryptionroot can differ in (a) cipher type, (b) cipher size, (c) master key, and (d) keyfile/keypass, then they each require a separate crypto process, right?

Simultaneous reads and writes to "kittens", "runts", and "dogs" means that the CPU is decrypting/encrypting using three separate crypto processes at the same time.

I'm still wrapping my head around this, and the fact that ZFS is some sort of evil sorcery that defies intuition.

 

[jgreco]

It's like compression. The most specific value wins. If you specify mainpool as gzip, and mainpool/kittens as zle, "zle" wins for things written inside mainpool/kittens. It doesn't compress the stuff with zle and then with gzip.

 

[AlexGG]

Simultaneous reads and writes to "kittens", "runts", and "dogs" means that the CPU is decrypting/encrypting using three separate crypto processes at the same time.

Yes, if you simultaneously write to three different datasets each with different encryption schemes, the CPU is doing three separate encryption processes simultaneously. However, every block of data is only encrypted once, with the settings specific for its target dataset. It does not encrypt same data three times. The crypto is not measured in processes, it is measured in number of data blocks encrypted (or amount of data encrypted). Assuming same strength ciphers with different keys, the CPU time spent in encryption is the same if you send three blocks to three different encrypted datasets with different keys, or if you send three blocks to one encrypted dataset.

 

[Patrick M. Hausen]

@winnielinnie There are no "crypto processes" in the same way as there are no "compression processes". The kernel knows what to do to access a particular block of data depending on crypto, compression, ... settings. It all happens inside of ZFS.

 

[winnielinnie]

Thank you all for the clarifications! Now it makes more sense, and it's good to know this is not akin to accessing different traditional encrypted block devices.


The way datasets are presented in the GUI (and even in the command prompt) gives the impression of "layered" blocks of encryption, when in reality each dataset, no matter how deep it is nested, is not dependent on its parents, regardless of encryption cipher or master key. This is probably the reason there was a Reddit post (regarding ZoL 0.8) in which they thought that "creating an encrypted root dataset means that you cannot create any non-encrypted datasets later on, since anything below the encrypted root dataset cannot be accessed until you 'unlock' the root dataset." This doesn't appear to be true, since you can force the creation of a non-encrypted child dataset underneath. (I tried it successfully.)

There was some confusion about this in another site and why it behaves this way. Apparently, a non-encrypted dataset (whether directly beneath a parent dataset or within an encrypted root dataset) can always be "theoretically" accessed / mounted, even if the encrypted parent dataset remains "locked". The reason this is not currently allowed / simplified (let alone encouraged) is due to "security and expectation" reasons. In other words, no matter where a non-encrypted dataset lives, it can always be mounted and accessed, even if it lives "under" an encrypted parent.



That leaves me with a few more questions in regards to the relationship between an "encryptionroot" and its child datasets that inherit its encryption properties.

Why does the hierarchal and tree-branch structure even matter for encrypted and non-encrypted? It seems like it's just a "gesture" at this point, right? Take my example below for instance:

The following is an incorrect assumption:

• mainpool/cats can only be accessed by unlocking mainpool
• mainpool/kittens/runts can only be accessed by unlocking mainpool/kittens (and mainpool)

It is an incorrect assumption because mainpool/cats can (theoretically) be accessed at any time, as its data is not encrypted, and it is not within mainpool's encryption, as you would expect from traditional encrypted block devices (i.e, LUKS). Someone can theoretically force mainpool/cats to mount, even if mainpool remains "locked", in order to access the data within mainpool/cats. (Even though it's not officially supported in the GUI, it is apparently possible, though discouraged from what I understand.)



Using the same example, another way to look at it is like this:

Every single dataset is its own file system and can (theoretically) be accessed independently, regardless how deep it is nested, and regardless of its parent's properties, and regardless of what it "inherits".

The following is a correct assumption? Each dataset is its own independent entity, its own file system not reliant upon the rest of the tree? This is what I am trying to truly grasp and make sure I understand:

• mainpool
  • Cipher: AES-256 (this can NEVER change, selected at creation)
  • Master Key: 1100...1100 (this can NEVER change, randomly generated at creation)
  • Passphrase to access Master Key: canttouchthis123 (may be changed or opt for keyfile instead)
• cats
  • Cipher: none (this can NEVER change, always without encryption)
  • Master Key: none (irrelevant)
  • Passphrase to access Master Key: none (irrelevant)
• kittens
  • Cipher: AES-256 (this can NEVER change, selected at creation)
  • Master Key: 1111...1111 (this can NEVER change, randomly generated at creation)
  • Passphrase to access Master Key: burnedout2020 (may be changed or opt for keyfile instead)
• runts
  • Cipher: AES-128 (this can NEVER change, selected at creation)
  • Master Key: 0001...1001 (this can NEVER change, randomly generated at creation)
  • Passphrase to access Master Key: comehackme (may be changed or opt for keyfile instead)
• dogs
  • Cipher: AES-256 (this can NEVER change, selected at creation)
  • Master Key: 1100...1100 (this can NEVER change, randomly generated at creation)
  • Passphrase to access Master Key: canttouchthis123 (may be changed or opt for keyfile instead)
• puppies
  • Cipher: AES-256 (this can NEVER change, selected at creation)
  • Master Key: 1100...1100 (this can NEVER change, randomly generated at creation)
  • Passphrase to access Master Key: canttouchthis123 (may be changed or opt for keyfile instead)

Notice that mainpool, dogs, and puppies all share the same Cipher + Master Key? Because the Cipher and the Master Key was generated at each dataset's creation, they are permanently set as such. So if you were to "break" the inheritance let's say by changing puppies to a different passphrase, the dataset puppies would still use the SAME cipher and the SAME master key as mainpool and dogs, since that was generated during its creation.


So what is the point of "inheritance" for a dataset's encryption properties? It doesn't create a hard relationship between the parent and its child; it simply just "borrows" the same passphrase, master key, and cipher for the "sake of convenience"? (Or perhaps it only borrows the cipher and passphrase, and still generates a different random master key during initial creation?)


What is preventing someone from creating a non-encrypted child dataset underneath an encrypted parent that is currently "locked"? The metadata and pool structure is not encrypted, and each dataset is its own entity. Are these just arbitrary limits and rules placed in the GUI? Why can't I just create a non-encrypted child under a currently locked encrypted parent? It may look like it lives "within" the encrypted parent, but in reality it is its own independent file system, not contained within an encrypted container, right? (Hint: I cannot do this with the GUI for Storage > Pools, but I sure can if I manually try it in the terminal, even if the root dataset is "locked". This will confuse the GUI later, as it spits out an error when trying to unlock the root dataset afterwards.)


To further the confusion, I am able to create a child dataset with a different cipher (AES-128) from its parent (AES-256), but then later on change it to "inherit" from its parent so that it shows as being "related" to its parent's encryption. Now unlocking the parent will also automatically unlock and mount the child: yet the child is still using AES-128 (as it should!) and still using its own unique master key (as it should!) So what exactly is "inherited"? Just the passphrase/keyfile for the sake of convenience of unlocking/mounting all child datasets simultaneously? The cipher and master key is decided and made permanent upon creation, they can NEVER change.


When you create a new child dataset and accept the default to "inherit" its parent's encryption, does it still generate a random master key different from its parent, and only inherit the cipher and passphrase at the time of creation?


I have to admit, this doesn't seem to be clearly documented anywhere and you have to go fishing to really figure it out (among contradictory claims), hence why I'm asking in here.

 

[AlexGG]

Notice that mainpool, dogs, and puppies all share the same Cipher + Master Key

No, that's not really how it works.

1. Snapshots and clones share the master key of their origin dataset. Or else they can't work really, because snapshots and clones share physical data with their origin dataset, and require the same key to decrypt that shared data.

2. Inherited datasets will have their own master key, while inheriting algorithm and key width (and maybe passphrase? along with passphrase salt maybe?). Because there are other things like HMAC salt, key salt, PKDF2 iteration count, and maybe some more, which are part of the encryption package.

3. The point of inheritance, I guess, is convenience.

4. Nothing is preventing someone from creating a plain dataset under encrypted, as I understand it. Because the dataset is indeed an independent container, not physically contained inside its parent dataset (rather, the two are stored side by side). However, if GUI does not allow this, that's fine by me. I'm not sure the console allows this, but I can't recall anything inside the filesystem which prevents this in principle.

5. If you create a child dataset (128-bit) in parent (256-bit), then child gets its 128-bit master key and parent gets its 256-bit master key. Which are different keys. If you later change back to inherit, the only thing which changes is passphrase. It is not possible to change encryption algorithm, key width, or master key, on a dataset.

 

[winnielinnie]

1. Snapshots and clones share the master key of their origin dataset. Or else they can't work really, because snapshots and clones share physical data with their origin dataset, and require the same key to decrypt that shared data.

Which makes sense. Cipher + Master Key cannot be changed or be different since that is what dictates how data will be read and written.

2. Inherited datasets will have their own master key, while inheriting algorithm and key width (and maybe passphrase? along with passphrase salt maybe?). Because there are other things like HMAC salt, key salt, PKDF2 iteration count, and maybe some more, which are part of the encryption package.

Which ties in the point below as to "what" exactly is being "inherited", and why it's even important to discern and label "encryptionroots" and their respective "inherited children." I realize that TrueNAS is an appliance, and users are encouraged to only make changes and do actions via the GUI, but it would benefit many people if there was a better breakdown of what's really going on under the hood. As it stands now, there's no way in the GUI to review the current details of an encrypted dataset (master key digest, salt, etc). To add to the confusion, the Storage > Pools screen implies that child datasets who inherit encryption from a common parent are supposedly part of the same encrypted family, even if they have different ciphers and key widths. (The analogy of a single encrypted block device which can house multiple file systems.) They cannot be locked / unlocked independently, even though they are independent (outside of the GUI). The ritual of unchecking "inherits" undoes this, but then it also breaks seamless unlocking of multiple datasets at once. See below. ***I believe there's probably a better way to handle and present this in the GUI. You can peruse the web to see that confusion around native ZFS encryption (especially from Linux users, especially when sending/receiving streams) is fairly common.

3. The point of inheritance, I guess, is convenience.

Appears to be that way.

4. Nothing is preventing someone from creating a plain dataset under encrypted, as I understand it. Because the dataset is indeed an independent container, not physically contained inside its parent dataset (rather, the two are stored side by side). However, if GUI does not allow this, that's fine by me. I'm not sure the console allows this, but I can't recall anything inside the filesystem which prevents this in principle.

I believe you can do this in the console, but then it breaks things with the GUI. The GUI seems to hide the flexibility of using encrypted datasets that is possible with ZFS, which I guess is for good reason since "TrueNAS as an appliance" is not meant to compete with FreeBSD + terminal. Though it is odd, and can be frustrating, when you cannot "unlock" an independent child dataset, even when it is not inheriting anything, if it so happens to reside further down the tree and there is a locked parent dataset further up the tree. Seems like an arbitrary restriction. If you try it, you will be prompted for a keyfile/passphrase, only to receive an error message after-the-fact because "XYZ dataset needs to be unlocked first."

5. If you create a child dataset (128-bit) in parent (256-bit), then child gets its 128-bit master key and parent gets its 256-bit master key. Which are different keys. If you later change back to inherit, the only thing which changes is passphrase. It is not possible to change encryption algorithm, key width, or master key, on a dataset.

Exactly, as in point 2.


*** Here's an example I conceived that could handle this issue better in the GUI:

Rather than lump everything into the "inherits" property when creating or editing a dataset's encryption properties, let us "group" encrypted datasets together with the same passphrase so that they can unlock all at once, even if they have different ciphers, master keys, key widths, etc. They can be called "encryption groups" or "unlock groups". This means there would be no arbitrary limitation, and we can independently lock/unlock any datasets at will without losing the convenience of simultaneously unlocking multiple datasets with the same passphrase/keyfile when we desire. Best of both worlds.


I don't want to sound ungrateful or critical, but it seems like the 12.0-RELEASE has a rushed and unpolished implementation of ZFS native encryption. My honest opinion. **** Don't even get me started on the mess that can happen when you have an encrypted root dataset and try to use send/recv.

**** As it stands now, it's safer to always create a separate encrypted dataset one step down from a non-encrypted root dataset, and then treat that as if it's your root dataset. Make sure not to "enable encryption" upon pool creation. It's misleading since the user might believe this is their only chance at encrypting their data and they need to make a decision upon pool creation!

• mainpool<--- don't save data in here
  • cryptroot<--- don't save data in here
    • every
    • thing
    • else
      • will
      • go
    • here
    • and
    • here

This also leaves room to have a separate "root" dataset that is for all the non-encrypted stuff, which makes management more sensible:

• mainpool<--- don't save data in here
  • cryptroot<--- don't save data in here
    • every
    • thing
    • else
      • will
      • go
    • here
    • and
    • here
  • plainroot<--- don't save data in here
    • this
    • stuff
    • is
      • not
      • important

 

[AlexGG]

I'm not sure it will be beneficial. It is certainly possible to introduce the concept of unlock groups. However this will be an addon, and will not match the underlying ZFS implementation. For whatever reason (and most likely there were good reasons I'm not aware of), the filesystem is implemented based on inheritance. Then, there are all sorts of the decisions about how to make it into GUI, which are up to TrueNAS developers, who have their own pretty complex considerations on how to do it. Someone else will probably chime with something, because I do not really work with nested datasets and have no experience with, say, customer support side of this.

 

[gary_1]

Don't even get me started on the mess that can happen when you have an encrypted root dataset and try to use send/recv.

Can you elaborate on this? I've yet to upgrade to 12.x as it's likely going to require a pool wipe. I was planning to have the root dataset encrypted (key/auto-mount) and then zfs send over the rest of the datasets from one of my backups. Followed by creating/copying data to new datasets for the odd one I want password protection on to allow it to be left locked most of the time. Anything I should be aware of that might put a spanner in the works?

 

[winnielinnie]

Can you elaborate on this? I've yet to upgrade to 12.x as it's likely going to require a pool wipe. I was planning to have the root dataset encrypted (key/auto-mount) and then zfs send over the rest of the datasets from one of my backups. Followed by creating/copying data to new datasets for the odd one I want password protection on to allow it to be left locked most of the time. Anything I should be aware of that might put a spanner in the works?

You must enable the "raw stream" (-w flag) if you're going to send encrypted datasets from the new mainpool to your new backups if you want to preserve properties and metadata; otherwise, you will lose encryption on the target, and end up with a non-encrypted dataset nested underneath an encrypted root dataset. Your data is saved in plaintext if this happens, not encrypted.

I have yet to discover what happens if I were to change the passphrase of the original dataset(s) and attempt to send/recv to my backup pool, using the -w flag. Will it spit out an error? Refuse to continue? In theory, it should simply update the new dataset to reflect these changes, but I'm not feeling especially up to the gamble at this moment, considering how new 12.0 is. I might test this out in a virtual machine.

As for the first time you migrate from your backup pool (old 11.x) to the newly created mainpool (new 12.0), you need to enable encryption inheritance on the receiving end with the recv "-x encryption" flag. This will inherit the top level root dataset's encryption properties, which you can then change later (after successfully migrating your datasets over). A passphrase or unique randomly-generated key can be applied in such a case.

Read this thread for another angle on how to approach this: https://www.truenas.com/community/t...eli-to-full-drive-encryption-migration.89337/

 

[gary_1]

You must enable the "raw stream" (-w flag) if you're going to send encrypted datasets from the new mainpool to your new backups if you want to preserve properties and metadata; otherwise, you will lose encryption on the target, and end up with a non-encrypted dataset nested underneath an encrypted root dataset. Your data is saved in plaintext if this happens, not encrypted.

Has a bug report been filed on this?

Seems like it'd be very easy to get caught out by this as I expect quite a few to create an encrypted root dataset and then leave all child datasets as inherited, do a snapshot/replication of the root to a backup site and assume the destination only has access to encrypted data.

Where is this flag enabled? Is it a new option in 12.0 UI for replication tasks?

 

[winnielinnie]

Where is this flag enabled? Is it a new option in 12.0 UI for replication tasks?

I'm assuming when you enable the option "Include Dataset Properties" in the Replication Task page, it will automatically apply the "raw stream" flag if it detects the dataset is encrypted. I'm not entirely sure, as the wording is still vague, and I haven't tried it myself.

I use a manually created script (as an on-demand Cron Job) to replicate all my datasets to a USB drive which I physically store at another location. The GUI complains about everything if I try to do a Local replication. I could never get it to work as an "on-demand" Replication Task.

 

[winnielinnie]

I have yet to discover what happens if I were to change the passphrase of the original dataset(s) and attempt to send/recv to my backup pool, using the -w flag. Will it spit out an error? Refuse to continue? In theory, it should simply update the new dataset to reflect these changes, but I'm not feeling especially up to the gamble at this moment, considering how new 12.0 is. I might test this out in a virtual machine.

I went ahead and tested this in a virtual machine (12.0-RELEASE), and it does in fact behave as I suspected "in theory". So that's good news! It means you can freely change the encryption properties of the source dataset(s), and everything will work normally.

In other words, for the source dataset you can change the keyfile, give it a passphrase, change the passphrase, re-inherit the parent's encryption properties, etc, but as long as you send with the raw (-w) flag, the destination will receive it like normal and update the encryption properties on its end to reflect the changes you made.

At least this gives the user more flexibility in managing their datasets' encryption properties after filling them up with files and folders, without having to worry about ruining send/recv operations in the future.

 

[I like to move IT]

I junst want to say that i realy like this topic and the way the topic is discussed. I am new to ZFS (but with Napp-IT) and i would like to store my data in an encrypted filesystem, so its neccessary to understand how ZFS encryption is working to handle it in a safe way.

Thanks. Please keep it up!

 

[bb182]

I read about native ZFS encryption from multiple places online (Reddit, Oracle docs, wikis, etc), and there seems to be contradictory claims, such as if you encrypt the root dataset you can never create a non-encrypted child dataset. However, I tried this myself and I was in fact able to create a child dataset without encryption by unchecking the "inherit" option. (An icon with an [X] padlock appears next to the child dataset to signify it is non-encrypted.)

The restriction on creating unencrypted children of encrypted datasets (including the root dataset) was removed, at least partially in response to the Reddit post you mentioned.

When no passphrase is ever used to protect the master key, where is the path that the .json file is stored? (It contains the 64-character strings used as keys to decrypt the master key.) I know you can backup the entire pool's "key database" and save it somewhere on a USB stick or client PC, but I'm curious to know where it resides in the local TrueNAS system.

Keys are stored in the database (/data/freenas-v1.db). They are in the table storage_encrypteddataset. Like other credentials in the database, they are encrypted (key is at /data/pwenc_secret) so that a configuration backup is less sensitive. The json file is generated on the fly when you go to export the keys.

 

[winnielinnie]

Keys are stored in the database (/data/freenas-v1.db). They are in the table storage_encrypteddataset. Like other credentials in the database, they are encrypted (key is at /data/pwenc_secret) so that a configuration backup is less sensitive. The json file is generated on the fly when you go to export the keys.

Solid! Thank you!

 

[winnielinnie]

Going to paste this in here from another thread, as it is relevant to the overall topic:

I understand the GELI, I think. It was encrypting the pool at the disk level, and anything that was in that pool was thus encrypted when the pool or disk was detached.

With GELI (and thus 11.3 and earlier), what was really happening when you chose to create a pool with encryption, was that partitions were being created and encrypted for each physical device in the vdev (ada0p2, ada1p2, ada2p2, etc). None of the encryption is done via ZFS itself. These encrypted block devices (ada0p2, ada1p2, ada2p2, etc) are decrypted with a keyfile or passphrase upon re-importing them, which makes them available for the zpool import; and naturally there is now transparent access to the ZFS metadata and data. Without first decrypting these partitions, there would be no zpool to import since the metadata is not yet available (just a bunch of "random" garbage.)

With the new native ZFS encryption, none of that applies anymore. The block devices (e.g, physical partitions) are left as-is, used to build vdevs, which are used to create a pool, and then any such encryption is left to the end-user on a per dataset level. Importing the pools does not first require any block devices to be decrypted: you can import a zpool with native ZFS encryption on its datasets and simply "skip" the prompt when it asks if you want to unlock the datasets upon import. This means you'll have successfully imported a zpool with a bunch of locked (inaccessible) datasets. With GELI, the step that requires decrypting the block devices first is mandatory: you cannot skip this step when trying to import a zpool.

The new ZFS encryption is at a data set level and may or may not inherit from the parent?

Pretty much yes. The way it is presented in TrueNAS 12 gives you the impression that you are creating an "encrypted pool" when you first create a pool. This is not true. It is simply giving you the option to encrypt the very top-level root dataset (which shares the same name as the pool name). This does not dictate what you can and cannot do with any other child or parent datasets later on. However, if you leave the default options when creating new datasets, it will by default "inherit" its parent's encryption options: when the new dataset is created it will generate a new Master Key for itself, inherit the Cipher of the parent (e.g, AES GCM), inherit the Key Width of the parent (e.g, 256-bit), inherit the Keyfile or Passphrase of the parent, and inherit the Iterations of the parent.

The following CANNOT be changed later, at any time:

• Master Key
• Cipher
• Key Width

The following CAN be changed after the dataset's initial creation, without affecting the saved data, files, and folders, simply by unchecking the "Inherit" box for the dataset's encryption properties:

• Keyfile
• Passphrase
• Iterations

TrueNAS 12 hides some of the underlying functionality from the end-user for the sake of behaving as an appliance, rather than an operating system that is configured and tweaked as you would a distro such as Ubuntu Linux or FreeBSD. This is why certain things seem counter-intuitive. Everything is supposed to be done in the GUI, not via the terminal. While it's possible to create your own crazy zpool and dataset hierarchies that mix and nest different types of encryption (and non-encryption) all over the place, it's not recommended and strays away from a smoother workflow. You could technically "create" a new zpool with an encrypted top-level root dataset, and then create a non-encrypted dataset immediately underneath it. The state of the root dataset (locked vs unlocked) does nothing to protect the data in the nested non-encrypted dataset. The GUI may give the impression it does, because the parent is "locked" and intuitively we assume that whatever is nested underneath a locked parent is inaccessible. (This is not the case, as no matter how deep a child dataset is nested, if it is not encrypted its files and folders will always be accessible in the plain.)

My goal is just to not need to worry about what's on the disk if I need to send it in for warranty replacement or when I retire it.

There is a caveat concerning native ZFS encryption vs traditional block device encryption: what is exposed to the public. I'll give my own summary understanding comparing three common options:


Native ZFS, at rest (or powered off):

• Inaccessible
  • File Data
  • File Names and Properties
  • Sizes of individual files (unsure about this one)
  • Directory listings and structures
  • Logs (if logs are saved here)
• Accessible
  • Encryption Options and Hashes
  • ZFS Metadata and Options
  • Number of Files and Blocks (via pointers and inodes) [1]
  • Dataset Names
  • Snapshot Names
  • Free Space
  • Used Space

LUKS, at rest (or powered off):

• Inaccessible
  • File Data
  • File Names and Properties
  • Directory listings and structures
  • Sizes of individual files and folders
  • Number of files and folders
  • Logs (if logs are saved here)
  • Free Space (can be inferred if underlying devices were not filled with random data prior to creation)
  • Used Space (can be inferred if underlying devices were not filled with random data prior to creation)
  • File System and metadata within
• Accessible
  • Encryption Options and Hashes

VeraCrypt container, at rest (or powered off):

• Inaccessible
  • File Data
  • File Names and Properties
  • Directory listings and structures
  • Sizes of individual files and folders
  • Number of files and folders
  • Logs (if logs are saved here)
  • Free Space (container filled with random data upon creation)
  • Used Space (container filled with random data upon creation)
  • File System and metadata within
  • Encryption Options and Hashes
• Accessible
  • Apparently [nothing, just a suspicious mess of random data [2]

[1] Seems like native ZFS encryption is "leakier" than I first suspected, but it makes sense considering it requires certain information and metadata in order to send and recv, create snapshots, scrub, etc, without requiring encrypted datasets to be unlocked.

[2] Until decrypted, a VeraCrypt partition/device appears to consist of nothing more than random data (it does not contain any kind of "signature"). Therefore, it should be impossible to prove that a partition or a device is a VeraCrypt volume or that it has been encrypted.

 

[gary_1]

Assume you have a pool has the root dataset encrypted with a key and then you have various jails and/or services that make use of data from child datasets that are password encrypted and thus not available until manually unlocked.

Is there any "smarts" in the system to know/prevent starting a jail or service until all datasets it depends on are mounted/unlocked?