How to create a passphrase protected "pool"?

[Mastakilla]

Hi everyone,

I'm currently working on replacing my TrueNAS Core system with TrueNAS Scale. My TrueNAS Core system has the legacy encryption on the "pool-level" (I know it's not really on the pool, but it applies to all datasets below) and requires me to enter a passphrase when booting up the system.

I'd like that same kind of encryption (but then the non-legacy equivalent) on my TrueNAS Scale as well, so
1) Requiring a passphrase to be entered at boot
2) Not having to worry about setting encryption for each dataset separately

I tried to set this up like this:

But apparently this sets it up using a keyfile instead of a passphrase (without giving the option to use a passphrase).

And when I want to change this later, I'm running into the following problem:

Code:

 Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 427, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 465, in __run_body
    rv = await self.method(*([self] + args))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 44, in nf
    res = await f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/dataset_encryption_operations.py", line 194, in change_key
    verrors.check()
  File "/usr/lib/python3/dist-packages/middlewared/service_exception.py", line 70, in check
    raise self
middlewared.service_exception.ValidationErrors: [EINVAL] id: backuppool contains the system dataset. Please move the system dataset to a different pool before changing key_format.

So what is the proper way of setting up my system dataset with passphrase protection?

 

[Ericloewe]

I'd like that same kind of encryption (but then the non-legacy equivalent) on my TrueNAS Scale as well

Not available. I agree that it's sort of odd, but it was deprecated before Scale was even first released. Key management was a massive pain in the ass, and iX never really fixed it.

 

[Mastakilla]

Thanks for the respone Ericloewe!

Do you mean that it is not possible to set passphrase protected encryption on the "pool"-level dataset?
And that to achieve passphrase protected encryption on all my datasets, I need to
1) Disable encryption when creating my pool
2) Enable passphrase encryption for every dataset that I create in my pool individually?

 

[Ericloewe]

Well, it sounds like you're asking for full-disk encryption, rather than ZFS' native encryption - and the answer to that is "not anymore". You can setup encryption for a pool's root dataset, so that it applies to all data (not all metadata!) in the given pool. If that's fine, we're down to quirks of the workflow (though I'm not sure about the passphrase part, that may be a TrueNAS UI limitation).

 

[winnielinnie]

I believe for SCALE, you cannot use a passphrase if any of the following are involved:

• System Dataset (i.e, .system)
• Apps Dataset (i.e, ix-applications)

Meaning that if you want to use a passphrase for the top-level root dataset, then it has to be for a pool that neither contains the System Dataset nor Apps dataset.

 

[Ericloewe]

If I'm not mistaken, the fundamental design choice there is that the system must boot and load everything automatically, even if storage might have to wait.

 

[sretalla]

Put your system dataset on your boot pool, then do what you want to your other pool.

Remember that pool-level encryption won't obfuscate dataset names no matter what you do, so anyone in possession of all (or enough) disks from your pool can load them up with zfs to look at what you called your datasets... name datasets accordingly.

 

[winnielinnie]

name datasets accordingly.

They might raise an eyebrow if your "organic-food-recipes" dataset is more than 12TB in size.

 

[Ericloewe]

Which is why I find kinda weird that full-disk encryption was dropped. Yeah, ZFS native encryption is a great thing to have, but solves subtly different problems.

 

[Ericloewe]

They might raise an eyebrow if your "organic-food-recipes" dataset is more than 12TB in size.

Have you been to a recipes website lately? I swear, each recipe must need its own VM with all the bloat and infinitely-scrolling crap and mountains of images, plus that unrelated video they throw on. Plus ads, but those are served from elsewhere. 12 TB might get you as many as 100 recipes!

 

[sretalla]

They might raise an eyebrow if your "organic-food-recipes" dataset is more than 12TB in size.

I record my recipes with VERY high definition photos, 3D video of the finished product and copious/long method notes (including 8K video with DTS7.1 sound the whole time it's in the oven), what do you mean 12TB is a lot?

 

[winnielinnie]

Which is why I find kinda weird that full-disk encryption was dropped. Yeah, ZFS native encryption is a great thing to have, but solves subtly different problems.

Even with SCALE it can easily be implemented by keeping the boot drive unencrypted (seamless reboots, access to web UI, etc), wherein the data drives/partitions can be encrypted with LUKS. A GUI to unlock a LUKS container would be trivial to implement. (Caveat: This still wouldn't resolve the issue of "immediately available" Apps and System datasets if you don't want them on the boot pool. In fact, you can't even place Apps on the boot pool.)

Not saying I would use it (as I prefer native ZFS encryption and its flexibility, such as "raw" sends), but I would support the option for those users who wish to rely on a more traditional encryption method.

EDIT: I would file such a feature request for SCALE, but I don't use SCALE (not sure if I will?), and I believe such a feature doesn't align with their scope for TrueNAS.

 

[Ericloewe]

I think it was entirely down to key management. At the time, it would also have meant supporting GELI on FreeBSD and LUKS on Linux, with not that much code reuse between them. I get it, it's a pain and self-encrypting drives address 99.9% of the reasons to have full disk encryption.

 

[Mastakilla]

Thanks for all the responses!

I do wonder if I understand the concept keyfile vs passphrase encryption correctly:

• Keyfile encryption requires TrueNAS to load a keyfile before you are able to access the dataset. This means that keyfile must readable for the TrueNAS OS, so must be somewhere in the OS filesystem or perhaps on USB stick that must be plugged in when you want to access the dataset.
  This is useful only if you want to "protect" your data when somebody steals only your HDDs (and forgets to steal the rest of the server) or when you send your HDDs only to someone, for example to the manufacturer for warranty.
  This doesn't protect your data at all if the HDDs remain together with the server (which has access to the keyfile).
  To be honest, I don't really see any sensible use case in this, so I guess I must misunderstand it somehow. Why would you send "enough" (to rebuild the RAID) of your HDDs at once to someone, like the manufacturer for warranty? HDDs don't die all at once... Also I don't see a thief stealing the HDDs only. They steal the whole server... So what am I missing here?
  Perhaps you can put your keyfile on an USB drive and only insert the USB drive when (re)booting? And try to hide the USB drive while the server is running or powered off? Is that the "sensible use case"?
• Passphrase encryption requires you to enter a passphrase before you are able to access the dataset. This is useful for protecting your data when (re)booting your server. It doesn't require any special hardware or physically hiding something. It only requires you to have a proper way of managing passwords...

Also, I don't mind that people can see the names of my datasets. So, except for the annoyance that I need to fill my dataset passphrase once for dataset (or is there a way around that?), I don't think that it matters me that much if it's full disk encryption or dataset encryption.

All I did so far is create a pool. I didn't start with creating datasets yet. But I do notice that the TrueNAS GUI apparently did create a .system dataset already:

Code:

root@truenas[~]# zfs list
NAME                                                          USED  AVAIL  REFER  MOUNTPOINT
backuppool                                                   24.0M  87.0T   281K  /mnt/backuppool
backuppool/.system                                           16.9M  87.0T   486K  legacy
backuppool/.system/configs-ae32c386e13840b2bf9c0083275e7941   396K  87.0T   396K  legacy
backuppool/.system/cores                                      281K  1024M   281K  legacy
backuppool/.system/ctdb_shared_vol                            281K  87.0T   281K  legacy
backuppool/.system/glusterd                                   339K  87.0T   339K  legacy
backuppool/.system/netdata-ae32c386e13840b2bf9c0083275e7941  13.7M  87.0T  13.7M  legacy
backuppool/.system/rrd-ae32c386e13840b2bf9c0083275e7941       281K  87.0T   281K  legacy
backuppool/.system/samba4                                     658K  87.0T   658K  legacy
backuppool/.system/services                                   281K  87.0T   281K  legacy
backuppool/.system/webui                                      281K  87.0T   281K  legacy
boot-pool                                                    2.35G  50.9G    96K  none
boot-pool/ROOT                                               2.34G  50.9G    96K  none
boot-pool/ROOT/23.10.1.3                                     2.34G  50.9G  2.33G  legacy
boot-pool/ROOT/Initial-Install                                  8K  50.9G  2.33G  /
boot-pool/grub                                               8.20M  50.9G  8.20M  legacy

I don't have a mirrored bootdisk and thought that simply regularly making a backup of your config was sufficient. Is there a reason that the devs store this folder by default to a redundant filesystem? If you loose the OS disk and have a redundant .system, don't you need to reinstall the OS anyway and isn't the having only the config sufficient and as fast for recovery?

Also if apps can't be stored on the boot pool, doesn't that then prevent the trick to move .system to the boot pool and then passphrase encrypt the data pool. Wouldn't that make it impossible as apps shouldn't be stored on encrypted pool according to the devs?

I've also learned to try not go around the devs limitations too often. So would it then be best for me to passphrase-encrypt my data datasets individually and not at the root level dataset, so that .system and apps can stay on the "prefered" redundant / partly-unencrypted pool which is the same pool as for my data? Does that require typing in my password for each encrypted dataset individually or can they be "mass unlocked"?

 

[Mastakilla]

(tiny bump )

 

[sretalla]

Seems you understand it well enough.

Maybe this will help:

SOLVED - ZFS Encryption USB auto unlock and mount with pass-phrase as I do with GLI.

For years I've automatically unlocked on boot and mounted my encrypted geli disks using remotely mounted USB drives as the geli example script below, so If you steel the box without the remote USB's it's useless. So now I need to do the same with the new ZFS encryption since GLI is no longer...

www.truenas.com

 

[Mastakilla]

Thanks for the confirmation / link!

I've now created an unencrypted pool, which has an unencrypted .system dataset by default.
Under that pool, I've created an "unshared-parent-dataset" (named "encrypted-ds") which is encrypted with a passphrase. I've added all of my non-builtin groups with "Traverse"-rights only to the ACL of this dataset.
Under this "encrypted-ds" dataset, I've added all datasets that I want to be encrypted. As they inherit the passphrase encryption from the parent dataset "encrypted-ds", I only have to enter the passphrase once to unlock all of them. I've then added specific group to the ACL of specific datasets, to make sure everyone only gets the permissions they require.

Does this sound like "good practice"? Or am I doing something wrong / suboptimal with this?

I also notice that, when you unlock a GELI passphrase-encrypted "pool" with TrueNAS Core, that it automatically restarts all services and applications. But when I unlock my passphrase-encrypted dataset with TrueNAS Scale, it doesn't do any of that (maybe because I don't have any apps yet?).
So perhaps I need to / can use your instructions from the link above for "solving" this.

 

[sretalla]

Does this sound like "good practice"? Or am I doing something wrong / suboptimal with this?

Sounds like it gets you what you want and there's nothing I can see "wrong" with it.

when I unlock my passphrase-encrypted dataset with TrueNAS Scale, it doesn't do any of that (maybe because I don't have any apps yet?).

I don't expect that's a thing that will happen (haven't tested it myself).

You may want to add some code from here: https://www.truenas.com/community/threads/chart-application-restart-flow.104390/ or use the linked github project or even heavyscript to do that task.

into the automatic unlock script if that's something you want to be automated... otherwise, you need to accept that apps relying on data in encrypted datasets will be problematic and require a manual restart after unlocking.