SOLVED ZFS Encryption USB auto unlock and mount with pass-phrase as I do with GLI.

[Paul5]

For years I've automatically unlocked on boot and mounted my encrypted geli disks using remotely mounted USB drives as the geli example script below, so If you steel the box without the remote USB's it's useless. So now I need to do the same with the new ZFS encryption since GLI is no longer supported.

*******************************************************************************

#!/bin/sh
#Obsolete for new encryptions from Truenas >12
#decrypts on boot up encrypted usb/HDDs > Mounts them > executes net command.

geli attach -j /mnt/USB5/Utilities/passkeyfile/passkey -k /data/geli/tkd89_blah_blah_blah_xyz.key /dev/gptid/rff_blah_blah_blah_xy

zpool import -R /mnt HDD1

#Exits the script
exit 1

************************************************************************************

How would I do the same using the new ZFS encryption? Where would I find the keys for example, though I think with the new I should only encrypt with a password but still need to point it to it's location and which disk/dataset to decrypt and mount.

 

[sretalla]

It depends what you were using encryption for.

Using ZFS native encryption with a passphrase requires intervention to unlock on boot. For Key encryption, it will automatically unlock on boot without the need for your intervention with a script.

I haven't seen anyone identifying the key locations, but that's probably not hard to discover.

If you set a passphrase, you could implement a script similar to your current one using the API:

midclt call pool.dataset.unlock ... (look up in API docs for the required items)

 

[Paul5]

It depends what you were using encryption for.

To protect from physical theft amongst other. ZFS key auto unlock fails 100% in physical theft of the unit. It also prevents you from decrypting and mount a dataset when a specific IP is present.

Using ZFS native encryption with a passphrase requires intervention to unlock on boot. For Key encryption, it will automatically unlock on boot without the need for your intervention with a script.

I understand all that, ZFS is one or the other. Passphrase provides greater protection even if you use a remote script to auto unlock on boot.

I haven't seen anyone identifying the key locations, but that's probably not hard to discover

Either a json file or db file but of no use to me, to insecure providing no physical security. I have a couple of remote interlocked USB's. You steel the box and get nothing not even what is normally public on the network, everything gets locked down.

If you set a passphrase, you could implement a script similar to your current one using the API:

1 I Wouldn't have a clue How to implement it and non of the forum posts have a simple solution as I once put on how to it with Geli here, my post is the last one: https://www.truenas.com/community/threads/unlock-encrypted-volumes-after-reboot.59976/#post-575115 Relatively idiot proof, within reason for other like me.

2 I Have found many attempts by others of which those whom look like they may have succeeded have programming knowledge but have not put a simple how to for those of us who have no clue how to use an API yet alone implement one and from the looks of they are called externally, I need it to happen internally on boot/reboot.

3 It also looks like the API requires the passphrase in plain text which I and from what I read many other do not like and prefer using an obscure file in a remote USB for example.

4 Can you even call an API on boot with an init scrypt to decrypt a dataset for example.

5 I typed your suggestion <midclt call pool.dataset.unlock> into what looks like the new web manual (I came from 11.3 with different manual style) and I get nothing of any value, had to use 11.3 manual. I can't even find the API tab in Truenas as one image shows so I went to ( /api/docs/ ) and I do not have the knowledge to even comprehend how to manipulate, implement or even call an API.

I put an example of a simple GELI method I use above using a bash script as I have absolutely no clue on API's, for example what the hell would you do with this:

curl "https://$host/api/v2.0/pool/dataset/unlock" -k -X POST -H "accept: */*" -H "Content-Type: application/json" -H "Authorization: Bearer 1-$API_TOKEN" -d '{"id": "$pool","unlock_options": {"key_file": false,"recursive": false,"toggle_attachments": true,"datasets": [{"name" : "$pool" , "passphrase" : "$pass"}]}}'

And that's one simplistic example of all the attempts by others I can find. To old to tired in looking for a simple answer to a simple problem!

Unfortunately some of us need complete idiot proof examples that we can edit. The answer is available, just finding the right person who's done it to come across the forum.

 

[sretalla]

I can't even find the API tab in Truenas as one image shows so I went to ( /api/docs/ ) and I do not have the knowledge to even comprehend how to manipulate, implement or even call an API.

You can call the API with the middleware client (midclt) using the call function and then the API you want... hence the hint I gave you midclt call pool.dataset.unlock ..

I guess I should be grateful that your query pushed me to finally work out how to do more complex API calls (the ones that need more than one input variable), so here's what you need:

midclt call pool.dataset.unlock tank/data/encrypted '{ "key_file": false,"recursive": false,"datasets": [{"name" : “tank/data/encrypted” , "passphrase" : “somepassword”}]}’

Since you're running that locally on the server, you don't need any of the messy authentication, but you're indeed right in thinking you need to get the password from somewhere and you don't really want it on the server itself... I can see the use case of "I yank out that USB stick and then nobody can use that encrypted data (after a reboot)".

So, I'm apparently feeling very kind today as I've worked through the nightmarish hell of single, double and slanty quotes to get to a result which I think is pretty much what you had (although to be noted still has the passphrases stored in a file whether on a USB or whatever, so no better off than you were, but you have the removable option):

2 files:

/mnt/USB5/mypass.passphrase

Code:

our $USB_password = "some_preferably_long_and_complicated_passphrase_you_set_on_your_dataset";

/root/unlock.pl (you could locate that anywhere on your unlocked datasets too if you have any).

Code:

#!/usr/bin/env -S perl -w
# print "doing work";
do_Work();



sub do_Work {
    my $config_file='/mnt/USB5/mypass.passphrase';
    my $password;
    if (do $config_file)
       {
        $password = $USB_password
    } else {
      die "password file not found. can't proceed";
    }

   my @command = ("midclt call pool.dataset.unlock tank/data/encrypted", "\'\{ \"key_file\"\: false\,\"recursive\"\: false\,\"datasets\"\:\ \[\{\"name\"\ \:\ \"tank/data/encrypted\"\ \, \"passphrase\"\ \:\ \"$password\"\}\]\}\'");

   # print join(' ', @command);

   my @result=run_command(@command);

}
sub run_command {
    my @cmd = @_;
    my $out;
    # print join(3, 'run_command: '.join(' ', @cmd));
    my $command = join(' ', @cmd);
    $out = `$command`;
    print $out, "\n";
    chomp($out);
    return split(/\n/, $out);
}

You would need to set permissions on the .pl file for execute with chmod 755 unlock.pl or something like that.

You need to change the path (twice) in the command line in the .pl file too... I'll leave it to you to work out handling it with multiple datasets if that's what you need.

Once you tested that to your satisfaction, you would run it as a post-init script.

All the usual caveats... if you blow up your server running this, it's not my fault. (if you happen to like the script, that's OK though)

 

[Paul5]

sretalla, Thank you a 1000 times over. I'm half way through modifying it to suit and I can tell you that there is absolutely no way in hell that I would have ever been able to do what you did.

When I first looked at your code I went numb, all the forward and backward brackets etc... so I went back to my concept of zfs load key and mount. It worked but TrueNas needs the 'midclt call pool.dataset.unlock' for it must do other things during load and or after mount such as services. That failing I went back to your code and now that I have a better understanding of it (no programming knowledge whatsoever) You did an excellent job of writing it for the layman. Other than my mods all I had to do to make run was add

Code:

#!/usr/local/bin/perl       <Wouldn't work Unless I pointed it to Perl directory.>

and

Code:

`/address/to/MyOtherFileToLaunch.pl &`;       <So that I could launch other scripts and keep going.> 

One last question. When entering the script from the CLI on completion I always get a number 'XXX' for example. Always different number. What does it mean?

Again, thanks for helping, if not for you I would have to remain on the 11.3 train for ever.

 

[sretalla]

#!/usr/local/bin/perl <Wouldn't work Unless I pointed it to Perl directory.>

I admit that I tested that on TrueNAS SCALE, but not core, although I had thought I had previously tested it to work on both... apologies there.

One last question. When entering the script from the CLI on completion I always get a number 'XXX' for example. Always different number. What does it mean?

The returned number is a task/job ID from the system.

You could use it to query for the state of the job while it's running (probably not very long in this case) or after it's finished (success or fail).

You can see all the jobs listed out (formatted nicely if you use jq) and find that number id in the list:
midclt call core.get_jobs | jq

 

[sretalla]

#!/usr/local/bin/perl <Wouldn't work Unless I pointed it to Perl directory.>

I eventually worked out a version of that which works on both CORE and SCALE:
#!/usr/bin/env -S perl -w

 

[ovidius1980]

Hello.
I would like to pose a question. Can this work only for storage disks or for the os disk? So can I use this in order to not use a passphrase before booting freebsd ?

 

[sretalla]

Can this work only for storage disks or for the os disk? So can I use this in order to not use a passphrase before booting freebsd ?

If you can figure out a way to launch enough of a shell with zfs modules loaded and launch some kind of script to do the work, then I suppose so...

It was designed/tested with TrueNAS, so with the understanding that there's little value in encryption of the boot pool and no method to launch for the boot pool, so never tried that and not interested in getting involved with it.

EDIT: also relies on a middleware call, so specific to TrueNAS. to replicate that, you'll need to replace the call with zpool unlock or whatever equivalent.