Recovering zfs rename'd encrypted datasets missing on boot

AJCxZ0

Dabbler
Joined
Mar 11, 2020
Messages
13
Summary: I zfs renamed several encrypted datasets, rebooted and middleware can't find the keys to decrypt them. How do I get my encrypted datasets available on boot with their new names?

Details:
All operations were through the Web UI of my TrueNAS-12.0-U2.1 system except for the renaming of datasets with zfs rename from the shell. I've omitted what I understand to be irrelevant details including other unencrypted datasets, shares using the renamed datasets, how I learned that datasets with spaces can't be shared by NFS and how everything including the Web UI seemed to work perfectly until I rebooted.
  1. Created encrypted datasets A Drive, B Drive, C Drive, D Drive, Backup and VM Offline Datastore. in pool Atlantic.
  2. Renamed VM Offline Datastore to Datastore
  3. Created datasets Datastore/Offline and Datastore/Online (inheriting encryption from parent)
  4. Created unencrypted dataset Drive
  5. Renamed A Drive to Drive/A, etc.
On boot the renamed encrypted datasets were not mounted.

Code:
# zfs list -o name,keystatus,encryptionroot,mounted,mountpoint Atlantic/Backup Atlantic/Datastore{,/Online,/Offline} Atlantic/Drive{,/M,/P,/S,/X}
NAME                        KEYSTATUS    ENCROOT             MOUNTED  MOUNTPOINT
Atlantic/Backup             available    Atlantic/Backup     yes      /mnt/Atlantic/Backup
Atlantic/Datastore          unavailable  Atlantic/Datastore  no       /mnt/Atlantic/Datastore
Atlantic/Datastore/Offline  unavailable  Atlantic/Datastore  no       /mnt/Atlantic/Datastore/Offline
Atlantic/Datastore/Online   unavailable  Atlantic/Datastore  no       /mnt/Atlantic/Datastore/Online
Atlantic/Drive              -            -                   yes      /mnt/Atlantic/Drive
Atlantic/Drive/M            unavailable  Atlantic/Drive/M    no       /mnt/Atlantic/Drive/A
Atlantic/Drive/P            unavailable  Atlantic/Drive/B    no       /mnt/Atlantic/Drive/B
Atlantic/Drive/S            unavailable  Atlantic/Drive/C    no       /mnt/Atlantic/Drive/C
Atlantic/Drive/X            unavailable  Atlantic/Drive/D    no       /mnt/Atlantic/Drive/D


I learned from the kind folks at iXsystems that the key management is handled by the middleware service and that it was unaware of the changed from my renaming, hence the keys for the renamed datasets were not found. The

Code:
# fgrep '(ERROR)' /var/log/middlewared.log
...
[2021/04/02 13:19:39] (ERROR) ZFSDatasetService.check_key():673 - Failed to check key for Atlantic/C Drive
[2021/04/02 13:19:39] (ERROR) ZFSDatasetService.check_key():673 - Failed to check key for Atlantic/A Drive
[2021/04/02 13:19:39] (ERROR) ZFSDatasetService.check_key():673 - Failed to check key for Atlantic/B Drive
[2021/04/02 13:19:39] (ERROR) ZFSDatasetService.check_key():673 - Failed to check key for Atlantic/D Drive
[2021/04/02 13:19:39] (ERROR) ZFSDatasetService.check_key():673 - Failed to check key for Atlantic/VM Offline Datastore
[2021/04/13 15:47:18] (ERROR) PoolService.import_on_boot():1785 - Unlocking encrypted datasets failed for Atlantic pool with following datasets Atlantic/Datastore, Atlantic/Drive/C, Atlantic/Drive/A, Atlantic/Drive/B, Atlantic/Drive/X


I would like my encrypted datasets with their current (i.e. renamed) names to be unencrypted/unlocked/mounted/... at boot and everything to work as it did before I rebooted. I presume this involves configuring the middleware service to reference the keys with appropriate names consistent with having originally creating them, but as a noob to TrueNAS [which should surprise no-one at this point] I am hesitant to even try changing anything. How precisely do I achieve this?

If details of my hardware, vdevs, pool or other information is needed, then I'll be happy to provide it.
 

AJCxZ0

Dabbler
Joined
Mar 11, 2020
Messages
13
Renaming the locked dataset to its original name and rebooting did not result in the middleware reassociating the key and unlocking it.
What worked was "simply" unlocking the dataset with its new name in the web UI by
  1. clicking the tricolon (),
  2. selecting Unlock,
  3. unselecting ☑ Unlock with Key file,
  4. pasting the 65 character key on top of Dataset Key (and ignoring the file select),
  5. clicking [SUBMIT], and
  6. clicking [CONTINUE] on the Unlock Datasets dialog
The unlocked renamed dataset is subsequently unlocked on boot, as desired.

One-and-a-half things which confused me were steps 3 and 4: the default ☑ Unlock with Key file, as uploading the key as text in a file results in an error about expecting JSON with a Python error stack, and the while the small text in the UI does say to paste a key, it looks like it's (still) expecting a file upload.
 
Last edited:
Top