Upgrading from TrueNAS CORE 12 to 13, Export Secret Seed when backing up config,

[John Childermass]

Hi everyone,

I'm finally getting around to updating from TrueNAS CORE 12.0-U8.1 to 13. I'm planning to do this by switching trains. However, I want to back up the essential config info first, in case something goes wrong.

1) If I backup the config under System > General > Save Config, should I Export Secret Seed, or is it better not to? I'll just be updating in-place, so does that mean I don't need to export the secret seed, as it will remain available on the drive? Or is it safer to export it, as long as I'm careful with where I store this file?

2) I have a single pool, Encryption Type = Key, with all datasets inheriting encryption options from the parent. If I Export Dataset Keys, to get dataset_poolname_keys.json, is that all I need to decrypt the pool and its datasets if something goes wrong?

3) Apart from the config file in (1) and the key file in (2) is there anything else I should backup before doing the update? (Or, in general, that I should store for safekeeping if something goes wrong in the future, to ensure I can recover?)

Thank you in advance!

 

[Davvo]

I am not familiar with encryption, but you can look up here:

Using Configuration Backups

Provides information concerning configuration backups on TrueNAS CORE.

www.truenas.com

I would say yes, export everything including the secret seed to a safe place and then delete it once you are sure you don't need it.

For question 3 I use the multi_report script (link in my signature) that mails my config backup every week.

 

[winnielinnie]

Before any major "upgrade" or critical operation, take advantage of the "checkpoint" feature.

It's done at the pool level. You can "undo" back to the checkpoint if anything goes wrong. If everything looks good? Remove the checkpoint. (Don't forget to remove the checkpoint soon after you confirm everything is good!)



Before the upgrade, migration, or critical operation:

Code:

zpool checkpoint mypool

Confirm it exists (there should be a "size" under the VALUE column):

Code:

zpool get checkpoint mypool

After the operation. Everything looks good? Remove the checkpoint.

Code:

zpool checkpoint -d mypool

Confirm it's gone (there should be no "size" under the VALUE column):

Code:

zpool get checkpoint mypool

If things went sour, you can export the pool and then use the "--rewind-to-checkpoint" option when importing the pool again. (Caveat: You will lose everything that existed after the checkpoint's creation.)



This is not to be confused with snapshots (which are radically different, and done at a per-dataset level), nor should it be confused with "Boot Environments" which only apply to the boot-pool, as a convenient feature provided by TrueNAS itself.

Zpool checkpoints can also undo "feature upgrades".

If I backup the config under System > General > Save Config, should I Export Secret Seed, or is it better not to?

Always have this (up-to-date) config backed up somewhere safe, including the secret seed. It's needed to decipher encryption keys, among other things.

If I Export Dataset Keys, to get dataset_poolname_keys.json, is that all I need to decrypt the pool and its datasets if something goes wrong?

You only need one keystring / passphrase per encryptionroot. If only the top-level root dataset is ever "unlocked" (whether automatically with a keystring or manually with a passphrase), then it sounds like you only have one encryptionroot for the entire pool.

You can confirm it with this:

Code:

zfs list -t filesystem -r -o name,encryption,encryptionroot mypool

Pay attention to the last column. It's likely to list "mypool" for all your datasets.

 

[Davvo]

@winnielinnie I wonder how that works: if I delete a few files, will the checkpoint restore them?

 

[winnielinnie]

if I delete a few files, will the checkpoint restore them?

"Yes"...

...but you shouldn't think of checkpoints like that, nor use them for that reason. (Don't use them like snapshots.)

A checkpoint should be used as a very short-lived safeguard before doing an upgrade or critical operation, and then shortly afterwards it should be removed.

Think of a checkpoint as a legitimate "time machine" for your entire pool. Everything about your pool: files, data, metadata, snapshots, datasets, features, etc. Everything. Yes. Even entire datasets. So for example, if you create a checkpoint, do some sort of upgrade and accidentally delete an entire dataset, you can export the pool and re-import it with the "rewind" option, and viola! You just traveled back in time! Yes, even your permanently destroyed dataset exists once again! (Not even snapshots can bring a destroyed dataset back from the dead.)

One scenario:
Create checkpoint.
Do the "thing". (Good outcome!)
Delete checkpoint.

Alternative scenario:
Create checkpoint.
Do the "thing". (Bad outcome!)
Re-import the pool with "--rewind-to-checkpoint"
(Figure out how to proceed for next time to get a good outcome, or just don't do the "thing".)

Notice in both examples that you soon after make a decision whether to delete the checkpoint or rewind to it. You don't "sit" on the checkpoint for later. You don't start using the pool, writing new files, deleting files, doing a bunch of stuff. You should make your decision about the checkpoint before any of that. (Rewind or discard?)


* A checkpoint can even undo those pesky "non-reversible" zpool feature upgrades. (You know how if you upgrade a pool, you cannot remove the new feature in case you lose compatibility with an older system? Rewinding to a checkpoint will even revert those feature upgrades.)

 

[John Childermass]

Thank you very much, @Davvo and @winnielinnie ! I'm sorry for the delay in responding — I haven't had time to work on it in the last couple of weeks.

I've done what you suggested, saving the config file, and the encryption keys (you were right, there was just the one ENCROOT, which was the pool name).

Unfortunately, when I finally start the update procedure, I'm seeing very slow download speeds, typically around 100 kB/s at most, and once the download reaches only a few percent it fails with an incorrect checksum. Do you know how I can fix this, or do I just need to keep trying?

On https://www.truenas.com/docs/core/coretutorials/updatingtruenas/updatingcore/ I see it says under Manual Updates "You cannot use manual updates to upgrade from older major versions." so I guess that's not an option for me?

 

[winnielinnie]

Perhaps it's an issue with your ISP and/or modem and/or router and/or network card on the TrueNAS machine?

 

[John Childermass]

Maybe... although other speeds through the same router seem fine, and the TrueNAS machine seems to get good enough speeds for other activities. I might try a VPN to somewhere else, based on the discussion in https://www.truenas.com/community/threads/slow-download-of-truenas-scale-iso.112486/ ...

 

[Davvo]

You are not using a Realtek NIC, right?

 

[John Childermass]

Good news! Overnight the download must have somehow managed to struggle through in the background. I came back to a message saying there was an update pending in the UI, and the update seems to have gone smoothly. I followed @winnielinnie's advice (thank you!) and removed the checkpoint once everything seemed good.

You are not using a Realtek NIC, right?

I don't think so... the motherboard is a Supermicro X10SLM-F, and I think both NICs are Intel? I'm using the one that shows up as igb0.

 

[winnielinnie]

Glad to hear, but...

I might try a VPN to somewhere else

...this part was ambiguous. Are you implying you're connected to a VPN service, and that you think you might get better results with a different exit node?

 

[John Childermass]

Glad to hear, but...


...this part was ambiguous. Are you implying you're connected to a VPN service, and that you think you might get better results with a different exit node?

No, I am not usually connected to a VPN service. It was just that the thread to which I linked seemed to suggest that slow downloads might be caused by some weird CDN problems, so I wondered if I would get better results if I (temporarily) used a VPN to pick an exit node somewhere else.