How to migrate two encrypted systems with replication from 9.10.2-U6 to 11.2-U4.1?

[Spacemarine]

I am happily running two 9.10.2-U6 installations in different locations. I replicate data from one the other every day and both installations are encrypted to protect against physical theft.

I want to upgrade to 11.2-U4.1. I bought 4 new USB-sticks, two for each installation so I can make fresh installations and go back to the old (unchanged) system in case anything goes wrong. Is this assumption correct?

I want to be extra cautious as these systems contain sensitive data. I am therefore worried if I need to take some special care about the encryption or the replication?

Can I just set up replication between the two systems after the re-installation and it will just continue to work where it has stopped? I don't want to copy the entire data again.

Is there anything to consider about the encryption besides what is mentioned in the manual?

Thanks!

 

[cods69]

Apparently you should not update directly from previous advice given.
As for a guided upgrade path through specific versions, I have not seen any documented other than at the suggestion level.
One given to me was manual upgrade (tar) to:
11.0-U4
then go to:
11.1-U6
Whether this still stands, I have no idea sorry. I chickened out of my upgrade and I'm not using encryption.

 

[Ericloewe]

Here's a list of everything you need to have backed up before attempting this:

• Your primary key and its associated passphrase
• Your backup key
• Optionally: GELI metadata for every single encrypted disk (this allows you to recover from a GELI destroy command)

tl;dr out of the way, let's address some details:

I bought 4 new USB-sticks, two for each installation so I can make fresh installations and go back to the old (unchanged) system in case anything goes wrong. Is this assumption correct?

Mostly yes. Pool upgrades may complicate this somewhat, but aren't immediately necessary and aren't always a problem.
As an aside, SSDs are a better option.

I want to be extra cautious as these systems contain sensitive data.

Sensitive as in "definitely shouldn't be seen by others" or as in "definitely cannot be lost"? The latter can be effectively mitigated, the former is a trade-off between perceived security and ease of recovery (I say perceived because a 5 $ wrench can decrypt a surprising amount of data).

Can I just set up replication between the two systems after the re-installation and it will just continue to work where it has stopped? I don't want to copy the entire data again.

If no funny business happened, yeah. You don't even need to set anything up, just upload your old config and be done with it. Everything should work.

Is there anything to consider about the encryption besides what is mentioned in the manual?

If you're not absolutely sure that you followed the manual when you last replaced a disk, re-key the pool before attempting any of this and then back up the keys.

Apparently you should not update directly from previous advice given.
As for a guided upgrade path through specific versions, I have not seen any documented other than at the suggestion level.
One given to me was manual upgrade (tar) to:
11.0-U4
then go to:
11.1-U6
Whether this still stands, I have no idea sorry. I chickened out of my upgrade and I'm not using encryption.

This sort of stuff really irritates me. It's frustrating that this may be a problem and even more so that it's unclear if it is a problem.

Honestly, I'd give the direct upgrade a try and see if everything is working correctly.

 

[cods69]

This sort of stuff really irritates me. It's frustrating that this may be a problem and even more so that it's unclear if it is a problem.

Honestly, I'd give the direct upgrade a try and see if everything is working correctly.

I completely agree, it's irritating, especially when people read contradicting views on what should be an upgrade path with backing by, and consensus with, the developers/testers.
Case in point:
https://www.ixsystems.com/community/threads/11-2-who-has-installed.71696/page-6#post-498311

I don't want to seem ungrateful as we are all using a free product and many contribute to testing in ways, however a "best stable case" upgrade path, backed by dev/testers/forum mods, would be greatly appreciated by the many, where focus on nuances could be highlighted in focused forum threads, rather than sprinkles of guesswork.

 

[Spacemarine]

Sensitive as in "definitely shouldn't be seen by others" or as in "definitely cannot be lost"? The latter can be effectively mitigated, the former is a trade-off between perceived security and ease of recovery (I say perceived because a 5 $ wrench can decrypt a surprising amount of data).

In the former sense (and to a lesser extent also in the latter sense, but since I only upgrade one system at a time, so this is not really a concern). I know that it is easy to make people give up their password by threatening them, but this is not the scenario that I want to protect myself from. I am more concerned of my systems getting stolen and curious eyes digging through them. Or being falsely accused of something and having the police take my equipment away and go through my data. Encryption protects me perfectly well from these scenarios.


Anyhow, I managed to upgrade both systems successfully. If someone want to know how I did it:

I just bought new USB-sticks and did a completely fresh install. The primary key was saved to a different computer and the passphrase was in my head like always. After the install I re-entered all the settings manually and re-keyed the pool. Everything works perfectly now. I will keep the old USB-sticks for a while so I can go back any time until I update the pool.

 

[Ericloewe]

and re-keyed the pool.

Why? That doesn't get you anything. Did you backup your keys (including the recovery key) afterwards?

 

[Spacemarine]

Why? That doesn't get you anything.

You are probably right. I wanted to make sure that there is no recovery key floating around somewhere, that could fall into the wrong hands. But removing the recovery key would probably have been enough.

Did you backup your keys (including the recovery key) afterwards?

I backed up the main key, but I did not create a recovery key. I rely on always remembering the passphrase. I know that if I forget it, all my data will be lost. But that's a risk I'm willing to take.

 

[cods69]

In case anyone comes across this in the future, I found this recommendation in another thread.
https://www.ixsystems.com/community/threads/roadmap-freenas-11-1-u8.77005/#post-535613