SOLVED replication task and encryption probably just understanding issue

[John Doe]

Hi folks,

can you help me to understand the issue


Situation is as following (white background is main truenas "non remote", black background is remote truenas "remote")






My Non Remote Site has not encrypted dataset which shall be transported to my remote truenas server with encrypted datasets.
This shall serve as backup but encrypted.

My understanding with ZFS native encryption (not GELI) is, that eventhou the datasets are encrypted, replication task can still perfom, so that the dataset can be locked all time while perfoming replication task.

so my question is, is my understanding true?
And in case it is, does the root dataset be unlocked but the dataset the replication task is pointing to can be locked?

I am just getting this 2 different errors and cannot really figure out why.

 

[John Doe]

up

 

[sretalla]

The errors you're getting seem pretty definitive and describe exactly what the problem is... you have a dataset named the same as the replication job target already in existence and it's an unencrypted dataset, so can't be used to receive an encrypted dataset.

 

[John Doe]

thanks for your reply.
maybe I should have mentioned, that it ran just fine the first time, issue came up on the 2nd run.

My understanding with ZFS native encryption (not GELI) is, that eventhou the datasets are encrypted, replication task can still perfom, so that the dataset can be locked all time while perfoming replication task.
so my question is, is my understanding true?
And in case it is, does the root dataset be unlocked but the dataset the replication task is pointing to can be locked?

 

[sretalla]

You’re talking about the source, the error messages (and I) were talking about the target side.

I don't think the locked status of the source is important.

 

[John Doe]

Sorry I do not get your point.
The source datasets are not encrypted at all. Maybe check the atached pictures in the 1st post.

I want to know if the target dataset needs to be unlocked to be able to receive the data froma replication task and how is it with the root dataset of remote end?

 

[sretalla]

My point doesn't change about the error in the first message... you're sending over test2 (a child of test) which is encrypted, but there's already a test2 on the target in the way and it is not encrypted.

For the other error, it seems you're wanting to do incrementals, which will indeed require the key to be loaded on the remote side in order to understand which snapshots are currently there.

 

[John Doe]

I do still struggle with it.

once I unlock the remote end, everything just runs fine.
My understanding of zfs implemention of encryption is, that the dataset can be copied to the remote end WITHOUT having the remote datasets unlocked but for some reason I get the above mentioned messages when remote end is locked.

I know that feature is somehow new and very detailed knowledge but I just want to know, is my above understanding correct.

forget about the error messages for now.

 

[winnielinnie]

once I unlock the remote end, everything just runs fine.
My understanding of zfs implemention of encryption is, that the dataset can be copied to the remote end WITHOUT having the remote datasets unlocked

For raw streams, this is true.

However, in your case, you're not doing a raw stream (i.e, encrypted source -> raw stream (same everything) -> destination).

Therefor, you either need to first unlock the target (or target's parent) before replicating, or you need to provide the key (used by the target side) when configuring the replication task on the source side. Otherwise, how will the replication task know how to encrypt the plain data before it reaches the target?

 

[winnielinnie]

once I unlock the remote end, everything just runs fine.

Because on the remote end, you're unlocking the parent dataset "test", which appears to have been created as an encrypted dataset, prior to running any replications. So now everything depends on "test" being unlocked beforehand.

 

[John Doe]

...or you need to provide the key (used by the target side) when configuring the replication task on the source side.

I actually did that.

replication task on source has the password of the locked receiving side but it tells me cannot receive incremental stream: inherited key must be loaded

 

[winnielinnie]

I actually did that.

replication task on source has the password of the locked receiving side but it tells me cannot receive incremental stream: inherited key must be loaded

Because you're going from RAIDz2 (source) to test (target). In other words "test" is acting like your target's counterpart of the source's "RAIDz2".

Did test exist prior to any of this? Did you create test as an encrypted parent dataset?

UPDATE: Not to mention you have the top-level root dataset of your target ("Samsung1tb") also encrypted. It cannot remain locked in order to receive a plain stream that you wish to target further down the hierarchy / nest.

 

[John Doe]

ah okay i might got it.

so on target side I can have non encrypted dataset or must have unlocked encrypted datasets to receive the stream?

In case I have no encrypted datasets on target side but the tick mark for encryption within the replication task, all data is encrypted (not only for the transfer)?

 

[winnielinnie]

so on target side I can have non encrypted dataset or must have unlocked encrypted datasets to receive the stream?

Correct. Unless you are using "raw streams", but this requires the source to be encrypted. "Raw stream" is a feature that allows you to send an encrypted dataset/snapshots "as is" without requiring anything to be unlocked on the target, nor will the target user have the access key to unlock it. (Unless you send them the key or share your passphrase.)

In case I have no encrypted datasets on target side but the tick mark for encryption within the replication task, all data is encrypted (not only for the transfer)?

Yes, it should be. The parent dataset will be "non-encrypted", but this doesn't matter, since the children nested underneath will be encrypted.

You can see this from the icons or by the command zfs list -r -t filesystem -o name,encryption poolname

UPDATE: Oh no! I had hit "Send" yesterday, but apparently this forum was having server issues, and thus my reply was never sent!

 

[John Doe]

thanks a lot for posting again. that really helped!