Local Backup via Replication Task fails

[yggdrasil]

Hi all,

I'm trying to set up a Replication Task on my FreeNAS box to back up my pool "NAS" to another pool "bckup" on the same box. To my surprise there's no possibility to set this up easily in the GUI, so I'm trying a replication to localhost. I followed the "Replication Tasks" section in the pdf docs, but the task errors out with "Permission denied (public key,password)".
Are there any pitfalls I'm not aware of?
Why isn't there a GUI option for local backup?

Any help appreciated.

 

[cyberjock]

I used the documentation on Sunday just to see if it was accurate and I can validate that it is accurate. I did know what I was doing (for the most part.. you don't set up replication regularly) but I had no problems. So to be honest I have to think that this is user error.

Unfortunately that error could be caused by quite a few things being wrong. My advice is to go back and undo everything you've done for push and pull and attempt to use the manual slowly and logically.

 

[yggdrasil]

I basically did exactly that, too, that's why I wonder if I made a some basic mistake. I'll redo the process soon and document exactly what I'm doing, maybe some can spot the error then. And yes, I very much suspect a user error as well ;)
I still don't understand why there isn't a simpler GUI option for this.

 

[cyberjock]

You can only simplify it down so far. Go to far and you start leaving yourself open to security problems "behind the WebGUI cloak". To be honest, the current configuration is about as simple as you are going to get... right now. Some new things coming may simplify it down a bit more.

 

[yggdrasil]

I try to write down exactly what I do here.

I have a user, yggdrasil, which already has a public key in autherized_keys{,2}, and my 2 pools, NAS and bckup. I want to replicate NAS to bckup, so I setup a snapshot task for all the NAS pool. I click on Storage -> ZFS Replication -> Add Replication. Here I enter the following:

Volume/Dataset: NAS
Remote Dataset: bckup
Recursively Replicate: yes
Initialise: yes
Remote Hostname: localhost
Dedicated User enabled: yes
Dedicated User: yggdrasil
Fast ciphers: yes
Rest is left at default.

Now, I'm already not sure (and the docs don't seem to clarify this): the dedicated user, is he used on PULL, PUSH, or both? I'm working under the assumption that it's both. I already disabled root login in ssh and password login without public key. I click ssh keyscan and click ok.

Now, I copy the key from View Public Key, go to the yggdrasil user and paste it in SSH public key under the already present one.

Now I check, and it works. WTF? I did the exact same thing as several times before?! Well, it ran now for quite a while, but only replicated a few megabytes. I still have the feeling that this indirect local copy via ssh is more complex than it needs to be for local backup. I'll leave it running over night and report back tomorrow.

 

[panz]

I have the same configuration. One "main" pool and one "backup" pool. I completely abandoned the automatic replication from the FreeNAS' GUI and choose to join the "by hand" send / recv process and I'm very happy.

Just take a snapshot from the GUI; name it something like 2014-08-21-h-1630. Then send it recursively to your backup pool, with a CLI syntax like:

zfs send -Rv main@2014-08-21-h-1630 | zfs recv -Fdv backup

Open a separate terminal window (or use Tmux) and monitor the sending / receiving process with

zpool iostat 1

which will give you some bandwidth statistics.

Next time you want to do a backup take another snapshot from the GUI, let's name it 2014-08-22-h-2030

Now we are ready to send the differences from the first snapshot and the second one:

zfs send -Rv -i main@2014-08-21-h-1630 main@2014-08-22-h-2030 | zfs recv -Fdv backup

If you want to verify what you're sending before committing the operation add the "dry run" switch "-n" on the receiving side, in a matter that the recv CLI part is:

| zfs recv -Fdvn backup

and FreeBSD will print out what it is going to do with your command line, without writing to the backup pool. If all is ok delete the -n switch and run the command.

 

[yggdrasil]

panz: thank you, but I already knew all that. I was thinking for a bit about writing my own cron job instead, but I really wanted to find out what mistake I committed in the GUI. The initial zfs send is still not finished, so I'll wait and see how it goes.

I still maintain that for purely local backups, there should be a separate GUI menu without having to mess about with public keys and ssh itself.

 

[yggdrasil]

Alright, after the initial send finished, it started sending the new snapshots as well, abeit quit slowly and for some reason, it sent old snapshots as well? At least according to the status c0lumn. After that, it errored out again with "cannot receive incremental stream: most recent snapshot does not match incremental source". I'm now trying to initialise the bckup pool manually and see what I can do with that.

Might there be problems, that I have not only a global snapshot task to the NAS pool (that I have only for the replication task), but also "local" snapshot tasks for single datasets?

 

[panz]

And then...

https://bugs.freenas.org/issues/5293

 

[yggdrasil]

Thank you panz, I'll probably leave it to manual replication at the moment...

 

[Apollo]

If you want to verify what you're sending before committing the operation add the "dry run" switch "-n" on the receiving side, in a matter that the recv CLI part is:

| zfs recv -Fdvn backup

Nice find, I was looking for that very same solution but I couldn't find the option for it.

 

[Apollo]

Alright, after the initial send finished, it started sending the new snapshots as well, abeit quit slowly and for some reason, it sent old snapshots as well? At least according to the status c0lumn. After that, it errored out again with "cannot receive incremental stream: most recent snapshot does not match incremental source". I'm now trying to initialise the bckup pool manually and see what I can do with that.

Might there be problems, that I have not only a global snapshot task to the NAS pool (that I have only for the replication task), but also "local" snapshot tasks for single datasets?

Check that you have proper letter case, if the snapshot has lower or upper case, the commend must list the snapshots exactly the same.

The reason it cannot receive incremental stream is that the receiver doesn't see the snapshot (difference in letter case) or the option you used to do the replication didn't involve keeping the intermediary snapshots, only the last one, maybe.

 

[yggdrasil]

Check that you have proper letter case, if the snapshot has lower or upper case, the commend must list the snapshots exactly the same.

The reason it cannot receive incremental stream is that the receiver doesn't see the snapshot (difference in letter case) or the option you used to do the replication didn't involve keeping the intermediary snapshots, only the last one, maybe.

Since I used the GUI options, this should not be the problem I'm seeing.

 

[DaPlumber]

I'm happily replicating locally using only the GUI, but note:

1) Even with the new "Cipher" of "Disabled" you still need to get the key correct, mine currently looks like:

localhost ssh-rsa
blahblahblahkeystringgobbldygookmanylines
...
localhost ecdsa-sha2-nistp256
blahblah...​

I did NOT get those strings from "View Public Key", I think I dug them out of root's key files.
2) I haven't done any testing, so left lz4 compression on.
3) I always wound up having to check the "initialize remote side once" checkbox whenever I changed anything, So I'm a little cautious breaking the replication link for anything other than a true failure. Once you lose the snapshot stream it doesn't seem possible to catch up, even if the intermediate snaps are available.
4) If you're going to to use USB, USB3, or eSATA for the backup pool (like I do) be aware that there are limitations and risks and whining about it when they bite you in the butt will get you nothing but derision from the CyberPooch, (and me for that matter!) :p

 

[panz]

Local replication (using the GUI) is not as reliable as it should be.

 

[DaPlumber]

Nor near as flexible as it could be either. There should be benefits to tracking snaps in a DB beyond slowing down the GUI. ;)

At a minimum I'd expect easy promotion to clone on a separate mount point and ensuring permissions are usable on the destination without jumping through hoops as above.

Yeah, I'm an entitled **** aren't I? :D