11.3 replication errors

[dstewart51]

Hi all, been fighting this for over a week and come to believe this is a bug. I have a pair of 4ru iX certified systems, one production and one is a replication target. With 11.2x the snapshot replication worked great, but after updating to 11.3 and also 11.3u1 I have not been able to get the replication to work properly. I was able to get the SSH+NETCAT to generate the snapshots on the replication target, but each time the task runs it destroys all of the destination snapshots and generates them new. Same with just SSH. I also turned off "Replicate from scratch if incremental is not possible" after the initial job completed, but subsequent jobs just errors out with;

[2020/03/04 06:32:24] ERROR [replication_task__task_77] [zettarepl.replication.run] For task 'task_77' non-recoverable replication error NoIncrementalBaseReplicationError("No incremental base on dataset 'tank' and replication from scratch is not allowed")

If I leave "Replicate from scratch if incremental is not possible" enabled, after the initial job I see this for each subsequent job;

[2020/03/03 17:25:13] WARNING [replication_task__task_82] [zettarepl.replication.run] No incremental base for replication task 'task_82' on dataset 'tank/LAMarket', destroying all destination snapshots
[2020/03/03 17:25:13] INFO [replication_task__task_82] [zettarepl.snapshot.destroy] On )> for dataset 'tank/LAMarket' destroying snapshots ['auto-20200218.0800-2w', 'auto-20200219.0800-2w', 'auto-20200220.0800-2w', 'auto-20200221.0800-2w', 'auto-20200224.0800-2w', 'auto-20200225.0800-2w', 'auto-20200226.0800-2w', 'auto-20200227.0800-2w', 'auto-20200228.0800-2w', 'auto-20200302.0800-2w']

It appears it is not detecting or generating the incremental base and defaults to always starting from scratch. The full logs of the two jobs referenced above are attached.

 

[dstewart51]

Resorted back to the legacy replication tasks.

 

[Jedi940]

I have this same issue. I would like to know how to get my replication tasks upgraded properly. Luckily I started with one of the non critical and smaller data sets but I can not afford to re-replicate all my data. I don't have the bandwidth.
Here is the configuration of my new job:

The snapshot task selected is the same one that was used on the legacy system.

Here is a screenshot of the old task. All the other old tasks are still working.

Attached you fill find the log file showing the task deleting all the existing snapshots. Any help with this would be appreciated.

 

[reco]

same here,
had to recreate all replication tasks and snapshots ;(

Additionally look like the api has a default limit to only return 20 results. so the max number of replication tasks which will run are 20 :(

i posted a hack here. not sure if thats the right way to do fix it though, quite new to freenas.

autorepl.py limited to 20 tasks

looks like the new api has a limit of 20 results. since autorepl.py is using the api it would only run 20 replication tasks. is there a configuration somewhere to allow more than 20 replications tasks? thanks

www.ixsystems.com

christof

 

[panz]

Same problem here: just installed FreeNAS-11.3-U2.1 (I was on the 9.x series before).
I have two local pools: alderaan (the main pool, with all my Windows shares) and bakura (the backup pool, for replication).
My FreeNAS 9.x was working beautifully with *local* replication.

Now with FreeNAS-11.3-U2.1 I get this message:

"Replication "alderaan - bakura" failed: Target dataset 'bakura' does not have snapshots but has data (793848 bytes used) and replication from scratch is not allowed. Refusing to overwrite existing data."

BTW the bakura replication target pool was freshly created after FreeNAS-11.3-U2.1 install (so, it was not imported from v. 9.x); alderaan, instead, was imported.

Before starting replication I deleted all the old snapshots from alderaan.

 

[Heracles]

Hey guys,

I just did some upgrade myself and ended up with this.

Does it help for your cases ?