Replication failing b/c of new dataset?

[winnielinnie]

I understand the need to keep TrueNAS stable but I honestly just don't understand how we're supposed to trust zetarepl when it falls over in such basic ways.

There's other risky things about zettarepl that can bite you by surprise, simply from using the GU exclusively:

Staged snapshot schedule

Hi! How would I best go about creating a schedule that creates snapshots of a dataset like this: 48 hourly 31 daily 48 monthly 12 yearly I think that I should create a separate schedule for each "frequency", but is there a better way?

www.truenas.com

I knew the second they added "Almost" in the GUI to "Full Filesystem Replication" that trouble was a foot.

The tooltip was changed because of the previously mentioned bug report. Since they did not want to address the other underlying bug (i.e, "full" replication isn't really full with each subsequent incremental transfer, as it skips all intermediary snapshots), I pointed out how the description of the tooltip is misleading. (I was shocked when I realized my intermediary and manually-created snapshots were not being transferred to the backup pool!)

For these three reasons (and perhaps more unknown?), I feel very uneasy about zettarepl and its relevant GUI tasks in the TrueNAS web UI. It feels halfheartedly slapped together, and now it's just going to remain "as is". Even though I'm far from a developer, I had to resort to creating my own custom scripts that essentially work as expected (regardless of what child datasets are added or removed.)



A Replication Task should intelligently replicate what the user expects.

What you "select" on the source side should incrementally replicate to the destination side, including everything in between and all children if using "recursive" or "full filesystem", and if a new child is created later, then it too should be transferred (full or incremental, depending if it's the first time.) If a child is deleted later, then it should be ignored.

In fact, I like the concept behind "syncoid" whereas it creates a snapshot on-demand upon being invoked to replicate to a destination (manually or on a schedule). If this was allowed in a GUI, imagine being able to click a button to do a manual incremental backup? It would be in the same spirit as backing up to an external drive, with the power of ZFS. (Which is what I'm doing with my custom scripts.)

 

[Forssux]

This is one (of multiple) reasons why I use homemade scripts and manually-invoked Cron Tasks, rather than the GUI's built-in Replication Tasks (the Periodic Snapshots and Replication Tasks use zettarepl under-the-hood).

If "-I" (big i), rather than "-i" (little i) was used in their code, you could do what you wanted to in your opening post. However, because of this bug report, you'll see we're stuck with it indefinitely, especially for TrueNAS Core:

https://jira.ixsystems.com/browse/NAS-109476

I believe it wasn't really well designed from the start, but rather than re-design/re-code it from scratch, it's just going to stay this way. Plus, if there isn't a demand from business or corporate customers, it gets low (if any) priority.

Is there any chance that you can share these script. I'm just a home user but I think It could help me..

 

[winnielinnie]

@Forssux I'll private message you.

It's a very "rudimentary" script, and contains very few safeguards. (I wrote it for myself and my very particular replication needs.) I use it to manually invoke a backup. It contains no auto-pruning nor progress indicator. (Pruning is done through how Periodic Snapshots are configured.) I know it's "done" when I receive an email and/or the Task Manager shows the Cron Task is 100% complete. (It displays 10% until it finishes at 100%, since it's not aware of the replication stream's progress.)