Replication on unreliable internet

[MrNaz]

I would like to have a bunch of satellite sites replicate their TrueNAS boxes to a central site. Some of the satellite sites have significant datasets. Assume that all sites have big bandwidth pipes, and also that there is enough HDD space at all sites, including at the central site.

If I set up several sites to replicate datasets on local TrueNAS systems with the One Bit Daddy TrueNAS, I have the following questions:

1. If the internet connection goes down for a period, will the replication take up where it left off, including all intermediate snapshots?
2. If a site changes a lot of data at one time, and the internet bandwidth is not enough to finish sending the snapshot before the next one begins, are the replication jobs smart enough to run one after the other and not stomp on each other?
3. Am I able to have different snapshot retention policies at the satellite sites and the central site? I.e., can I retain more snapshots at the central site if I wish?

Thanks in advance!

 

[Basil Hendroff]

The answer is YES to all those questions. TrueNAS replication is a very mature, resilient and reliable service. I currently replicate across the internet to a remote site as well as locally across wireless and I don't make any assumptions about the availability of either transport method. A couple of suggestions though:

1. If you can, I strongly recommend you bring each satellite box back one at a time and do your initial replication locally. This will ensure your data is synced in the shortest possible time. It also limits bandwidth stress by minimising excessive data transfer across the internet.
2. Replicate across encrypted links e.g. TLS, VPN to the satellite sites.

 

[ChrisRJ]

The underlying mechanism for cloud sync tasks is rclone (https://rclone.org), if you are interested in more details.

 

[sretalla]

The underlying mechanism for cloud sync tasks is rclone

Which isn't necessarily what's being discussed here... I think we're talking about Replication Tasks (which use zettarepl in the background).

 

[ChrisRJ]

@sretalla , thanks. I had indeed made the conclusion that Internet means cloud sync tasks.