Rsync setup recommendations

[steve.long]

I'm going to be implementing rsync to push data offsite. I have a FreeNAS Mini XL 32TB onsite and an identical FreeNAS Mini XL offsite.

Besides the hostname, IP, network settings, etc., do the two NAS units need to be identical? Should I just have a clean slate for the pull NAS? Do I need to set up identical datasets?

I've seen a lot of advice on setting up rsync, but I've failed to find anything that tells me how to set up things before rsync.

Thanks!

 

[PhilipS]

Any reason not to use ZFS replication instead of rsync?

 

[steve.long]

I guess the only reason I was going with rsync is that I'm going to be backing up over the internet and I had seen that people were using rsync to do that. Perhaps ZFS replication would also work for this?

 

[PhilipS]

I use replication instead of rsync when possible. Just understand the following disadvantages:

The FreeNAS replication middleware/UI does not yet support ZFS resume, so when there is a network issue, it has to restart from the beginning of the incremental send. So if your snapshots have a large amount of data that changes and your WAN connection is prone to disruptions, this may not work well. If starting with a large amount of data, you would probably want to seed your target locally.

You need to trust the replication target (your data is just as it left the source). It can be encrypted in transit though.

 

[steve.long]

If starting with a large amount of data, you would probably want to seed your target locally.

I will definitely replicate locally and then backup incrementally from there.

You need to trust the replication target (your data is just as it left the source). It can be encrypted in transit though.

Can you explain what you mean here a bit more? By target, you mean the 'pull' FreeNAS, right? What reason are there for not trusting it?

 

[PhilipS]

Can you explain what you mean here a bit more? By target, you mean the 'pull' FreeNAS, right? What reason are there for not trusting it?

Yes, the pull FreeNAS. If you were replicating to a friends FreeNAS, they would be able to see all your data - if the 'pull' FreeNAS is in your control, then this is not an issue.

 

[steve.long]

I really like the idea of using ZFS replication because it sounds more efficient, but because I'm going to be using the WAN maybe rsync is a more trustworthy option. Do you have an opinion on this?

 

[PhilipS]

If your WAN connection is reliable to transfer your typical snapshot size without going down, then use replication.

 

[steve.long]

Yeah, I guess I'll need to figure out how big my snapshots are gonna be.

And how should I connect the two? My Push side is a business with a public IP address, but my Pull side will be a private address.

What I think I understand so far is that I should install OpenVPN to the Pull side and have the Push NAS open a VPN connection to the OpenVPN server. Then I can use whatever backup protocol I want. Does that sound right? Is there a way to SSH to a private address without installing a VPN?

 

[PhilipS]

Yeah, I guess I'll need to figure out how big my snapshots are gonna be.

And how should I connect the two? My Push side is a business with a public IP address, but my Pull side will be a private address.

What I think I understand so far is that I should install OpenVPN to the Pull side and have the Push NAS open a VPN connection to the OpenVPN server. Then I can use whatever backup protocol I want. Does that sound right? Is there a way to SSH to a private address without installing a VPN?

If the remote public facing IP doesn't change much, you could setup with a DDNS service and a port forward (I would setup the router on the private side to ignore SSH connection requests from IPs other than the IP the PUSH server is on and only allow key based authentication on SSHD)

An OpenVPN connection through something like a pfsense router would probably be safer. You would want the side with the private IP to be the client and have the server on the side with the public IP. You would have the benefit of be able to manage FreeNAS remotely this way as well.