Why does replication use the push paradigm instead of pull?

[mpfusion]

I am trying to understand why replication pushes to the remote server instead of having the replication server pull the data in. In every backup system I previously set up, the backup servers pulled the data from the main server.

The idea is that a compromised server should not be able to delete the offsite backups. In freenas a compromised server has SSH access to the offsite data and an attacker can easily wipe the data on the remote side. Technically it should be simple doing either way. So I wonder about the reasons for the current implementation.

A freenas server runs various different services which enlarges the attack vector. A backup server doesn't run any services, except SSH and is therefore less likely to get compromised.

Furthermore, if the server replicates to several backup servers, a compromised server can delete the data on all backup servers at the moment. Using the “pull paradigm” a compromised main server can't touch any offsite data and a compromised backup server can delete its own data and the data on the main server, but it can't access the data on other backup servers.

Can somebody shed some light on this? What am I missing here?

 

[cyberjock]

Well, there's some timing involved. You need to take a snapshot before any data is transferred. The replication task, if I'm not mistaken, doesn't start unless the server is near idle. The exception is if you get to the end of the schedule and it still hasn't run it'll eventually just run because it must.

Even if you flip things around the server initiating the connection must have the ability to authenticate to the other system. Period. So a compromised backup server could very easily wipe out both your primary and backup server. Nothing I see gained there from flipping it around.

Also if you do things properly and separate out the backup server into multiple datasets and use appropriate permissions I'm pretty sure you can isolate the backups from each other. I haven't tried to do this because I take the stance that if you can't keep your primary server secure you shouldn't have to worry about your backup server. Also there's no substitute for offline backups(which we all know we should be doing but 99% of us don't do).

The other thing is you have a single location to manage both snapshots AND replication. I can't tell you how many threads have revolved around getting this so wrong it doesn't work for them despite both settings being set on the same server. I don't even want to know how screwed up things would be if you had to coordinate a schedule on both servers. Yuk!

So from a technical standpoint it's better to do a push over a pull.

The other thing is, what are you wanting to gain from the pull? While you might argue the pull side and I might want to argue the push side, what's the big deal? Is there a very definitive gain for one over the other aside from the technical limitations.

Lastly, FreeNAS didn't setup the push over pull situation. That's how ZFS was designed. We're just using what someone else already did. Also keep in mind that you could *easily* write a script on your "pull" server to initiate transfers from your "push" server by simply SSHing into the "push" server and running the appropriate command. It would require work on your part, and you'd have to recognize that everything would be invisible to the WebGUI. But that's the breaks.

 

[mpfusion]

Well, there's some timing involved. You need to take a snapshot before any data is transferred.

This might actually be the reason it's implemented the way it is. Because replication is tied to snapshots which occue on the push side.

The replication task, if I'm not mistaken, doesn't start unless the server is near idle. The exception is if you get to the end of the schedule and it still hasn't run it'll eventually just run because it must.

I didn't know that.

Even if you flip things around the server initiating the connection must have the ability to authenticate to the other system. Period. So a compromised backup server could very easily wipe out both your primary and backup server. Nothing I see gained there from flipping it around.

As I tried to explain in my post, it's correct than one system has to have the ability to log into the other and thus, can wipe the data. But it's easier to secure a backup system because it's not running any services. Furthermore, as I explained, if you use two or more backup servers, any single one can be compromised without deleting all data. Currently a compromised main server may purge all replication servers' data.

Also if you do things properly and separate out the backup server into multiple datasets and use appropriate permissions I'm pretty sure you can isolate the backups from each other. I haven't tried to do this because I take the stance that if you can't keep your primary server secure you shouldn't have to worry about your backup server.

You might try to keep the server secure, but sometimes you just can't, even if you try. That's why it's important that offsite backups can't be purged from a compromised server.

The other thing is you have a single location to manage both snapshots AND replication.

That's true, and this might be one reason for the current implementation.

So from a technical standpoint it's better to do a push over a pull.

The implementation might be simpler, but security wise I disagree.

The other thing is, what are you wanting to gain from the pull?

Not loose all data if one server gets compromised.

While you might argue the pull side and I might want to argue the push side, what's the big deal? Is there a very definitive gain for one over the other aside from the technical limitations.

Yes, it is. See my explanations above.

Lastly, FreeNAS didn't setup the push over pull situation. That's how ZFS was designed.

ZFS has “zfs send” and “zfs receive” and the SSH connection can be initiated from either side. My point is not to swap send and receive, but to log into the main server from the replication server, instead of the other way round, like it's done now. The main server still uses “zfs send” and the replication server “zfs receive”s the data. It's just about which server authenticates to which one, not which actually pulls and which receives. Maybe the thread title is a bit misleading.

 

[solarisguy]

@mpfusion, replication and backup are two different goals, that can be attained using the same tools. Replication accurately copies data from one server to another, and is very useful when load balancing or protecting against a server loss. Backup copies data so it may be used to restore the original data after a loss due to software, hardware or human ;) malfunction.

That is why backups, as you had experienced, are often pull due to incremental backups. On the other hand, replication is almost always a push, as the server knows whether there is any new data to push.

You can mix replication with backups. For example you can be replicating all the day, and take your replica off-line at the end of the day. And rotate through 7 or 30 or 365 of such replicas.

As such, there is a plenty of room for a confusion, so please remember that replication and backup are not the same.