Bidirectional Sync of two FreeNASes

[sunrunner20]

I've got a problem where I have a Family Master FreeNAS at site A that replicates to Personal FreeNAS B (soon to be at Site B). Currently Sync is a A -> B ZFS Replications (Send/Recv).

However, as soon as B is at site B it will become my Main FreeNAS, which presents a problem- Senc/Recv does not allow sending changes back. So, I am going to need a method to send those changes back over a fairly slow link (7mbit - OpenVPN overhead).

I've got several requirements:

• "modified at A and modified at B" won't come up often, but I would like this to be left up to me to decide.
• Cannot just create a full copy of the data and then rsync changes around (Although a clone from the latest snapshot sent over and managed with a cron job would be acceptable, and the likely beginning of a solution)
• Would prefer it to handle interruptions smoothly as this isn't the fastest or stablest connection.
• Can't consume a ton of memory (local box only has 16gig), I have about 10TB of data.

Currently I've found a utility called Unison (http://www.cis.upenn.edu/~bcpierce/unison/docs.html), which looks promising. I haven't thought it completely through yet but Unison could keep the changes replicated over to master and from master, on a clone that I regularly destroy and then recreate once Unison indicates everything is in sync. The clone would be created from the a recent replicated snapshot.

Does anybody see any pitfalls here I should plan for, or risks I can't mitigate and just have to accept?

 

[Ericloewe]

The canonical solution would be rsync.

The inevitable crap performance can be mitigated by minimizing the amount of data that needs rsyncing. In other words, separate stuff into three overarching datasets:

• stuff_edited_by_A - Replicated from A to B, read-only on B
• stuff_edited_by_B - Replicated from B to A, read-only on A
• other_stuff - rsynced

Depending on the specific needs, this strategy can yield the significant advantages of replication for the bulk of the data, only using rsync for the stuff that needs it.

 

[Evi Vanoost]

ZFS sync is one-directional because it is a block-level transfer, not a file-level transfer. What you're looking at is probably something like Gluster or DRBD, both of which can have a ZFS store as a backend.

Newer options like Apache Helix, Sheepdog also exist and have similar roles with various pros and cons.

 

[Gab]

Hello,

Unison is working perfectly for that scenario. I am using it regularly to have a bidirectional sync of some data between several client machines and a FreeNAS server . In the end it give the same behavior as Google Drive or Dropbox, but with full control over options and servers.

Basically unison works like the following:
- you set up a "profile" that says what are the root folders to synchronize + various options (paths to ignores, specific behaviors, etc..). Unison is able to sync over SSH between remote machines.
- The first time you sync the two root folders, unison will compute the differences and ask you what to do exactly to have both replicas to be the same (the best is either to start with two roots with same contents, or one root with the content and the other empty => you have to say to replicate only from the root with content to the empty root).
- Then, for the next syncs, it will compute the differences and proceed automatically with the changes (new files, deleted files, updated files) from both sides. In the end, both replicas will be merged, and will be EXACTLY the same.
- In case of conflict, Unison will gently ask user what to do = do you want to sync from source to target (left=>right), or from target to source (right=>left), or review the diff, etc... Conflict resolution can be automatically solved with a few rules set up in the profile, but this is tricky

This is a very complete program, quite stable, and I am using it in production daily. There is even a GTK based GUI if needed (though I don't use it). The only drawback is that the initial learning curve is not 100% easy, though with a couple of hours playing with it, it gets ok. Other minor drawback is that it is written in a not so well known programming language (OCaml), and so, compiling manually might be tricky. But hopefully, FreeNAS 11 provide it by default, so no issue (in the past, FreeNAS 9, I had to recompile it manually from sources...).

Give it a try, I am sure it can solve your problem. And don't hesitate to ask if you need help.