Setting up 3 way sync between 3 sites.

[JasonW]

Hi All.

I am a newbie to FreeNAS and Linux so I am doing a lot of reading up on the differing terms being used that I don't recognise, all part of the fun of learning a new system.

What I have done is replace a PoS NetGear RN104 with an HP Microserver and built a FreeNAS box (upped the RAM to 8GB). This will be used only for archive so the traffic will be minimal but there will be data that will be required on occasions at any 1 of the 3 sites that I maintain. The Plan is to set up 3 identical systems, one at each site and in the evening have each of them synchronise all new data (if any) to the others. As a rule the only time new data will be added will be when a user leaves or when a project comes to completion. This data will then be archived for reference purposes only.

Can you point me in the right direction to setting this up? Also what are the pit fall of doing this? WAN performance shouldn't be an issue, the slowest connection is a 10Mb leased line.

Thanks for any advise!

 

[jgreco]

If you're merely adding files to an archival repository, rsync with appropriate flags is the ticket.

 

[danb35]

Between this and your more recent thread, it sounds like your goal is to have a set of data present on servers A, B, and C. Data could be added to any of those servers. At night, you want to sync the data so all the data (including anything added on A, B, or C) is present everywhere. Is that correct?

If that's your goal, rsync should be able to do it, but you'd need to set up separate jobs for each direction. That is, one task to sync A -> B, and a second task to sync B -> A. In total, I think you'd need four jobs to accomplish this:

• B -> A
• C -> A
• A -> B
• A -> C

There shouldn't be a need to sync directly between B and C.

 

[JasonW]

That's correct. It is unlikely that data will be added from B or C (if A is at my location) but the potential is there.
So I am currently updating the systems tot he latest stable release but this is what I have for a test Rsync task,both boxes are currently local.

Server A push to Server B
Path - /mnt/Archive/Archive
User - guest (All boxes currently have no authentication required to access and write to the archive folder)
Remote Host - 192.168.11.70
Rsync mode - Rsync module
Remote Module Name - 11.70
Direction - Push
It is set to push at 21:00 each night.
Recursive - Checked
Times - Checked
Compress - Checked
Archive - Unchecked
Delete - Unchecked
Quiet - Unchecked
Preserve permissions - unchecked
Preserve extended attributes - Unchecked
Delay Updates - Checked
Extra options - Clear
Enabled - Checked

Server B pull from Server A
Path - /mnt/Storage/Archive
User - guest (All boxes currently have no authentication required to access and write to the archive folder)
Remote Host - 192.168.11.25
Rsync mode - Rsync module
Remote Module Name - 11.70
Direction - Pull
It is set to pull at 21:00 each night.
Recursive - Checked
Times - Checked
Compress - Checked
Archive - Unchecked
Delete - Unchecked
Quiet - Unchecked
Preserve permissions - unchecked
Preserve extended attributes - Unchecked
Delay Updates - Checked
Extra options - Clear
Enabled - Checked

Although I have yet to let it run as per the schedule (doing that tonight), selecting run now on Server A doesn't seem to start the copy process. Will see what it does tonight.

 

[danb35]

I've not used the rsync module in FreeNAS, but these two jobs sound redundant--both would result in transferring data from A to B, but none from B to A.

 

[JasonW]

I may have taken the instruction too literal but i got "To configure rsync, you need to configure both ends of the connection:" from https://doc.freenas.org/9.3/freenas_tasks.html#rsync-tasks, not the case?

At the moment I am testing copying just a single part of the copy. Once I have the correct setup for one copy, the others should be simple.

 

[JasonW]

So I have been testing with the parameters set as above but I am unable to get Rsync to copy to data from the first to the second NAS, any thoughts where I am going wrong?

 

[JasonW]

Arrrrgh!!!!!! Stoopid user (me). Wrong name for module......... Now for phase 2.....

 

[JasonW]

Something I have noticed when reviewing the data to compare the syncs is that the size on disk is wildly more than the size of data, any thoughts on why this is so?

 

[cyberjock]

Something I have noticed when reviewing the data to compare the syncs is that the size on disk is wildly more than the size of data, any thoughts on why this is so?

Most likely it is due to compression being set differently between your machines.