Two-Way sync with OneDrive

[pagratios]

Hello team, I have seen similar posts but I couldn't find the solution to those replies

I have a QNAP NAS device that has installed an application with the name Hybrid Backup Sync which have a two-way sync, wherever I am doing a change/update this app tries to update the other location with this update.

I am trying to create a custom NAS server using TrueNAS Scale and I am trying to figure out if there is something like this.

I have seen that TrueNAS Scale has the Cloud Sync Tasks that support either a PUSH or a PULL model. I have tried to add two tasks for each type but when I added one file locally and one to OneDrive whichever task was running first then deleted the other file. Do I miss something? Is there any option to make it work like Hybrid Backup Sync?

 

[Samuel Tai]

Cloud Sync is just Rclone under the covers. Rclone can do bidirectional sync, but you'll likely have to create cron jobs with CLI scripts to do what you want, instead of going through the UI. Note bidirectional sync is a Rclone BETA capability, and the version packaged with TrueNAS may not have it included.

 

[pagratios]

Thank you for the reply :).
I will test it

 

[pagratios]

Is there any documentation/guide to run Cron jobs scripts to truenas?

 

[Samuel Tai]

Is there any documentation/guide to run Cron jobs scripts to truenas?

Advanced Settings Screen

Provides information on the System Settings > Advanced screen, widgets, and configuration screen settings.

www.truenas.com

 

[pagratios]

Advanced Settings Screen

Provides information on the System Settings > Advanced screen, widgets, and configuration screen settings.

www.truenas.com

TrueNas Scale installed rclone version didn't have bisync option. Bisync is experimental but is part of stable version

I installed the rclone following this guide https://rclone.org/install/ and ran it though WebGUI "System Settings > Shell"
Then I added the command
rclone bisync /mnt/pool/dataset remote:/path/to/remote/folder --progress --check-access --transfers 10 --force --localtime --verbose
to cron task

 

[joeschmuck]

@pagratios

If you come up with a fully working solution, this would be a good topic to fully document, the step by step process for others to follow. I'm certain there are others who want this kind of feature. I use a windows based solution to sync by backups (SyncToy) that has been around for years. For home use it's fine, for office use I'd find another automated solution as you are doing.

Best of luck to you.

 

[NugentS]

I raised an issue on Jira to upgrade v1.56.1 to 1.6 or better (as appropriate for when / if IX do it). NAS-119779. It does however look very easy to do yourself (from a very non-expert)

I wonder what it would / will break though.

Interestingly there is no config file for rclone which sort of implies that IX isn't using rclone is a "standard" manner, thus implying that changing the executable, or creating a new one with a slightly different name may be not a 100% dumb idea

 

[pagratios]

@joeschmuck
I tried the below steps to make it work

1. Go to TrueNAS Scale "System Setting > Shell"
2. run "sudo su" using your password
3. Follow this guide https://rclone.org/install/ to install the latest rclone that supports bisync
4. After installation run "rclone config" and follow the tool (In my case with OneDrive I used windows bash with "rclone authorize" to get my token)
5. Add to Remote Foler an empty file with name RCLONE_TEST
6. run "rclone bisync /mnt/pool/dataset remote:/path/to/remote/folder --progress --check-access --transfers 10 --force --localtime --verbose --resync" --resync is used for the first time but also each time an error occurs (more info here https://rclone.org/bisync/)
7. Create a Cron Task to run as root with the command "rclone bisync /mnt/pool/dataset remote:/path/to/remote/folder --progress --check-access --transfers 10 --force --localtime --verbose"

I did several tests and it seems that works fine, I am not sure what will happen if I put my real data that has a lot of files.

 

[Samuel Tai]

@joeschmuck

Follow this guide https://rclone.org/install/ to install the latest rclone that supports bisync

I wonder if rclone selfupdate would've accomplished the same thing?

 

[joeschmuck]

Follow this guide https://rclone.org/install/ to install the latest rclone that supports bisync

Where did you install it to exactly? I ask because I'm concerned about when TrueNAS gets an upgrade, will it be affected?

 

[NugentS]

I wonder if rclone selfupdate would've accomplished the same thing?

Seems to work. Doesn't immediately break anything obvious. Updates to 1.61.1

 

[pagratios]

@Samuel Tai most probably, I wasn't aware of the selfupdate

@joeschmuck if you check the order you will see that I am to "System Settings > Shell" and I have run the "sudo su"

 

[joeschmuck]

@joeschmuck if you check the order you will see that I am to "System Settings > Shell" and I have run the "sudo su"

I saw that but I was hoping you would have installed it into a VM/Docker/Jail, something that would persist after a TrueNAS upgrade. I suspect it will be gone after a TrueNAS upgrade. So be aware of this after an upgrade as with your situation, you do not want to be accidentally deleting data. Make sure Upgrades are not automatic.

 

[pagratios]

I saw that but I was hoping you would have installed it into a VM/Docker/Jail, something that would persist after a TrueNAS upgrade. I suspect it will be gone after a TrueNAS upgrade. So be aware of this after an upgrade as with your situation, you do not want to be accidentally deleting data. Make sure Upgrades are not automatic.

You are right, I might have an issue after TrueNAS update (I would hope next update to have the updated version)
I think someone could do the same steps but instead of directly to shell to a VM/Jail/Docker

I think the bigger issue with the update would be the metadata the bisync has in order to maintain what are the changes

 

[joeschmuck]

(I would hope next update to have the updated version)

Isn't it in Beta right now? Even if it were released today, it likely would not make it into the next version of TrueNAS, but if you submit a ticket to have it updated, then it "might" happen.

 

[pagratios]

bisync is part of the stable release, altough rclone says that bisync is an Experimental feature

 

[NugentS]

@joeschmuck I thought about it and ended up setting up and running a daily cron job, 5 minutes before the cloud sync tasks start of "rclone selfupdate".
Whilst its early days - and I won't really have a good idea if this is a good idea till after the next update. Its not caused any issues yet (and I have done a test restore, following a backup on the assumption that cloud sync tasks uses the rclone executable that the commandline uses

 

[joeschmuck]

@joeschmuck I thought about it and ended up setting up and running a daily cron job, 5 minutes before the cloud sync tasks start of "rclone selfupdate".
Whilst its early days - and I won't really have a good idea if this is a good idea till after the next update. Its not caused any issues yet (and I have done a test restore, following a backup on the assumption that cloud sync tasks uses the rclone executable that the commandline uses

That is a smart idea! Is 5 minutes enough time to perform the update should one exist? I guess you will find out.

 

[NugentS]

Having run it manually - yes it very definately is. Run manually it takes almost no time to run the self update