Two-Way sync with OneDrive

pagratios

Cadet
Joined
Feb 5, 2023
Messages
8
Hello team, I have seen similar posts but I couldn't find the solution to those replies

I have a QNAP NAS device that has installed an application with the name Hybrid Backup Sync which have a two-way sync, wherever I am doing a change/update this app tries to update the other location with this update.

I am trying to create a custom NAS server using TrueNAS Scale and I am trying to figure out if there is something like this.

I have seen that TrueNAS Scale has the Cloud Sync Tasks that support either a PUSH or a PULL model. I have tried to add two tasks for each type but when I added one file locally and one to OneDrive whichever task was running first then deleted the other file. Do I miss something? Is there any option to make it work like Hybrid Backup Sync?
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
Cloud Sync is just Rclone under the covers. Rclone can do bidirectional sync, but you'll likely have to create cron jobs with CLI scripts to do what you want, instead of going through the UI. Note bidirectional sync is a Rclone BETA capability, and the version packaged with TrueNAS may not have it included.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398

pagratios

Cadet
Joined
Feb 5, 2023
Messages
8

TrueNas Scale installed rclone version didn't have bisync option. Bisync is experimental but is part of stable version

I installed the rclone following this guide https://rclone.org/install/ and ran it though WebGUI "System Settings > Shell"
Then I added the command
rclone bisync /mnt/pool/dataset remote:/path/to/remote/folder --progress --check-access --transfers 10 --force --localtime --verbose
to cron task
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
@pagratios

If you come up with a fully working solution, this would be a good topic to fully document, the step by step process for others to follow. I'm certain there are others who want this kind of feature. I use a windows based solution to sync by backups (SyncToy) that has been around for years. For home use it's fine, for office use I'd find another automated solution as you are doing.

Best of luck to you.
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,945
I raised an issue on Jira to upgrade v1.56.1 to 1.6 or better (as appropriate for when / if IX do it). NAS-119779. It does however look very easy to do yourself (from a very non-expert)

I wonder what it would / will break though.

Interestingly there is no config file for rclone which sort of implies that IX isn't using rclone is a "standard" manner, thus implying that changing the executable, or creating a new one with a slightly different name may be not a 100% dumb idea
 
Last edited:

pagratios

Cadet
Joined
Feb 5, 2023
Messages
8
@joeschmuck
I tried the below steps to make it work

  1. Go to TrueNAS Scale "System Setting > Shell"
  2. run "sudo su" using your password
  3. Follow this guide https://rclone.org/install/ to install the latest rclone that supports bisync
  4. After installation run "rclone config" and follow the tool (In my case with OneDrive I used windows bash with "rclone authorize" to get my token)
  5. Add to Remote Foler an empty file with name RCLONE_TEST
  6. run "rclone bisync /mnt/pool/dataset remote:/path/to/remote/folder --progress --check-access --transfers 10 --force --localtime --verbose --resync" --resync is used for the first time but also each time an error occurs (more info here https://rclone.org/bisync/)
  7. Create a Cron Task to run as root with the command "rclone bisync /mnt/pool/dataset remote:/path/to/remote/folder --progress --check-access --transfers 10 --force --localtime --verbose"

I did several tests and it seems that works fine, I am not sure what will happen if I put my real data that has a lot of files.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,945
I wonder if rclone selfupdate would've accomplished the same thing?
Seems to work. Doesn't immediately break anything obvious. Updates to 1.61.1
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
@joeschmuck if you check the order you will see that I am to "System Settings > Shell" and I have run the "sudo su"
I saw that but I was hoping you would have installed it into a VM/Docker/Jail, something that would persist after a TrueNAS upgrade. I suspect it will be gone after a TrueNAS upgrade. So be aware of this after an upgrade as with your situation, you do not want to be accidentally deleting data. Make sure Upgrades are not automatic.
 

pagratios

Cadet
Joined
Feb 5, 2023
Messages
8
I saw that but I was hoping you would have installed it into a VM/Docker/Jail, something that would persist after a TrueNAS upgrade. I suspect it will be gone after a TrueNAS upgrade. So be aware of this after an upgrade as with your situation, you do not want to be accidentally deleting data. Make sure Upgrades are not automatic.
You are right, I might have an issue after TrueNAS update (I would hope next update to have the updated version)
I think someone could do the same steps but instead of directly to shell to a VM/Jail/Docker

I think the bigger issue with the update would be the metadata the bisync has in order to maintain what are the changes
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
(I would hope next update to have the updated version)
Isn't it in Beta right now? Even if it were released today, it likely would not make it into the next version of TrueNAS, but if you submit a ticket to have it updated, then it "might" happen.
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,945
@joeschmuck I thought about it and ended up setting up and running a daily cron job, 5 minutes before the cloud sync tasks start of "rclone selfupdate".
Whilst its early days - and I won't really have a good idea if this is a good idea till after the next update. Its not caused any issues yet (and I have done a test restore, following a backup on the assumption that cloud sync tasks uses the rclone executable that the commandline uses
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,970
@joeschmuck I thought about it and ended up setting up and running a daily cron job, 5 minutes before the cloud sync tasks start of "rclone selfupdate".
Whilst its early days - and I won't really have a good idea if this is a good idea till after the next update. Its not caused any issues yet (and I have done a test restore, following a backup on the assumption that cloud sync tasks uses the rclone executable that the commandline uses
That is a smart idea! Is 5 minutes enough time to perform the update should one exist? I guess you will find out.
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,945
Having run it manually - yes it very definately is. Run manually it takes almost no time to run the self update
 
Top