Cloud Sync RClone Generated Config Files

Xipher

Cadet
Joined
Jun 18, 2019
Messages
5
I've got a problem and I want to resolve it in a very specific way that I'm sure people will find... silly :)

FreeNAS doesn't seem to use the "--fast-list" option for the cloud sync configs it generates, so, every sync is costing me around $0.30 using B2 block storage (there's a /lot/ of files and directories in every sync) even if I don't change any files, that is "no good".

I really wish there was a way to just say have an "extra arguments" section in the GUI, but I know I won't get that based on prior discussions and bug reports I've read it seems?

I know I can just make a CRON job with my own arguments, but I'm weird, I don't want to do that.

I really have this thing where I want to fix things at the source of the problem to make it work like it should nice and clean, so:

How does FreeNAS generate the configuration files it passes to rclone when a cloud sync job is created and run, I want to change it.

Also the other reason for wanting this change is that jobs get added and removed quite a bit and have to be added and removed by mildly unskilled users, so in changing it at the means of generation I'm just making it easy and foolproof, I have a method to my madness.
 

Xipher

Cadet
Joined
Jun 18, 2019
Messages
5
Gotcha! I'd also love to be able to change the amount of checkers/transfers and some stuff like the --delete-before switch...

Where can I modify the means in which the configs are generated?
Figure I'll give this a bump and see if anyone can point me in the right direction on this...
 

Dreded

Explorer
Joined
Nov 12, 2013
Messages
65
my solution before 11.3 to the missing arguments was to simply make a cronjob that did the rclone process instead of the GUI... now that 11.3 is here and fast-sync is an option I am using the GUI again... but my cronjob was...

/usr/local/bin/rclone --config /root/WSrclone.conf -v --stats 15s --fast-list --transfers 16 sync /mnt/Basin/Storage/Corporate remote:Helios/Corporate

and inside that WSrclone.conf file....
root@Helios:~ # cat WSrclone.conf
Code:
[remote]
account = jhdkasgfdlKAGFDJ1GR1KJ
key = KHDLKASJGFLK23FKJHSAKFJHSAf32f83291847298dh2
type = b2
bucket = Helios


root@Helios:~ #
 

DATAstrm

Dabbler
Joined
Nov 24, 2021
Messages
14
Is there a way to find the configuration files for the rclone operation that the GUI is running? I have my cloud sync all set up and working from the GUI and it would be nice to be able to just copy (and modify) the already-working settings when setting up a cron job.

Alternatively, the above request is still relevant. It would be great if the GUI had a way to enter custom flags to the rclone job. Right now, I'd like to do the --hard-delete option for B2 Backblaze. I know the fast-list option was added, but adding more options is kind of like wackamole. Having a user-defined field for custom options would allow the GUI to do so much more.
 

dcampitelli

Cadet
Joined
Jan 14, 2022
Messages
5
I am setting up the GUI to upload to Google Cloud Storage Buckets. Unfortunately, the permissions on those buckets are set to Uniform bucket-level access. This causes an error when the default rclone job attempts to set ACL on the files that are being uploaded. The flag: --gcs-bucket-policy-only can be used to disable the ACL assignment, but I can't seem to find the config file for the GUI job.
 
Top