Cloud Sync Task - add extra rclone args to specify Azure Archive access tier?

[DouglasHeriot]

I want to configure a Cloud Sync Task to have rclone backup to Azure blob storage, using the "Archive" access tier.

rclone supports this using the command line args --azureblob-access-tier "Archive"
It can also be set using an rclone config file or environment variable.

Microsoft Azure Blob Storage

Rclone docs for Microsoft Azure Blob Storage

rclone.org

I cannot find any way in the FreeNAS Cloud Sync Task GUI to add this extra arg or config file option. When searching the forum, I found some other people with similar situations wanting to add rclone args, but ended up having to revert back to manually adding cron jobs instead. For example:

Google Drive sync cannot specify rood_folder_id

So, I am testing the cloud sync feature of FreeNAS and was greeted by an error error reading destination directory: couldn't list directory: googleapi: Error 403: The granted scopes do not give access to all of the requested spaces., insufficientScopes With some troubleshooting, I believe that...

www.ixsystems.com

Modifying a .py file in FreeNAS?

Hello, I'm pretty new to FreeNAS, so I wanted to check with the community about the recommended way to tweak behaviour not exposed in the UI. For background, I noticed that Cloud Sync to B2 was using up a whole lot of "Class C transactions", which cost money, and investigating that issue led me...

www.ixsystems.com

Rclone Options ('exclude' for example)

Is there a way to run Rclone (or 'Cloud Sync Tasks' as it is called in the GUI) with user-defined options, either through the GUI or by command-line editing of the Rclone config files? I've seen posts here where users ran Rclone direct from the command line inserting run-time options, but I'd...

www.ixsystems.com

Some of these have now been solved in the 11.3 release with many extra cloud sync options - but there's still no way to add any generic extra arguments.

While looking into the midlewared code I found that there is actually a way to add extra arguments - it just doesn't seem to be exposed in the UI.

Code:

args += shlex.split(cloud_sync["args"])

middleware/src/middlewared/middlewared/plugins/cloud_sync.py at 9ccc8c3ac19bd11f5226ad0ea22026c33e1017a3 · truenas/middleware

TrueNAS CORE/Enterprise/SCALE Middleware Git Repository - truenas/middleware

github.com

I worked out this means I can edit the "args" field of the freenas-v1.db sqlite database to add my arguments in. I performed this change, and it worked! New files are now created with the Archive access tier, almost 10 times cheaper.

Code:

$ sudo sqlite3 /data/freenas-v1.db
sqlite> update tasks_cloudsync set args = "--azureblob-access-tier Archive" where id = 1;

So, my questions are - is editing the database like this going to survive version upgrades, and be likely to continue working?

What's the best path to add a feature request for this - either for adding arbitrary arguments, or for specifically supporting the Azure access-tier option?

 

[DouglasHeriot]

Since doing this, I've discovered Azure has Lifecycle Management that can automatically transitions blobs to the Archive tier X days after last modification. That sounds like another reasonable solution until I can set the rclone parameter directly. https://stackoverflow.com/a/58852880/466698