Google Drive sync failed

[giuseppe30]

Hello,
from 1 week rclone google drive sync failed with some google documents sync with this error:
2021/07/13 11:39:07 ERROR : DOCUMENTI/xxxx: Failed to copy: failed to open source object: open file failed: googleapi: Error 403: Only files with binary content can be downloaded. Use Export with Docs Editors files., fileNotDownloadable

any idea?
Thanks

 

[giuseppe30]

ok so there is an error with rclone and new googleAPI https://github.com/rclone/rclone/issues/5455
I can try to update rclone v 1.56 beta version but the version is the same 1.53.1

Code:

root@lmv-nas-storage[~]# curl https://rclone.org/install.sh | bash -s beta
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4491  100  4491    0     0  19960      0 --:--:-- --:--:-- --:--:-- 19960
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    35  100    35    0     0     92      0 --:--:-- --:--:-- --:--:--    92
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.8M  100 13.8M    0     0  4719k      0  0:00:03  0:00:03 --:--:-- 4719k
Archive:  rclone-beta-latest-freebsd-amd64.zip
   creating: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/
 extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/README.html (text)
 extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/git-log.txt (text)
 extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/rclone
 extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/README.txt (text)
 extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/rclone.1 (text)

rclone v1.53.1-DEV has successfully installed.
Now run "rclone config" for setup. Check https://rclone.org/docs/ for more details.

 

[hysel]

ok so there is an error with rclone and new googleAPI https://github.com/rclone/rclone/issues/5455
I can try to update rclone v 1.56 beta version but the version is the same 1.53.1

Code:

root@lmv-nas-storage[~]# curl https://rclone.org/install.sh | bash -s beta
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4491  100  4491    0     0  19960      0 --:--:-- --:--:-- --:--:-- 19960
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    35  100    35    0     0     92      0 --:--:-- --:--:-- --:--:--    92
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.8M  100 13.8M    0     0  4719k      0  0:00:03  0:00:03 --:--:-- 4719k
Archive:  rclone-beta-latest-freebsd-amd64.zip
   creating: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/
extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/README.html (text)
extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/git-log.txt (text)
extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/rclone
extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/README.txt (text)
extracting: tmp_unzip_dir_for_rclone/rclone-v1.56.0-beta.5587.94b143929-freebsd-amd64/rclone.1 (text)

rclone v1.53.1-DEV has successfully installed.
Now run "rclone config" for setup. Check https://rclone.org/docs/ for more details.

Where you able to solve the issue with this method?

 

[giuseppe30]

Where you able to solve the issue with this method?

no, I was unable to install 1.56 beta version.

 

[elvisimprsntr]

Any reason you are not using the built in Cloud Sync Task to sync data to Google Drive?

 

[hysel]

Any reason you are not using the built in Cloud Sync Task to sync data to Google Drive?

It doesn't work there as well

 

[elvisimprsntr]

It doesn't work there as well

PUSH/SYNC works on my system.

 

[elorimer]

In my case it was one single file that was being choked on. I deleted it and the sync proceeded fine.

 

[hysel]

In my case it was one single file that was being choked on. I deleted it and the sync proceeded fine.

How did you know which file was it?

 

[elorimer]

How did you know which file was it?

From the sync log. It reported an i/o error for that file, and retried it 3 times before failing. Don't know why though--it was a very simple excel spreadsheet.

 

[giuseppe30]

the issues is caused by new google API with all Google Documents (google doc, sheet, etc).
TrueNas should update rclone to solve it.

 

[SafeRackAdmin]

I"m running into the same issues, only with S3 cloud syncs, and it's only java script files.

 

[xale]

Any updates on this? I've got several files that are giving me the same error, and deleting them isn't really an option; I'd prefer they were included in the sync. Will a fix be available soon?

 

[hysel]

Ok, I found the solution and it actually very easy :)

It is related to the version shipped with TrueNAS (1.53). This specific problem was solved in version 1.56

For reference, I am using TrueNAS-12.0-U5.1

1. Make sure you don't have any rclone instances running, you may need to restart your server
2. Open a shell window and update rclone to the latest version using the instruction at this page: https://rclone.org/downloads/
   1. In my machine, I used the latest BETA version
3. Once downloaded, edit the following file: /usr/local/lib/python3.9/site-packages/middlewared/plugins/cloud_sync.py
4. Located the following line and make this change:
   Before:
   
   Code:

   args = ["/usr/bin/local/rclone",
   "--config", config.config_path,
   "-v",
   "--stats", "1s",
   ]

   
   After:
   
   
5. Code:

   args = ["/usr/bin/rclone",
   "--config", config.config_path,
   "-v",
   "--stats", "1s",
   ]

6. Save and close the file
7. Restart the server
8. You are good to go

Hope this help you

Itamar

 

[giuseppe30]

Ok, I found the solution and it actually very easy :)

It is related to the version shipped with TrueNAS (1.53). This specific problem was solved in version 1.56

For reference, I am using TrueNAS-12.0-U5.1

1. Make sure you don't have any rclone instances running, you may need to restart your server
2. Open a shell window and update rclone to the latest version using the instruction at this page: https://rclone.org/downloads/
   1. In my machine, I used the latest BETA version
3. Once downloaded, edit the following file: /usr/local/lib/python3.9/site-packages/middlewared/plugins/cloud_sync.py
4. Located the following line and make this change:
   Before:
   
   Code:

   args = ["/usr/bin/local/rclone",
   "--config", config.config_path,
   "-v",
   "--stats", "1s",
   ]

   
   After:
   
   
5. Code:

   args = ["/usr/bin/rclone",
   "--config", config.config_path,
   "-v",
   "--stats", "1s",
   ]

6. Save and close the file
7. Restart the server
8. You are good to go

Hope this help you

Itamar

Hello @hysel
it's works on my machines with TrueNAS-12.0-U3

Thank you very much!