Hi,
I'm using TrueNAS 12.0 U5.1 and I'm trying to perform a cloud sync to a 1TB OneDrive on my Office 365 Business account. I'm not a business - just a home user.
My specs:
I have 2 x 2TB drives in a RAID 1 shared as "share1" (SMB)
I have 2 x 6TB drives in a RAID 1 shared as "share2" (SMB)
At some point, when I have the money, I'll replace the 2 x 2TB with 2 x 6TB and setup RAID-10.
It's running on a SuperMicro motherboard with a Core i3-9100 @ 3.6Ghz and 16GB ECC RAM.
It is mainly used as a home SMB server plus Plex in a jail and it runs perfectly. It has been running for just over a year and previously I used FreeNAS 9.x.
Tons of free RAM. Services using 5.5GB RAM. ZFS cache is 9.5GB (I'm reading this from the dashboard).
I have some success with cloud sync (some folders are syncing OK) but a number of problems:
Problem 1: I created a cloud credential via the GUI by selecting the "Microsoft OneDrive" provider and signing in to Office 365. In the "Drives list" drop-down only one drive appears "DOCUMENT_LIBRARY - xxxx" and it took a while to work out that this is my Office 365 SharePoint teams site "Documents" folder. My actual OneDrive doesn't appear in this drop-down list and I have no idea how to add it. Anyway, my SharePoint document folder has 1TB storage so I continued with this.... (but I'd like to know how to access my actual OneDrive).
So I now setup a "Cloud Sync Task" also via the TrueNAS web GUI and there are a load of options here but I picked:
Direction: Push (I want to backup from TrueNAS to Office 365 and TrueNAS triggers the sync via this task - a time schedule)
Transfer mode: Sync (If I add a file on TrueNAS I want it copied to Office 365, if I delete the file I want it removed from Office 365. I don't plan to change files at the Office 365 end - i.e. I'm using this as a backup for now).
Remote folder: /TrueNAS sync/XXXX (I created a folder called 'TrueNAS sync' directly in my Microsoft SharePoint teams site "Documents" folder.
Source: A folder on TrueNAS (XXXX)
I left every other setting alone.
So far so good, this works :) Folder XXXX on TrueNAS is synced to /TrueNAS sync/XXXX in Office 365 SharePoint and I've confirmed the files "made it".
But XXXX is just one folder of many on True NAS that I want to sync so I setup some additional Cloud Sync tasks to backup different folders at different times (so I now have about 5 Cloud Sync tasks in the TrueNAS GUI). This backs up about 10% of my NAS - just the folders that really matter and that I want to Sync (I only have 1TB storage in Office 365 vs 8TB in TrueNAS).
Problem 2: Most folders start syncing but eventually report a status of "Failed" (in the Cloud Sync tasks GUI). If I download the logs, I see lots of errors like:
2021/09/28 06:30:03 ERROR : On Top of the World.docx: corrupted on transfer: sizes differ 16409 vs 22112
2021/09/28 06:30:03 INFO : On Top of the World.docx: Removing failed copy
2021/09/28 06:30:03 ERROR : Among us SUS lyrics.docx: corrupted on transfer: sizes differ 23988 vs 32629
2021/09/28 06:30:03 INFO : Among us SUS lyrics.docx: Removing failed copy
2021/09/28 06:30:03 ERROR : Viola/Link to cripple creek viola for CMA tune.docx: corrupted on transfer: sizes differ 12225 vs 18650
2021/09/28 06:30:03 INFO : Viola/Link to cripple creek viola for CMA tune.docx: Removing failed copy
I really have NO IDEA what to do about this.
Problem 3: I start getting emails from Microsoft (some automated thing) telling me that thousands of files are being deleted on my SharePoint site and that they are all going into the recycle folder. Now I presume this is because of the thousands of "Removing failed copy" entries in the above log. Trouble is, they aren't really being deleted - they go into the Office 365 recycle bin and still count towards my Office 365 storage (so I have to manually delete them from the Office 365 end or I run out of storage real quick). When I stop syncing with Office 365, I stop getting the emails from Microsoft.
This error is only happening for about 10% of the files (a guess). LOTS of files have synced OK and look OK at the Office 365 end.
I cannot see anything special about the 10% of files with a problem - they open from TrueNAS just fine and don't appear to be corrupted.
Because 10% of the files aren't synced, every night when my daily backups run, they sync for hours and hours (because every night they are continually trying to sync the remaining 10%).
So this boils down to 3 problems:
a) Why can I only see my Office 365 SharePoint document share from TrueNAS and cannot see my actual Office 365 OneDrives shares?
b) Why do I get these corrupted on transfer errors? This is the main problem.
c) Why do I keep getting these emails from Microsoft (my guess is because of b) removing the failed copy).
A few last notes:
I have a pretty decent broadband connection (for the UK). About 300Mb/s down and 50Mb/s up. Very stable and I reliably get these transfer speeds.
The NAS seems very stable. I have no other problems and no reports of any corruption from the users (5 people in my family).
So can anyone see what I've done wrong? Or have seen this problem before? Should I raise a bug?
Thanks
Simon
I'm using TrueNAS 12.0 U5.1 and I'm trying to perform a cloud sync to a 1TB OneDrive on my Office 365 Business account. I'm not a business - just a home user.
My specs:
I have 2 x 2TB drives in a RAID 1 shared as "share1" (SMB)
I have 2 x 6TB drives in a RAID 1 shared as "share2" (SMB)
At some point, when I have the money, I'll replace the 2 x 2TB with 2 x 6TB and setup RAID-10.
It's running on a SuperMicro motherboard with a Core i3-9100 @ 3.6Ghz and 16GB ECC RAM.
It is mainly used as a home SMB server plus Plex in a jail and it runs perfectly. It has been running for just over a year and previously I used FreeNAS 9.x.
Tons of free RAM. Services using 5.5GB RAM. ZFS cache is 9.5GB (I'm reading this from the dashboard).
I have some success with cloud sync (some folders are syncing OK) but a number of problems:
Problem 1: I created a cloud credential via the GUI by selecting the "Microsoft OneDrive" provider and signing in to Office 365. In the "Drives list" drop-down only one drive appears "DOCUMENT_LIBRARY - xxxx" and it took a while to work out that this is my Office 365 SharePoint teams site "Documents" folder. My actual OneDrive doesn't appear in this drop-down list and I have no idea how to add it. Anyway, my SharePoint document folder has 1TB storage so I continued with this.... (but I'd like to know how to access my actual OneDrive).
So I now setup a "Cloud Sync Task" also via the TrueNAS web GUI and there are a load of options here but I picked:
Direction: Push (I want to backup from TrueNAS to Office 365 and TrueNAS triggers the sync via this task - a time schedule)
Transfer mode: Sync (If I add a file on TrueNAS I want it copied to Office 365, if I delete the file I want it removed from Office 365. I don't plan to change files at the Office 365 end - i.e. I'm using this as a backup for now).
Remote folder: /TrueNAS sync/XXXX (I created a folder called 'TrueNAS sync' directly in my Microsoft SharePoint teams site "Documents" folder.
Source: A folder on TrueNAS (XXXX)
I left every other setting alone.
So far so good, this works :) Folder XXXX on TrueNAS is synced to /TrueNAS sync/XXXX in Office 365 SharePoint and I've confirmed the files "made it".
But XXXX is just one folder of many on True NAS that I want to sync so I setup some additional Cloud Sync tasks to backup different folders at different times (so I now have about 5 Cloud Sync tasks in the TrueNAS GUI). This backs up about 10% of my NAS - just the folders that really matter and that I want to Sync (I only have 1TB storage in Office 365 vs 8TB in TrueNAS).
Problem 2: Most folders start syncing but eventually report a status of "Failed" (in the Cloud Sync tasks GUI). If I download the logs, I see lots of errors like:
2021/09/28 06:30:03 ERROR : On Top of the World.docx: corrupted on transfer: sizes differ 16409 vs 22112
2021/09/28 06:30:03 INFO : On Top of the World.docx: Removing failed copy
2021/09/28 06:30:03 ERROR : Among us SUS lyrics.docx: corrupted on transfer: sizes differ 23988 vs 32629
2021/09/28 06:30:03 INFO : Among us SUS lyrics.docx: Removing failed copy
2021/09/28 06:30:03 ERROR : Viola/Link to cripple creek viola for CMA tune.docx: corrupted on transfer: sizes differ 12225 vs 18650
2021/09/28 06:30:03 INFO : Viola/Link to cripple creek viola for CMA tune.docx: Removing failed copy
I really have NO IDEA what to do about this.
Problem 3: I start getting emails from Microsoft (some automated thing) telling me that thousands of files are being deleted on my SharePoint site and that they are all going into the recycle folder. Now I presume this is because of the thousands of "Removing failed copy" entries in the above log. Trouble is, they aren't really being deleted - they go into the Office 365 recycle bin and still count towards my Office 365 storage (so I have to manually delete them from the Office 365 end or I run out of storage real quick). When I stop syncing with Office 365, I stop getting the emails from Microsoft.
This error is only happening for about 10% of the files (a guess). LOTS of files have synced OK and look OK at the Office 365 end.
I cannot see anything special about the 10% of files with a problem - they open from TrueNAS just fine and don't appear to be corrupted.
Because 10% of the files aren't synced, every night when my daily backups run, they sync for hours and hours (because every night they are continually trying to sync the remaining 10%).
So this boils down to 3 problems:
a) Why can I only see my Office 365 SharePoint document share from TrueNAS and cannot see my actual Office 365 OneDrives shares?
b) Why do I get these corrupted on transfer errors? This is the main problem.
c) Why do I keep getting these emails from Microsoft (my guess is because of b) removing the failed copy).
A few last notes:
I have a pretty decent broadband connection (for the UK). About 300Mb/s down and 50Mb/s up. Very stable and I reliably get these transfer speeds.
The NAS seems very stable. I have no other problems and no reports of any corruption from the users (5 people in my family).
So can anyone see what I've done wrong? Or have seen this problem before? Should I raise a bug?
Thanks
Simon