Backup / Sync from QNAP to TrueNAS Scale

[calender7]

I am trying to sync data on a QNAP NAS to my Truenas scale NAS. I am using HBS (Hybrid Backup Sync) on the QNAP NAS. I get an error stating :Failed to complete sync job: "One-way Sync 1". Encountered error "rsync: [generator] recv_generator: mkdir "/2022 xxxx.sbd" (In remote rsync TrueNAS) failed. Permission denied (13)".

So, to me this means that I need to figure out how to allow HBS to mkdir on the TrueNAS machine. I'm at a loss on how to do that.

Self built NAS:

AMD Ryzen 5 5600G with Radeon Graphics
Cores: 6 cores (12 threads)
Corsair liquid cooling

Corsair 32 GB memory

1 x RAIDZ1 | 4 wide | 10.91 TiB

Thanks
Rich

 

[PhilD13]

I don't want to hijack the thread, but feel since no suggestions have been posted that maybe I should say something. I actually was working on some tips for you to try as I saw you were having trouble connecting the boxes together I really thought I had things worked out but as you see below that is all moot now as it simply does not work at all.

I will mention to you if you like to try is I found out If the QTS version is less than 5.xx (is at 4.xx) you will have issues with adding the public key to the QNAP user so Truenas can connect to the QNAP box from the Truenas rsync setup. The QTS5.xx+ has a place in the admin user to add the key in the GUI so if you run QTS 5+ you can add the key and they will connect. Some say in QTS4 you can add the public key to the admin user from the command line. You can connect a rsync job that is run from QNAP to Truenas (to push or pull data to Truenas. but in the QNAP Rsync connection setup to the Truenas box you have to use the Truenas box's ip address and not the hostname. QNAP does not seem to want to connect if the Truenas hostame is entered.

I'm trying to do essentially the same thing that you are; That is, transfer files from QNAP to Truenas Scale and from Truenas Scale to a different QNAP box so I can eventually retire the first QNAP box. No matter what I have tried, the process is just completely 100% broken. I thought today I had successfully transferred a couple of terabytes of data from QNAP to Truenas Scale without error according to the QNAP box. There were also no complaints from the Truenas box. All the directory structure is there on the truenas box but not one file in most directories or sub directories. There are some files in some directories but it seems more random now that I look at it. The Qnap box logs say successful - transfer completed and it certainly took long enough and acted like it transferred the files. Truenas server now shows 0.2% used instead of 26% used. I tested the Truenas server hardware for a week before I started using it and the server hardware runs just fine.

I am still working to figure this out and I would really like to give Truenas scale an honest serious try but I think I am going to have to very seriously rethink my idea of replacing the oldest QNAP (it's been EOL for years) box with a server running Truenas. I can't trust any setup that is apparently completely incompatible with each other and that might at anytime lose of fail to copy the data properly but report all okay. For now I may resort to trying an external laptop running SyncBack Pro backup program, just to verify that under actual use I can get files to and from Truenas Scale without issue and I didn't overlook anything hardware wise. The other alternative if I can't solve the issue is to pull the drives, sell all the hardware, and buy a new empty QNAP server.

Supermicro 6028R-E1CR16T 2U Server
W/ X10DRH-CT motherboard
128 GB DDR4 Registered memory
Intel Xeon processor E5-2640 v4 x 2
BPN-SAS3-826EL1-N4 backplane
Connected to LSI clone IT mode PCIe card
Pool is: 2 x RAIDZ2 | 8 wide | 7.28 TiB (16 8TB drives in 2 vdevs)
TrueNAS-SCALE-22.12.3.3 installed on 800gb a ssd mirrored boot.
SMB, smart, SSH, UPS services running.

 

[calender7]

Thank you for the input. You certainly did not hijack the thread. Lots of good information in your reply.
I’m actually thinking along lines of permissions. There is no error with the connection. The error log suggests that TrueNas will not allow the QNAP app Hybrid3 to write directories to the TrueNas drive. To that end I’m not sure what I have to do to Hybrid3 to mkdir on TrueNas. So, in the meantime, until I figure the permission issue out I’m in the process of making the directory structure on TrueNas mirror the directory structure on the QNAP NAS manually.

 

[PhilD13]

I did not have an issue with the Hybrid3 setup when I tried it. When I setup the sync job I had to setup/create the Rsync storage server in the Storage Space tab. I had to use the TN Scale server ip address (not hostname) and I had to add the user and password from the TN Scale server and I had to select Rsync-compatible server for server type as the others would not work, and check "use encryption port": and make sure it was set to 22 (default port) You can test the server connection in QNAP during setup from the storage server setup screen so if it does not connect you need to fix the permissions of the user on the TN end.

Can you use Putty and SSH logging in as the intended user for backups to the TN Scale server and navigate to the desired directories and perform any operations there? Can you do a simple rsync from the command line?

I did finally get something to work from the TN Scale Rsync tasks, but I think it is rather delicate, really finicky, and don't yet trust it. If the 3 tasks all goe a month or so doing nightly backups without any issues to the QNAP then maybe I will trust it.

I also think that the rsync on QNAP and TN Scale are not fully compatible and have found reference from QNAP that their older systems are not directly compatible with their new Hero (ZFS) systems as far as file transfers/copying. Also found a reference from TN documentation that suggested the use of RoboCopy on a windows computer to transfer files to and from QNAP servers.

If pulling something from QNAP, uncheck all the options in the TN rsync task (including archive) or it will fail. If pushing from TN, then I have found checking archive will work. Or at least right now it works.

 

[andyjay777]

Rsync from QNAP to Truenas worked OK for me. Circa 30TB transferred.

Rsync from QNAP to TN -> Only copying folders

Summary of the issue I have built a TrueNas SCALE sever to replace my QNAP. I have set up an Rsync task on the QNAP to send data to TN. It shows that data is copying on QNAP, but only the folders are copied. On the TrueNas side, the network usage is showing 100 MB/s, but the storage pool...

www.truenas.com