Hello,
I have a transfer rate problem for backups via Rsync and SSH to my NAS and although I have read many posts about it and done many tests, I still don't understand the cause.
Here is the situation:
- Web server (WS): a virtual machine rented from Kontabo to host web services (with SSD)
- Backup server (BS) : a bare metal server rented from OVH / Kimsufi (with HDD) that I will cancel soon
- Home TrueNAS (HTN): my NAS hosted at home, full configuration is in my signature, storage usage <80%. I want to use it as my new backup server.
- Rsync in version 3.2.3 protocol version 31 on all machines
I have configured a daily backup of the Web server on the Backup server using Rsync via SSH. The backups are going well, with an average throughput between 1 and 2 MB/s and peaks up to 14 MB/s.
When I transfer files with the same method from the BS to my HTN, I get throughputs between 8 and 12 MB/s without problems.
But when I try to do the same backup from the WS to my HTN, the transfer starts around 4MB/s but after about 5 min, drops to 56 to 60 KB/s and never comes back. The transfer is way too slow to do daily backups (it might take 3 weeks to finalize the backup).
I reproduced all these tests with a single multi-gig file to eliminate the problems of multiple small files. I tested without compression or file verification.
At no time was there any CPU or RAM overload on any machine.
I don't know what to do anymore to identify the problem.
Here are some commands I used, not all of them are there because I tried different options without success:
Thank you for your help
I have a transfer rate problem for backups via Rsync and SSH to my NAS and although I have read many posts about it and done many tests, I still don't understand the cause.
Here is the situation:
- Web server (WS): a virtual machine rented from Kontabo to host web services (with SSD)
- Backup server (BS) : a bare metal server rented from OVH / Kimsufi (with HDD) that I will cancel soon
- Home TrueNAS (HTN): my NAS hosted at home, full configuration is in my signature, storage usage <80%. I want to use it as my new backup server.
- Rsync in version 3.2.3 protocol version 31 on all machines
I have configured a daily backup of the Web server on the Backup server using Rsync via SSH. The backups are going well, with an average throughput between 1 and 2 MB/s and peaks up to 14 MB/s.
When I transfer files with the same method from the BS to my HTN, I get throughputs between 8 and 12 MB/s without problems.
But when I try to do the same backup from the WS to my HTN, the transfer starts around 4MB/s but after about 5 min, drops to 56 to 60 KB/s and never comes back. The transfer is way too slow to do daily backups (it might take 3 weeks to finalize the backup).
I reproduced all these tests with a single multi-gig file to eliminate the problems of multiple small files. I tested without compression or file verification.
At no time was there any CPU or RAM overload on any machine.
I don't know what to do anymore to identify the problem.
Here are some commands I used, not all of them are there because I tried different options without success:
Code:
rsync -ve "ssh -i /path/to/key/file" -a --recursive --delete-after --ignore-errors /path/to/dir/ id@HTN.cdn:"/path/to/backup/pool/dir/"
Code:
rsync --no-i-r --info=progress2 -e "ssh -i /path/to/key/file" -a --recursive --delete-after --ignore-errors /path/to/dir/ id@HTN.cdn:"/path/to/backup/pool/dir/"
Code:
rsync -aW --no-i-r --info=progress2 --info=name0 -e "ssh -i /path/to/key/file" -a --recursive --delete-after --ignore-errors /path/to/dir/ id@HTN.cdn:"/path/to/backup/pool/dir/"
Thank you for your help
