Tips to make rsync faster

[mvcad]

Hi All,
I've been trying to copy a 2TB dataset from a synology NAS to our new FreeNAS server and I've noticed very slow transfer speeds. If I copy a file from windows explorer from a folder in the synology to a folder in FreeNAS I get more than 700 Mbps
with rsync I am only getting about 50!.

My rsync command runs from an automatic task in synology that runs a shell file. the command looks like this:
/usr/syno/bin/rsync -avz --omit-dir-times --delete $SOURCE $USER@$SERVER:$TARGET >> $LOG 2>&1

Without -z parameter I am getting 100 Mbps. Also I tried using ssh transfer with -p but it didn't make much difference.

We bought a new FreeNAS mini XL whit these specs

The data volume is 4 off 4TB WD Red in RidZ2 configuration with mirrored Samsung SSD evos 256 GB as SLOG.
any tips to make it faster? I tried last weekend for 3 days and it did't finish.

Thanks in advance.

 

[sretalla]

I saw that what you're doing is actually rsync over SSH (so encrypting comms) (due to the colon in the command).

Doing this:
/usr/syno/bin/rsync -avz --omit-dir-times --delete $SOURCE $USER@$SERVER$TARGET >> $LOG 2>&1

may then switch to rsync native (no encrypt of traffic in transit), so I would expect an improvement as a result... how much remains to be seen.

 

[Johnny Fartpants]

https://github.com/jbd/msrsync

 

[wblock]

Is this a one-time copy, or something that will happen periodically?

The processor in the Synology might be the bottleneck, having to compress and encrypt all the data. But then, remember that rsync is not aimed at speed but to minimize network traffic. Can you copy the files with Windows, then use rsync to copy any attributes that Windows does not get?

https://github.com/jbd/msrsync

Says "The main limitation is it does not handle remote source or target directory, they must be locally accessible (local disk, nfs/cifs/other mountpoint)."
But that might be an idea: mount the FreeNAS directory on the Synology with whatever SMB client they have, then copy or rsync it as if it was local.

 

[mvcad]

Is this a one-time copy, or something that will happen periodically?

Thanks for the suggestions, It will be a one-time only

 

[mvcad]

I saw that what you're doing is actually rsync over SSH (so encrypting comms) (due to the colon in the command).

Doing this:
/usr/syno/bin/rsync -avz --omit-dir-times --delete $SOURCE $USER@$SERVER$TARGET >> $LOG 2>&1

may then switch to rsync native (no encrypt of traffic in transit), so I would expect an improvement as a result... how much remains to be seen.

I tried your suggestion but I am getting an error

Code:

ERROR: module is read only
rsync error: syntax or usage error (code 1) at main.c(1034) [Receiver=3.0.9]
rsync error: rsync service is no running (code 43) at io.c(687) [sender=3.0.9]

My script looks like this:

Code:

#!/bin/bash

USER="rsyncbackup"
SERVER="192.168.0.50"
SOURCE="/volume1/Contractors_shared/"
TARGET="/mnt/Vol8TB/test/backup/"
LOG="/volume1/homes/admin/backup.log"

/usr/syno/bin/rsync -av --omit-dir-times --delete $SOURCE $USER@$SERVER:$TARGET >> $LOG 2>&1

Am i doing something wrong?

 

[sretalla]

The error seems to indicate that the rsync service isn't running on the end where we changed the script.

Do you have it configured on the synology?

 

[sretalla]

It's also saying that the target is read only. You need to look at the rsync config on FreeNAS.

 

[mvcad]

well, if I put the colon back in it works fine.
Do I need a space or a different character between $SERVER and $TARGET?. WITHOUT THE COLON IS THIS:

Code:

/usr/syno/bin/rsync -av --omit-dir-times --delete $SOURCE $USER@$SERVER$TARGET >> $LOG 2>&1

 

[sretalla]

well, if I put the colon back in it works fine.
Do I need a space or a different character between $SERVER and $TARGET?. WITHOUT THE COLON IS THIS:

Code:

/usr/syno/bin/rsync -av --omit-dir-times --delete $SOURCE $USER@$SERVER$TARGET >> $LOG 2>&1

Putting the colon back returns to using ssh, so is encrypting traffic. If you work out the rsync config on both sides, you can use rsync natively and avoid the encryption.

 

[sretalla]

You begin the path with a slash, so that's already enough to separate server from the path.

 

[mvcad]

If you work out the rsync config on both sides, you can use rsync natively and avoid the encryption.

Thanks, I will have to learn how to do that, hope I can get it to work.