FreeNAS to FreeNAS Copy

[bent98]

I just built a new FreeNAS server and I want to copy all the files off that old one and put them on the new one.
I configured Rsync between the two servers and it looks like its moving 1GB in 3 mins which means if I was to transfer the whole 13TB it would take me over 11 days!!! Plus there is no status bar, I went in through windows to see what dir and file its copying to understand progress and speed.

Is there something faster? I usually use robocopy on my pc to back up to FreeNAS or network shares. This is different as I am going from NAS to NAS . I am backing large individual files that range between 4-30 gigs each. I need to make sure there's a crc check to ensure file integrate when copying. Please advise if there's a faster way.

 

[Thomas102]

Hi,
It is probably encryption that limit rsync speed.
You can try to setup replication between your old and new FreeNAS.

 

[diedrichg]

Replication: zfs send/receive is what you want. http://doc.freenas.org/11/storage.html#replication-tasks

 

[bent98]

Hi,
It is probably encryption that limit rsync speed.
You can try to setup replication between your old and new FreeNAS.

I am not using SSL do to tranfer but rather Rsync module.

 

[Thomas102]

I am not using SSL do to tranfer but rather Rsync module.

Not sure what you mean with Rsync module. Rsync connect through ssh. so SSL is used.

 

[danb35]

Replication: zfs send/receive is what you want. http://doc.freenas.org/11/storage.html#replication-tasks

Yep, pretty much this. It should run pretty close to wire speed (even though it runs over ssh as well).

 

[Thomas102]

And there is an option to disable encryption for replication.

 

[danb35]

And there is an option to disable encryption for replication.

True--it still goes through SSH, but you can disable crypto if desired. But I strongly doubt the encryption is the bottleneck here.

 

[bent98]

Am I following Section 8.31 or 8.32? Do I need to create a snapshot first.

BTW
I just tried doing a copy between two CIFS share in Win 10. I am getting 99mb per sec. What about using robocopy?

 

[bent98]

WTF
Robocopy is copying 40GB in 18 mins
Window 10 via explorer between shares 7.5 mins

 

[danb35]

Do I need to create a snapshot first.

Yes, replication (as implemented through the web GUI) requires that you have a recurring snapshot task set up. And even manual replication requires that you create a snapshot first, as it's the snapshot that gets replicated.

 

[bent98]

I wound up just using windows to copy between shares. It average around 96MB the whole time. Thanks for everyone help. Now its time to run a SMART long test.

 

[diedrichg]

I wound up just using windows to copy between shares.

I would now use some sort of synchronization program that compares your files. I would want to make sure that everything is 1:1.

 

[Chris Moore]

If you need to do this in the future, or if someone else finds this later, you can use scp (secure copy) from the command line to send files from one FreeNAS to another at near line speed. Here is a reference on how to use it: http://www.hypexr.org/linux_scp_help.php
The reference is for Linux, but it is almost identically implemented in BSD.
I used it when I setup the mirror between my two NAS systems and I have rsync running hourly to keep everything up to date.

 

[Stux]

You can use sftp too.