zfs replication too slow? ~200m

[John Doe]

Hi I am having 2 FN systems connected in the LAN. I set up the replication task to have a proper backup.
So far it is working good, as I can see it. But it seems to be very slow. Networking graph shows me avg. 184M

Push Freenas 11.1-U5; 64 gb ram; Raid z2 8x 3tb; 8 core @ 3ghz
Pull Freenas 11.1-U6; 16 gb ram; stripe 3x 4tb; 4 core @ 2.3 ghz

CPU performance monitor on the pull system shows me ~50%

while the replication is ongoing I changed cipher to fast, but it seems to have no effect. is there a way to boost this up, without messing up the existing progress? Already did 13% in the past night

 

[kdragon75]

We need to identify the bottleneck. Please provide DETAILED hardware list with EXACT models, All network details and hardware, pool configuration on both ends and anything else that comes to mind.

 

[John Doe]

Push
Xeon E5-1620v4 3,5GHz 2011-3
Supermicro X10SRA-F
4x 16GB Samsung DDR4-2133 reg. ECC Ram
8x 3tb WD Red Raid Z2

pull:
Celeron J1900
Q1900B-ITX
2x 8gb RAM
3x 4tb WD green Stripe

switch:
linksys 1200AC v2

i do not think it is hardware related, as I can usual copy ~80mb/s (pull server) and 120mb/s push server via smb. I rather think it is something related to software, means settings

 

[GEOptic]

Gentlemen,

I have similar configuration, except replication uses a direct link (no switch).

I tried BBCP as discussed here :​

https://forums.freenas.org/index.ph...ove-freenas-11-replication.61207/#post-492939

But i'm still trying to figure how to solve the replication speed issue…

 

[John Doe]

havent found a solution, took me 4 days for 6tb...

 

[kdragon75]

For a one time zfs copy, I use netcat instead of ssh. It's not "secure" but this is all on my storage network with no external access. Search zfs send over netcat. There are several great articles on it.