Replication between servers taking a long time

[DaveJustShowedUp]

I have 2 Scale servers. The first I setup on an old PC as a test for a Plex server. It worked great so I acquired a newer box. Both are TrueNAS-SCALE-22.12.4.2.
Old server: HP Envy desktop, Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz, 16GiB memory, 3x 1TiB HDD striped.
New server: Msi MOBO, Intel i5-11400, 16 GiB, 2x 10TiB WD Red Plus HDD mirrored.

I want to copy all my media files from old to new server, so I set up a snapshot on my old server and a replication job on my new server.
The media, about 1.7Tib, on the old server is in /mnt/tank/Media and I want to copy it to /mnt/hdd-pool/plex-media, which has 2x 10Tib drives mirrored.

The snapshot finished on the old server in about 26 minutes yesterday afternoon. I started the replication job shortly thereafter, about 20 hours ago, and it is still running, about 40% done according to the job log. The job progress says this: Sending 1 of 2: tank/Media@auto-2023-11-30_07-00 (762.75 GiB / 951.81 GiB) [total 817.13 GiB of 1020.4 GiB]

Storage status on my new server says that 820 GiB are used, but the destination directories, /mnt/hdd-pool/plex-media, are empty.

Two questions, is this normal for a replication job? I have a run-of-the-mill home network. And if the files are being copied, where have they landed? If I have completely screwed this up and should copy files a different way, I am totally fine with burning my new server to the ground and starting over.

 

[winnielinnie]

The snapshot finished on the old server in about 26 minutes yesterday afternoon.

The snapshot finished in 26 minutes? Taking a snapshot is instantaneous. A second or two at most. What exactly took 26 minutes?

Storage status on my new server says that 820 GiB are used, but the destination directories, /mnt/hdd-pool/plex-media, are empty.

You'll have to wait until the replication is finished. Unlike an "rsync" or "cp" job, this doesn't simply copy "files" from one folder to another. It's all ZFS block-based.


Back of the napkin calculation clocks your replication at around 10 - 20 MiB/s.

Is any virtualization involved? What are your network cards?

 

[DaveJustShowedUp]

"Finished in 26 minutes" is what the message on the source server data protection page said yesterday afternoon. I don't have a screen shot to share. No VMs here.
My network card on the source server:
RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
vendor: Realtek Semiconductor Co., Ltd.

The destination server:
product: Ethernet Connection (10) I219-V
vendor: Intel Corporation

But I was getting 56MiB/s (according to winscp) when copying media files from my win10 desktop to the source server.

I still don't see any media files in what I thought was my destination directory, /mnt/hdd-pool/plex-media. Does a replication copy blocks then allow me to ls -l in the shell once it's finished? Thanks.

 

[DaveJustShowedUp]

Welp, the replication job just finished: 1.7 TiB in almost 27 hours. And I can see the media files.

 

[winnielinnie]

That's way too long. Something's wrong. That's about 17 MiB/s.

 

[DaveJustShowedUp]

I think so, too. Someone suggested that it's my router's fault. Home network, tp-link archer a7. Source server is in downstairs office, ethernet cable to router upstairs. Target server is in upstairs bedroom with ethernet cable to same router.

 

[ChrisRJ]

Yes, if you don't have a proper switch that is very likely to be an issue. The cheapest unmanaged 1 Gbps switch will probably make a big difference. BTW: About 15 years ago I had the exact same issue and was shocked by how poorly the switch portion of my Fritz!Box performed.

 

[winnielinnie]

How long is the run from NAS to NAS?

If feasible, you can purchase a lengthy CAT-6a cable and direct connect the two servers together (assuming they have an extra ethernet port/adapter available.)

You'll have to assign static IP addresses on their own subnets, which differ from your router's.

So for example (assuming your router is 192.168.1.1/24):

• Upstairs Server: 192.168.2.2/24
• Downstairs Server: 192.168.2.3/24

Then you'll have to change any references for the old IP addresses (replication tasks, rsync tasks, SSH keys, etc) to reflect the new IP addresses.

 

[ChrisRJ]

Well, given the price of a 5-port 1 Gbps switch (less than 20 Euros IIRC) I would not bother with the complexity of a direct connection.

But that is of course a personal decision and what @winnielinnie describes will work.

 

[DaveJustShowedUp]

Thanks, all. I'm not in a "production environment", just the family, so I'm probably going to move both servers upstairs to share a switch. Don't think I'm going to run replication that much for now, I'll just copy files manually as I merrily compute along. That is all, nothing to see here.

 

[winnielinnie]

I'll just copy files manually as I rip DVDs archive Linux ISOs.

They're watching. They're always watching. Just looking out for you, buddy...