rsync hangs - solution

calc

Cadet
Joined
Apr 14, 2021
Messages
3
I thought this information might be useful for people transferring in data for new installs. I didn't see any mention of this particular issue when searching the forum.

I have a newly installed TrueNAS SCALE 21.02 system.

I started to transfer several terabytes of data into the server via rsync/ssh but it was much slower than gigabit due to the added encryption. During that time I transferred several hundred gigabytes of data with no problems.

So I went ahead and setup NFS to try to transfer the data in via rsync on a NFS mount.

However every time I started the rsync it hung after only a few minutes.

When it hung a strace of rsync just shows its waiting on a unix socket.

# strace -fttTvyys 4096 -p 2935 strace: Process 2935 attached 02:21:39.098242 select(5, [], [4<UNIX:[48402->48401]>], [], {tv_sec=24, tv_usec=453068}) = 0 (Timeout) <24.477928> 02:22:03.576830 select(5, [], [4<UNIX:[48402->48401]>], [], {tv_sec=60, tv_usec=0}) = 0 (Timeout) <60.003174> 02:23:03.580308 select(5, [], [4<UNIX:[48402->48401]>], [], {tv_sec=60, tv_usec=0}) = 0 (Timeout) <60.041621>

Looking at the stack of rsync I see:

# cat /proc/2933/stack [<0>] poll_schedule_timeout.constprop.11+0x46/0x70 [<0>] do_select+0x60d/0x7a0 [<0>] core_sys_select+0x2b1/0x3f0 [<0>] kern_select+0xb7/0x100 [<0>] __x64_sys_select+0x24/0x30 [<0>] do_syscall_64+0x57/0x190 [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9

After digging around I found this is a known bug in rsync itself.

Despite the issue title in my case it wasn't actually hanging in the same spot each time.

"rsync -avv /src/ /dst/" hangs constantly at the same place #159

The last comment in the bug report has a number of options that can be used as a workaround if you aren't able to upgrade to the most recent rsync.
 
Top