Rsync doesn't progress past 2.8 TB

[artlessknave]

replication uses the settings in system, ssh keypairs/connections for authentication

 

[Thedriveguy]

Hmm, turns out my largest ZFS dataset that I RSync, (mostly video media, but music, photos & ebooks too), has only 17,230 files. It may be 2TB used, yet most of that is larger videos, like 1080p movies and some T.V. shows. The rest of the large files would be 480p videos.

Yeah I can see why it is a memory problem because I have around 40,000 files for about 22.3gb so I have around 717,488 files in just one dataset. Big Big Big Problem.

 

[artlessknave]

in theory, you might be able to split your rsyncs up as different dirs....but that sounds annoying as hell.

 

[Thedriveguy]

replication uses the settings in system, ssh keypairs/connections for authentication

I did set it up like that but for some reason I can set it up going from my remote server to the main, but I can't the other way around for some reason.

 

[Thedriveguy]

in theory, you might be able to split your rsyncs up as different dirs....but that sounds annoying as hell.

Yeah that sounds really annoying

 

[Thedriveguy]

I have been looking around other forum posts and it seems that the difference in versions of truenas include some bugs as said in this forum.

 

[artlessknave]

oh. wait. there was an authentication change between 12 and 13. something about security. if you were to upgrade both to 13, that might go away.

 

[Thedriveguy]

oh. wait. there was an authentication change between 12 and 13. something about security. if you were to upgrade both to 13, that might go away.

Ok ill try it

 

[Thedriveguy]

oh. wait. there was an authentication change between 12 and 13. something about security. if you were to upgrade both to 13, that might go away.

It worked I think it is backing up now!

Thanks everyone for the recommendations!