rsync not working the same as manually entering it

[Cyprus125]

Hello all,

So I've been using a Dell R510 server with 60TB of raidZ2 storage. i also have have my old system (it's a Helios 4 SBC 4 bay storage device) and in it i have 3 14TB and 1 8TB drive.

2 of the 14TB drives are for media set one
1 14TB drive is media set two
1 8TB drive is media set three

For ease of explaining the problem.

In scale under data protection I have 3 rsync tasks, one for each media set. Sets one and three work each day with no problems. No updates the the Zpool on the server, it acts like rsync should.

but media set two every time I the task runs and it starts writing the hold thing of the media set in a .~tmp~ sub folder for each folder set. Filling the drive before anything is complete and it fails.

Nothing has changed since the last time, but ever time it runs it buggers up.

It's 10TB of data that I then have to wipe and spend 5 days re-writing the data. I've tried using the command manually via SSH, I've let the task run the first copy, but no matter how its copied to the drive when the task runs on its own time, it dumps a data then does not need to be written.

help needed :\

 

[whodat]

Easiest if you share each of the commands here so they can be reviewed. Are you running the commands in a script? If so, share the scripts.

In the meantime, perhaps this might help... I noticed yesterday when running rsync, that when using TAB to auto-complete the source directory , the trailing slash / on the target dir was not being read correctly. This resulted in a nested dir of the same name being written in the target. Example:

# cd /mnt/target/dir # rsync -avh /mnt/source/dir/ .

In the above example, the final / on /mnt/source/dir/ was being ignored when TAB to autocomplete was used, which resulted in:

/mnt/target/dir/dir/{files}

Instead, by manually entering the final / on /mnt/source/dir/ , the result was as expected:

/mnt/target/dir/{files}

Perhaps this is a bug in the zsh shell...

In your case, be careful of copying text and pasting text commands from non-plaintext sources e.g. websites, documents, etc. as exotic characters might be included in your copy/ paste buffer.

 

[Cyprus125]

so the thing that is messing up is part of the data protection from the software.

here is the command i used 2 times in the past to try and "correct" truenas's inital copy of data.

rsync -avhP /mnt/CrackMedia/Media/Media/TV/Old/ crack@192.168.1.33:/srv/dev-disk-by-uuid-17270beb-86b2-4127-a596-a86338f50d55/TV/

how do i see what the data protection part of the software is doing. cause right now i checked and i have the data protection doing the rsync and it has all the files being put in ./~tmp~ folders.

 

[whodat]

Can you screenshot your rsync tasks in the WebGUI?

 

[Cyprus125]

ok i've been looking into rsync, and the only thing i can find that is simular to what i'm experianceing is for some reason 1 of 3 rsync's is using the "-T" which copys all files into a temp folder before the final move. how do i get data protection of true nas not to do that?? how can i see the command truenas is trying to do daily? if i could see what the OS is trying to do maybe i can fix it :|

 

[Cyprus125]

 

[whodat]

Which of the screenshots is the one you're having issues with? The last one (Old)?

 

[Cyprus125]

yes the last one, in command promp can i see what the command and attributes it's doing for these. i'd like to run the command the way the OS is so i can see what is happening.

 

[whodat]

This post seems relevant:

rsync issues when building new server: ~tmp~ dirs & endless syncs

Hi forum. Long time listener, first time caller. I am setting up a brand new FreeNAS 11.1 box and migrating all data and jails from my ageing 9.3 install. I couldn't do zfs snapshot transfer (apparently I needed 9.10 or later on the older server, I'm not sure why) so I switched on rsyncd on the...

www.truenas.com