SOLVED Schedule consecutive RSYNC tasks?

[Avro]

I've managed to get rsync working to backup a portion of my FreeNAS server to a Synology 2-bay box. As I don't need (or have the room) to back up everything on my FreeNAS server, I have five rsync tasks scheduled to selectively go through different volumes and sync them to the Synology NAS.

I've set up schedules for each of the five tasks in the GUI, and when setting their start times simply took a wild guess as to how long each might run for on a given day:
task 1 - 08:00
task 2 - 08:30
task 3 - 09:30
task 4 - 10:00
task 5 - 11:00

I'd suspect that task 5 would have the most new data to copy, so have left it to the end.

Is there any way to scheduled these tasks to run consecutively, so that when the first one finishes the second one starts? If there's no new data, then all of the rsync work could be over with in a matter of minutes. If there's a lot of new data, then one task could run into another.

I did a Google search and one on here and couldn't find my answer.

Thanks,

Bill

Server specs:
FreeNAS 9.3
SuperMicro MBD-X9SCM
Intel Xeon E3-1230
24 GB ECC RAM
6 x WD Red 4TB HDDs, RAID-Z2

 

[dlavigne]

Did you find a solution for this?

 

[toadman]

Instead of using the GUI I suppose you could write a small script that executes the jobs in order and would run them one right after the other. You could use cron to kick it off at a specific time.

 

[Avro]

Instead of using the GUI I suppose you could write a small script that executes the jobs in order and would run them one right after the other. You could use cron to kick it off at a specific time.

Yeah, I was thinking the same thing, much like how I can stack batch files in Windows task scheduler to run one after another. I've created a user on my FreeNAS system specifically to run rsync tasks, and that's the user that I would run the cron job under. In testing the script however, I can't seem to get the command line operation to work. It was fine when I was setting the rsync stuff up last week, and the GUI works fine if I 'Run Now', but I must be doing something wrong. Specifically, I'm getting:

Code:

rsync -rtve ssh /source/directory/folder/ user@IPAddress:/destination/directory/folder
sending incremental file list
ERROR: module is read only
rsync error: syntax or usage error (code 1) at main.c(1131) [Receiver=3.0.9]
rsync: connection unexpectedly closed (112 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2]

Again, if I run the rsync task from the GUI it works: I can watch new files copy over. When I run the rsync command from the command line or the five lines in my script, I get the errors.

I have the file saved in /mnt/tank/User , and that's the location I'm running the command from for testing.

Any suggestions?

 

[Nick2253]

Can you kick off the GUI task, and check using ps (or whatever the appropriate equivalent is) what command the GUI is using?

 

[Avro]

Code:

PID USERNAME	THR PRI NICE   SIZE	RES STATE   C   TIME	WCPU COMMAND
92042 user1		  1  22	0 57156K  4804K select  0   0:03   6.52% rsync
92043 user1		  1  21	0 10376K  6356K select  5   0:02   5.61% ssh

I used top, and it shows the rsync user running rsync and ssh. Seems normal.

I've noticed that when I log in as root, the command line starts with:

Code:

root@Server01:~ #

however when I log in as the rysnc user, it simply starts with:

Code:

%

Would this make a difference? One of the (many) guides I followed to set this up was one where the rsync user was created as part of the wheel group, and the login shell was set to sh, then changed to csh after everything was working.

 

[Nick2253]

Would this make a difference?

This won't make a difference. You can customize this for each user if it matters to you.

 

[Linkman]

If you use something like ps -f (or top -c) it should give you the full command line, not just the command name, if that would help you.

(Note: I'm on Debian Linux at the moment, so possible FreeNAS (FreeBSD) options might differ.)

 

[toadman]

So does the script work if you run it as root? If yes, at least you know what the problem is at that point. (You'd have to check all paths vs what the system is using when it runs the GUI task to check for permissions errors.)

 

[Avro]

Got it working, thanks in part to the help I received here. I put the five rsync command line entries into one script and run that as a cron job from the GUI.

In diagnosing the issue:
top -a was the command that gave me the path.
I adjusted permissions on the /.ssh folder, even though it was working prior to all this. At one point FreeNAS told me it couldn't write to the known_hosts file.

So, why would the GUI execution of the rsync task work but the script wouldn't, and the command line would usually fail? This should have been the key clue...
Answer: fast (...fat) fingers. This is a FreeNAS to Synology operation, and the Synology target directory is /NetBackup. If you spell it like that, it works. However, if you type 'NetBakcup' in the script, and then copy/paste that typo four more times for the rest of the tasks, it's doomed to fail. If you grab the typo from a txt file and past it into PuTTY, that fails too. Since I'd gotten it right when setting up the tasks in the GUI, they worked.

I appreciate the assistance, and am sorry for something as stupid as a typo wasting folks' time.