Replication does not run

[kavermeer]

I have set up my system with snapshot tasks during the day, and replication tasks in the evening. However, it seems that my snapshots are not replicated at all, and I wonder whether this is because the time periods do not overlap.

A quick look at autosnap.py shows that it exits if there are no snapshots that need to be created (so it stops processing in the evening), and autorepl.py is only called at the end of autosnap.py. Therefore, autorepl.py may not run during the evening, while during the day it doesn't do anything.

Does this make sense? Is this described anywhere? I do not get any errors, it just doesn't seem to work...

 

[dlavigne]

Which version of FreeNAS? What are your Begin and End settings for the replication task (these set the replication window).

 

[kavermeer]

The version is: FreeNAS-8.3.0-RELEASE-x64 (r12701M)

In my replication task, I have 'Begin': '20:45:00' and 'End': '23:59:00'. The snapshot task is non-recursive, with a lifetime of 1 year, begin '08:30:00', end '18:30:00', 15 minutes interval, Mo-Fr.

 

[kavermeer]

Would a solution be to run autorepl from a cron job? Or would that cause other problems?

 

[dlavigne]

It actually sounds like there is a replication error occuring (assuming that snapshots exist). Try the replication troubleshooting section in 6.2.3 (http://www.freenas.org/images/resou...s8.3.1_guide.html#__RefHeading__464_145473606) to see if there is an issue with SSH or with a stuck snapshot.

 

[kavermeer]

SSH is fine. I've had stuck snapshots in the past (when the replication task was still running during the whole day), but those were reported by email. Simply reinitializing the remote side will of course fix things, but will also make it impossible to determine what went wrong.

Could you enlighten me on what's wrong in my analysis of the autosnap.py script in relation with the periods of the snapshots and the replication tasks? That would help me in understanding the system, and in finding a solution!

 

[dlavigne]

Unless you have found a bug, a snapshot task calls replication after the snapshot is created, unless the replication window is set to a different time. In that case, snapshots are queued until the replication begin time, at which time replication of the queued snapshots begins.

These tasks are already in cron. Can you paste the contents of your /conf/base/etc/crontab.

 

[kavermeer]

I agree with your explanation. Except that, as mentioned in my first post, it seems that autosnap exits if there are no snapshot tasks at that time:

Code:

# Do not proceed further if we are not going to generate any snapshots for this run
if len(mp_to_task_map) == 0:
    exit()

So, if I have snapshots tasks between 8 am and 8 pm, and replication tasks between 9 pm and 12 pm, autosnap will exit before 8 am and after 8 pm, thereby not giving autorepl a chance to run. Between 8 am and 8 pm, autorepl is run, but at those times it doesn't do anything, because it's not between 9 pm and 12 pm.
My crontab:

Code:

# /etc/crontab - root's crontab for FreeBSD
#
# $FreeBSD: src/etc/crontab,v 1.33.2.1 2009/08/03 08:13:06 kensmith Exp $
#
SHELL=/bin/sh
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin
#
#minute    hour    mday    month    wday    who    command
#
*/5    *    *    *    *    root    /usr/libexec/atrun
#
# Save some entropy so that /dev/random can re-seed on boot.
*/11    *    *    *    *    operator /usr/libexec/save-entropy
#
# Rotate log files every hour, if necessary.
0    *    *    *    *    root    newsyslog
#
# Perform daily/weekly/monthly maintenance.
1    3    *    *    *    root    periodic daily
15    4    *    *    6    root    periodic weekly
30    5    1    *    *    root    periodic monthly
#
# Adjust the time zone if the CMOS clock keeps local time, as opposed to
# UTC time.  See adjkerntz(8) for details.
1,31    0-5    *    *    *    root    adjkerntz -a
 
*/5    *    *    *    *    root    /usr/local/bin/python /usr/local/bin/graph.py
0    *    *    *    *    root    /bin/sh /root/save_rrds.sh
 
0    *    *    *    *    root    /usr/local/bin/python /usr/local/bin/mfistatus.py > /dev/null 2>&1
*/5    *    *    *    *    root    /usr/local/bin/python /usr/local/www/freenasUI/tools/alert.py > /dev/null 2>&1
*    *    *    *    *    root    /usr/local/bin/python /usr/local/www/freenasUI/tools/autosnap.py > /dev/null 2>&1
 
15    3    *    *    *    root    /usr/local/bin/python /usr/local/www/freenasUI/tools/cachetool.py expire >/dev/null 2>&1
30    3    *    *    *    root    /usr/local/bin/python /usr/local/www/freenasUI/tools/cachetool.py fill >/dev/null 2>&1
0    3    *    *    *    root    find /tmp/ -iname "sessionid*" -ctime +1d -delete

 

[kavermeer]

Tonight, I manually ran the autorepl script at a time that matches the time of the replication task. It replicated all the snapshots. So it does indeed look like a bug to me: A replication task is only performed if there is an overlap between the period (begin-end) of a (any) snapshot task, and the period (begin-end) of the replication task.

Bug report: http://support.freenas.org/ticket/2345