Replication Progress

[mdrisser]

This isn't a question, more like a solution.
I have had, and seen that others have had, the question of how to see the progress of a replication that is currently underway. I have seen 3 different answers to this.
1. It is too difficult to do
2. It used to be in the Web GUI, but it was removed for some unknown reason

The third answer was a snippet in a forum somewhere that showed that it actually is possible to see the progress, using the 'ps' command in a shell (ps auxwwU root | grep 'zfs:') (No, that is not a typo, the two 'w' given as part of the arguments to ps are required to remove the line length limit for the line that is returned). I tested it and it does indeed seem to show the progress of the replication, although I make no claims as to the accuracy of the progress shown. For those of us who, like me, need to be able to give an answer to the higher ups about the status, I've put together a little Python Script that will do that. I present that script to you here. It is not the most elegant solution, and could probably be improved upon, but it does give us something to give to the higher ups who may ask.

Code:

#!/usr/local/bin/python

import os

# If your replications are running as a user other than root, change the name
# here
f = os.popen("ps auxwwU root | grep 'zfs:'")
rep_status = f.read()
f.close()

rep_status = rep_status.split(' ')
snapshot = rep_status[29]
percentage = rep_status[30]
percentage = percentage.strip('(').strip(':')
transferred = rep_status[31]
transferred = transferred.strip(')')
transferred = transferred.split('/')
trans_gb = int(transferred[0]) / 1024 / 1024 / 1024
trans_size_gb = int(transferred[1]) / 1024 / 1024

print "Snapshot: %s\n\t%scomplete\n\tTransferred %dGB of %dGB" % (snapshot, percentage, trans_gb, trans_size_gb)

I hope you find this useful.

 

[mdrisser]

I've gone a little bit farther in showing the replication status. Following is a PERL script I wrote which queries two FreeNAS servers and displays the replication status for each one. Additionally it creates and appends to a file, /tmp/replication_report.txt, so more easily track the progress and compare results. The script does have a few module dependencies, but these are easily enough met using CPAN.

The script is written to be run on a Unix/Linux machine, but can easily be adapted to run on a Mac or Windows box.

Code:

#!/usr/bin/perl -w
=head1 NAME
replication_status.pl

=head1 DESCRIPTION
This script connects to a FreeNAS(R) server via SSH, it then runs a simple
one line command on the server, which returns information regarding the
currently running replication process.

The returned information is converted into a more readable format, which
is printed to the screen as well as being appended to a file for later
review and comparrison of progress.

=cut
##############################
##### MODULE "IMPORTING" #####
use strict;
use Net::OpenSSH;
use Math::Round;
use Term::ReadKey;
use Getopt::Std;
use Path::Class;
use autodie;
use POSIX qw(strftime);

#####################
##### VARIABLES #####
my $res1 = "";
my $res2 = "";

##### !!!IMPORTANT!!! #####
# Change the line below to the username you will be using to login to your FreeNAS(R) server(s) with
my $SSH_USR = "YOURUSERNAME";

# Copy and change the following line as many times as needed to cover all
# of your FreeNAS(R) servers
my $tns1 = "192.168.1.28";
my $tns2 = "192.168.3.21";

# The following line is what gets run on the FreeNAS(R) server to show
# the progress of the replication.
my $PS = 'ps auxwwU root | grep "zfs:" | sed "s/\// /g"';

# The following lines allow you to enter your password without it being displayed...
print "Enter SSH password:\n";
ReadMode 2;
my $SSH_PASS = <>;
ReadMode 1;
chomp($SSH_PASS);

#####################
##### FUNCTIONS #####

# The following function is the 'heavy lifter' of the script; connecting
# to the designated FreeNAS(R) server, running the command and formatting
# the results.
sub get_rep_status {
my ($tns, $USR, $PASS) = @_;
# First we will connect to the FreeNAS box...
print "Connecting to $tns...\n";
my $ssh = Net::OpenSSH->new($USR.":".$PASS."@".$tns) or die("Can't SSH to $tns: " . Net::OpenSSH->error);
# ...and get the output of ls on the replication directory
print "Checking replication status...\n";
my $stats = $ssh->capture($PS);
my @stats = split(" ", $stats);
my $snapshot = $stats[13];
my $perc = $stats[14];
my $sent = $stats[15];
my $size = $stats[16];
# Remove unwanted characters
$perc =~ tr/(://d;
$size =~ tr/)//d;
# Get the size in GB
$sent = round($sent / 1024 / 1024 / 1024);
$size = round($size / 1024 / 1024 / 1024);
$ssh->disconnect();
my ($seconds, $minutes, $hour, $monthday, $month, $year, $wday, $yday, $isdst) = localtime(time);
my $status = "\tSnapshot: $snapshot\n\tComplete: $perc\n\tSent: $sent GB of $size GB\n\t$hour:$minutes:$seconds - $monthday/$month/$year\n";
return $status;
}

#########################
##### "MAIN" SCRIPT #####
# Connect to each server in turn
# Copy and change the following line as many times as needed to cover all
# of your FreeNAS(R) servers
$res1 = get_rep_status($tns1, $SSH_USR, $SSH_PASS);
$res2 = get_rep_status($tns2, $SSH_USR, $SSH_PASS);

# Print the results to the screen
print "\n$res1\n";
print "\n$res2\n";

# Write the results to a file for later comparisson
my $dir = dir("/tmp"); # /tmp

my $file = $dir->file("replication-report.txt"); # /tmp/file.txt

# Get a file_handle (IO::File object) you can write to
my $fh = $file->open('>>');
$fh->print("\n$res1\n\n$res2\n----------------------------------------\n");
$fh->close();

I hope you find this useful as well.

 

[skyline65]

Maybe the script could go here?

https://forums.freenas.org/index.php?forums/useful-scripts.47/

 

[mdrisser]

That would be great, but one of the Admins will need to put them there, "Insufficient privileges to post..."

 

[mdrisser]

For everyone's convenience, I've created a github repository for these scripts, you can find them here: https://github.com/mdrisser/fn_rep_status/releases