[HOW TO] FN11: Migrate to CrashPlan 6.6 w/o losing your jail backups

[HOW TO] FN11: Migrate to CrashPlan 6.6 w/o losing your jail backups

This guide works for CrashPlan for Small Business version 6.6. I borrowed heavily from Hazimil’s original resource, and much thanks and kudos to him.
https://forums.freenas.org/index.ph...-crashplan-in-an-ubuntu-vm-on-freenas-v11.65/

This is version 1.1 ,and my first resource, so please let me know how I can improve or clarify anything.

Goals:
Setup NFS on FreeNAS
Create Ubuntu VM
Mount NFS shares on the VM
Install Crashplan 6.6+ on the VM
Optional: Configure Crashplan to replace a previous FreeNAS device in Crashplan with the CrashplanVM *

* Tread carefully on this one. It worked for me on the first try, but I can not guaranty my steps are fool-proof.

Assumptions:
FreeNAS server name is called: freenas
FreeNAS server IP number is: 192.168.0.50
Main data volume is called: Reds3TB (Find / replace with "tank" or whatever you call yours)

Step 1
Download the latest Ubuntu LTS “desktop” to your FreeNAS.

# cd /mnt/Reds3TB/ISOs
# wget "http://mirror.pnl.gov/releases/16.04.3/ubuntu-16.04.3-desktop-amd64.iso"

Step 2
Using the FreeNAS GUI, create a new zvol.
Storage -> Volumes -> /mnt/Reds3TB -> create zvol
This is the “hard disk storage” for the Virtual Machine you wish to create, using the following settings:

--> zvol Name: UbuntuVM_CrashPlan
--> zvol Comments: Ubuntu vm for CrashPlan
--> zvol Size for this zvol: 50 GiB
--> zvol Force size: default
--> zvol Compressiong level: Inherit
--> zvol Spare volume: default
--> zvol Block Size: default

createzvol.png


Step 3
Using the FreeNAS GUI, create a new VM, using the following settings:

-->Name: CrashPlanVM
-->Description: Ubuntu VM for Crashplan
-->Virtual CPUs: 2
-->Memory Size (MiB): 4096
-->Boot Method: UEFI
-->Autostart: (untick)

I would recommend 2 CPUs, if you have available CPU resources. I haven’t performance tested 1 vs 2 vs 4, but with 2 CPUs just during setup I’ve seen loads above 2, and sustained 1.0X during file synchronizing.

“Code42 typically recommends allocating 1 GB (1024 MB) of memory per 1 TB of storage (or per 1 million files).” https://support.code42.com/CrashPla..._settings_for_memory_usage_with_large_backups

The Memory Size of the VM should be about 1GB higher than the amount of data you want to back up in Terabytes. ie: if you are going to backup 8TB of data, set it to 1024*9 = 9216 MiB. This leaves 1GB of headroom for the OS. You may want to add even more if you have plans to use this VM to run other jobs.

createVM.png


Step 4
You now need to add some “devices” to the VM, the first of these is the hard-disk storage (i.e. zvol) you created in Step 6. Add a device using the following settings:

-->VM: CrashPlanVM
-->Type: Disk
-->Zvol: /mnt/Reds3TB/ubuntu_crashplan
-->Mode: AHCI

addzvoltovm.png


Step 5
Next add a bootable CD-ROM to the VM, so that it can access the install .iso. Add a device using the following settings:

-->VM:CrashPlanVM
-->Type: CD-ROM
-->CD-ROM (ISO): /mnt/Reds3TB/media/ISOs/ubuntu-16.04.3-desktop-amd64.iso

addcdromiso.png


Step 6
Ensure you have a VNC application installed on your computer (not FreeNAS or the VM). I use VNC Viewer by RealVNC. TightVNC is also good.

Step 7
You now need to select and start the VM, once started you then need to connect to it via VNC. The IP number is the same as your FreeNAS server, and the port number is displayed in the "Info" box on the VM overview page in the FreeNAS GUI.

At the moment, the VM won’t actually start until you connect to it via VNC (we will change this setting later).

For example, if the IP address is 192.168.1.100 and the port is 5901, you connect to 192.168.1.100:5901.

Set the quality level (under Options in VNC Viewer) to High to avoid “RFB protocol error: invalid message type 254”. (https://forums.freenas.org/index.ph...rror-connecting-via-vnc-at-linux-setup.54582/)

vncquality.png


Step 8
Once connected via VNC, you need to follow the install process of the Ubuntu .iso:
Select Install Ubuntu from the menu and wait a few seconds for the installer to boot.
Select your install language, then press Continue.
Select “Download updates while installing Ubuntu”, and leave the third party software option un-checked. Then press Continue.
“Erase disk and install Ubuntu” is selected by default, but you should select “Use LVM with the new Ubuntu installation”, then press Install Now.
A pop-up will ask if you want to “Force UEFI installation?”, press Continue in UEFI mode.
You then get another pop-up asking you to “Write the changes to disks?”, press Continue to accept.
Select your location and press Continue.
Select your Keyboard layout and press Continue to accept.

You are now asked to create a user, which will have sudo privileges ( since you don’t have access to the root login within Ubuntu, by default).

-->Full Name: Crashplan VMUser
-->Your computer's name: crashplanvm
-->Username: crashplanvmuser
-->Password: <password>
-->Log in automatically

createubuntuuser.png


Wait for the install to finish. Once the install has finished, you need to press Restart Now. It will ask you to remove the install medium. Go back to the devices tab for the VM, select the cdrom and delete it:

deletecdrom.png


Then press enter. However, it may not restart and the five dots will continue cycling. Go to the VMs tab in the FreeNAS GUI, select the VM, and press stop on the bottom buttons. Close VNC.

stopvm.png


Step 9
Configure the VM to boot from the zvol and make some configuration changes as follows:

Edit the VNC device and untick "Wait to boot" (this means it will now boot automatically and not wait for a VNC connection).

waittoboot.png


Edit the VM and tick Autostart (this will ensure that the VM starts automatically on a FreeNAS server reboot).

autostart.png


Once you have made the above changes, you can restart the VM.

If you see the UEFI Interactive Shell, and the VM does not boot into Ubuntu automatically, see Step 14 in Hazimil’s doc: https://forums.freenas.org/index.ph...-crashplan-in-an-ubuntu-vm-on-freenas-v11.65/

Step 10
We are now at the stage where we want to share the volume(s) we want to NFS mount to the Crashplan VM.

In the FreeNAS GUI, select Sharing -> NFS -> Add Unix (NFS) Share

nfs_share.png


Define the path you want to mount (aka the top level of the data you want to backup), and click All Directories and Read Only options:

nfs_share1.png


Optional:
Click Advanced and lock down the NFS share to just the IP of your Crashplan VM. Don’t do this if you are on a DHCP network, because the IP of your Crashplan VM might change when it reboots.

nfs_share2.png


Select OK and Yes to the “Do you want to start this service” pop up.

Repeat this step for as many individual paths that you want to present to the Crashplan VM to backup.

Optionally, you can share the root of each of your volumes and do the choosing of what to backup in the Crashplan GUI. This is what I am doing in my example above.

Step 11

PRO TIP
If you prefer a Putty window to VNC, open a terminal in the Crashplan VM and install SSH. Some things in this How To will be easier to cut and paste in a Putty session, and some will need to be done in the VNC session.

$ sudo apt-get install -y openssh-server
$ sudo service ssh status
[sudo] password for crashplanvmuser:
● ssh.service - OpenBSD Secure Shell server
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enab
Active: active (running) since Tue 2017-12-12 11:43:41 PST; 1min 48s ago
Main PID: 27616 (sshd)....

Type "ifconfig" in the terminal on the Ubuntu VM, note your VM's IP and fire up a Putty session using that IP.

/End PRO TIP

Step 12
On the Ubuntu Crashplan VM we need to ensure NFS client packages are installed:

$ sudo apt-get install nfs-common

installnfs1.png


Press “Y”. This should only take a few seconds to install.

Step 13
Create the mount points on the Crashplan VM that you want to NFS mount from the FreeNAS server. Try to have a naming convention that isn’t too complicated and will grow. eg /mnt/freenas directory using mkdir /mnt/freenas, then create directories for each dataset you wish to backup, i.e.

$ sudo mkdir /mnt/freenas
Under there you will create the directory names where you will mount the paths you shared.

The simplest way is to just mount your volumes, if that's what you shared in Step 10.
ie:
$ sudo mkdir /mnt/freenas/HugeRaidZ2/
$ sudo mkdir /mnt/freenas/SmallRaidZ/

You'll be able to narrow down what you actual backup in the Crashplan client.

Or you can get more specific, if that's your style:
$ sudo mkdir /mnt/freenas/HugeRaidZ2/movies
$ sudo mkdir /mnt/freenas/SmallRaidZ/kidsshows
$ sudo mkdir /mnt/freenas/HugeRaidZ2/Music
$ sudo mkdir /mnt/freenas/SSD/logs
Etc.

Note that the folder names do not have to match the paths you shared via NFS in the FreeNAS GUI, but try not to make it confusing for yourself, alright?

Before mounting, chown the folders to your VM user’s name:

$ sudo chown crashplanvmuser:crashplanvmuser -R /mnt/freenas

Step 14

Test one of the nfs shares you created in Step X

$ sudo mount -t nfs IP_OF_YOUR_FREENAS:/mnt/Reds3TB /mnt/freenas/Reds3TB

ie:
$ sudo mount -t nfs 192.168.1.100:/mnt/Reds3TB /mnt/freenas/Reds3TB

If that works, then you can add the entry into /etc/fstab.

First unmount it:

$ sudo umount /mnt/freenas/Reds3TB

Using your favorite linux editor ( I’ll be using vi ) add the lines to fstab that will automount the nfs shares at boot time.

$ sudo vi /etc/fstab
Add the line that mounts the nfs share to the desired folder:
eg:
192.168.1.100:/mnt/Reds3TB /mnt/freenas/Reds3TB nfs ro 0 0

“ro” = mount read only, which is redundant since we checked Read Only on the NFS share in the FreeNAS GUI, but doesn’t hurt.

Run the automounter. If this works it will auto mount at boot time:
$ sudo mount -a

Getting a blank prompt back, means no errors and it worked!

Verify:
crashplanvmuser@crashplanvm:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 2.0G 0 2.0G 0% /dev
tmpfs 395M 6.0M 389M 2% /run
/dev/mapper/ubuntu--vg-root 45G 4.5G 38G 11% /
tmpfs 2.0G 112K 2.0G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/sda2 473M 125M 324M 28% /boot
/dev/sda1 511M 3.4M 508M 1% /boot/efi
tmpfs 395M 60K 395M 1% /run/user/1000
192.168.1.100:/mnt/Reds3TB 5.0T 2.5T 2.5T 51% /mnt/freenas/Reds3TB

Go ahead and cd into the folder and look around. If you are on the VNC Desktop, you can browse there using the Files app.

Hopefully you’ll have access to all your files. If not you might need to tweak the file and folder permissions in your share. (If people have permissions issues in the forum, I’ll add some optional steps here).


Step 15
Let's install CrashPlan!

In the Crashplan VM VNC session, load Firefox and navigate to https://www.crashplan.com, find and click Download, then select Linux version 6.6+, and save the file.

Either from Putty or the terminal in the VM:
$ cd Downloads
$ ls CrashPlan*
CrashPlanSmb_6.6.0_1506661200660_4347_Linux.tgz

Uncompress the file:
$ tar -zxvf CrashPlanSmb_6.6.0_1506661200660_4347_Linux.tgz
crashplan-install/install.sh
crashplan-install/uninstall.sh
crashplan-install/
crashplan-install/scripts/
crashplan-install/CrashPlanSmb_6.6.0.cpi
crashplan-install/INSTALL
crashplan-install/README
crashplan-install/install.defaults
crashplan-install/scripts/CrashPlan.desktop
crashplan-install/scripts/CrashPlanDesktop
crashplan-install/scripts/CrashPlanEngine
crashplan-install/scripts/crashplan
crashplan-install/scripts/run.conf

Become root to install:
$ sudo su -
[sudo] password for crashplanvmuser: <password>

Navigate back to the Downloads directory for crashplanvmuser:
# cd /home/crashplanvmuser/Downloads/crashplan-install/

Run the installer and accept the defaults for the prompts:
# ./install.sh

Welcome to the Code42 CrashPlan installer.

Press enter to continue with installation.

Validating environment...
detected root permissions
/bin/grep
/bin/sed
/bin/cpio
/bin/gzip
/usr/bin/cut
/usr/bin/head
/usr/bin/tail
/usr/bin/who
/usr/bin/getopt

What parent directory do you wish to install CrashPlan into? [/usr/local]
/usr/local/crashplan does not exist. Create /usr/local/crashplan? (y/n) [y]

What directory do you wish to link the CrashPlan executable to? [/usr/local/bin]

What directory do you wish to store incoming backup data? [/usr/local/var/crashplan]
/usr/local/var/crashplan does not exist. Create /usr/local/var/crashplan? (y/n) [y]

What directory contains your SYSV init scripts? [/etc/init.d]

What directory contains your runlevel init links? [/etc/rc5.d]

Your selections:
CrashPlan will install to: /usr/local/crashplan
And put links to binaries in: /usr/local/bin
And store data in: /usr/local/var/crashplan
Your init.d dir is: /etc/init.d
Your current runlevel directory is: /etc/rc5.d

Is this correct? (y/n) [y]

downloading the JRE using wget
--2017-12-12 15:38:33-- https://download.code42.com/installs/proserver/jre/jre-8u121-linux-x64.tar.gz
Resolving download.code42.com (download.code42.com)... 216.17.8.19
Connecting to download.code42.com (download.code42.com)|216.17.8.19|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://download2.code42.com/installs/proserver/jre/jre-8u121-linux-x64.tar.gz [following]
--2017-12-12 15:38:34-- https://download2.code42.com/installs/proserver/jre/jre-8u121-linux-x64.tar.gz
Resolving download2.code42.com (download2.code42.com)... 216.17.8.57
Connecting to download2.code42.com (download2.code42.com)|216.17.8.57|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 73683404 (70M) [application/octet-stream]
Saving to: ‘jre-8u121-linux-x64.tar.gz’

jre-8u121-linux-x64.tar.g 100%[====================================>] 70.27M 13.9MB/s in 5.3s

2017-12-12 15:38:40 (13.3 MB/s) - ‘jre-8u121-linux-x64.tar.gz’ saved [73683404/73683404]

Java Installed.
Unpacking /home/crashplanvmuser/Downloads/crashplan-install/./CrashPlanSmb_6.6.0.cpi ...
386569 blocks
'/usr/local/crashplan/app.asar' -> '/usr/local/crashplan/electron/resources/app.asar'

Your Linux system is currently configured to watch 8192 files in real time.
We recommend using a larger value; see the CrashPlan support site for details

Starting CrashPlan Engine ... Using standard startup
OK

CrashPlan has been installed and the Service has been started automatically.

Press Enter to complete installation.

Important directories:
Installation:
/usr/local/crashplan
Logs:
/usr/local/crashplan/log
Default archive location:
/usr/local/var/crashplan
Readme:
/usr/local/crashplan/doc

Start Scripts:
sudo /usr/local/crashplan/bin/CrashPlanEngine start|stop
/usr/local/crashplan/bin/CrashPlanDesktop


To start the Desktop UI:
/usr/local/bin/CrashPlanDesktop

Installation is complete. Thank you for installing Code42 CrashPlan.

Step 16
In the output of the installer, there was an important hint:

“Your Linux system is currently configured to watch 8192 files in real time.
We recommend using a larger value; see the CrashPlan support site for details”
https://support.code42.com/CrashPlan/4/Troubleshooting/Linux_real-time_file_watching_errors

We are going to fix that now.

While still the root user:
# cat /proc/sys/fs/inotify/max_user_watches
8192

With your favorite text editor, open /etc/sysctl.conf
# vi /etc/sysctl.conf

Add these two lines to the bottom of the file (as of this writing, the setting does not exist by default Ubuntu in 16.04, so we are going to add it):
# Added for Crashplan
fs.inotify.max_user_watches=1048576

Remember to include the hash for the comment. It should look like this:

inotify.png



Enable it (will retain setting through reboots)
# sudo sysctl -p /etc/sysctl.conf
fs.inotify.max_user_watches = 1048576

Verify
# cat /proc/sys/fs/inotify/max_user_watches
1048576

Restart the VM and run the verify again after it comes up:
$ sudo cat /proc/sys/fs/inotify/max_user_watches
1048576

At this point you can start Crashplan and get on with setting it up as a new device.

Optional Step 17
Attaching to your old backup archive. Cross your fingers, and do whatever you do to invoke good luck.

Huzzah! I was able to use the “replace a device” feature of the Crashplan client when setting it up on the VM to retain my 1.1 TBs of backups from my old Crashplan plugin jail. To do this, I had to first figure out what my old jail mount points were, and then replicate the mount point names in the fstab of the VM.

If you don’t remember what your old jail mount points looked like, you might be able to find them on the Crashplan web page. Navigate to the old device’s Edit -> Backup tab and there is a snippet of directory structure in the “Included files” window:

replaceexisting2.png


Using this hint, I created a second mount point to replicate the above file structure, matching the syntax of the jail I previously was using. For my old jail, /mnt/Reds3TB/media was mounted as /mnt/CrashPlan in the jail. To replicate that with nfs mounts the following worked:

$ sudo vi /etc/fstab

192.168.1.100:/mnt/Reds3TB/media /mnt/CrashPlan nfs ro 0 0

Save it
$ sudo mkdir /mnt/CrashPlan
$ sudo mount -a

Test it:
$ ls /mnt/CrashPlan/Data/wedding/wedding\ games.doc
/mnt/CrashPlan/Data/wedding/wedding games.doc


On the Crashplan client walk through the steps to Replace a Device, and verify your previous backup structure exists.. (didn’t screen capture a few of these, so I am borrowing from Crashplan’s walk through)

Step 1
choose your old FreeNAS Crashplan jail device to replace, and recross your fingers/say a prayer/etc:

Step1-replace-device-update.png


Step 2:
Verify the file structure matches what you setup in /etc/fstab. Don’t add any new files or folders at this time.

replace_device_browser.png


On Step 3, Transfer settings to new device.

transfer.png


However, since my old FreeNAS server was down, I had to wait until the configuration setup timed out with:

transferfail.png


Don't Panic. Don’t press retry. Just wait for a login screen to reappear. Be Patient.

Meanwhile on the Crashplan website, you should see that the device “crashplanvm” now has ownership of your previously backed up data, and the old backup device name is gone. Breath Deep.

Log back into the Crashplan client. Again it said it was going to transfer settings to the new device. This time, the screen came back quickly with a “Your device is ready!” message:

ready.png


Press Finish.

If all went well, it should start comparing files, which will take awhile, but at least you don’t have to re-upload all the data:

synchronizing.png


I believe it's just comparing hashes of the files, to ensure what you have locally matches what CP has in the cloud. CPU load went to nearly 3 at the beginning of this on my system, so it possibly could have been sped up with additional CPUs allocated to the VM. YMMV.

Step 18

Update Crashplan to use the Memory you allocated to the VM for your backup size. You probably want to do this before you start the bulk of your backups, but after the synchronization is finished.

Quoted from https://support.code42.com/CrashPla..._settings_for_memory_usage_with_large_backups

Open the CrashPlan app.
If necessary, sign in to your account.
Use the keyboard shortcut for your operating system:
Linux: Ctrl+Shift+C
The CrashPlan command-line area opens.
Enter the following command, using a value appropriate for the size of your backup selection (for example, 1536 for a 1.5 TB selection):
java mx 1536, restart
The CrashPlan app closes.

And that should be it! Best of luck to you!
  • nfs_4.png
    nfs_4.png
    34.4 KB · Views: 711
  • SetupFreeNASCrashplan_wNFS.png
    SetupFreeNASCrashplan_wNFS.png
    390.4 KB · Views: 765
  • ssh.png
    ssh.png
    56.1 KB · Views: 622
Author
joerawr
Views
1,727
First release
Last update
Rating
5.00 star(s) 1 ratings

Latest updates

  1. Cleaned up grammar, spelling, formatting...

    Cleaned up grammar, spelling (neither my strong point), formatting and eliminated some...

Latest reviews

No idea what it is about but wow - this looks like one thought out manual! Thank you
Top