Guide to backup home computers to CrashPlan on FreeNAS

[stryk9]

Just a little background on why I am even taking the time to write this. You might be saying, there are already guides out there to do this. While this is true none of these guides take into account the uniqueness of the dated FreeNAS CrashPlan plugin.

Overview

After following countless guides online, I was still unable to fully configure my laptop to backup to a CrashPlan installation running on my FreeNAS machine. After playing around and messing with it for the better part of the day I figured it out. It turns out that due to the fact the current version of the CrashPlan plugin, as of FreeNAS-9.3-STABLE-201511040813 is CrashPlan 3.6.3_1. Since the newest version of CrashPlan is actually 4.4.1, this causes an issue when trying to connect to a headless CrashPlan installation on FreeNAS.

Big thanks to sirkkalap for his write up where most of this came from. https://github.com/sirkkalap/freenas-crashplan-howto

How-to : Crashplan & Freenas
Pre-requisites

1. Install CrashPlan 3.6.3 on your computer you want backed up but do not log in once the installation is complete.
http://downloads.tomsguide.com/-CrashPlan,0301-47759.html
2. FreeNAS-9.3-RELEASE-x64

Install
Step 1: Install the Crashplan plugin

Install Crashplan Step 2 : Accept TOS Step 3 : Enable Crashplan plugin Step 4 : Create a sshd user for the Crashplan jail, enable TCP forwarding

To configure the Crashplan running inside a FreeNAS jail we need to use another machine that has graphical user interface. The jail has no user interface, only text console. Below is an overview of the connection that lets us use another machines GUI to configure the server running inside headless jail.

To login to jail when no sshd is available in it you must either a) use jexec command on FreeNAS console or b) use the jail console in FreeNAS WebUI.

a) Connect to jail. If you are on FreeNAS login. Note that the JID may be different.

Code:

[root@freenas] ~# jls
   JID  IP Address      Hostname                      Path
     1  -               crashplan_1                   /mnt/zpool/jails_2/crashplan_1
[root@freenas] ~# jexec 1 /bin/tcsh

b) Open terminal to jail from WebUI Install bash (it is required during the crashplan automatic updates)

Code:

pkg install bash
# crashplan is expecting bash to be in /bin
ln -s /usr/local/bin/bash /bin/bash

Enable sshd and crashplan (crashplan may already be enabled). The instructions below are taken from the FreeNAS wiki

Edit /etc/rc.conf

Code:

...
sshd_enable="YES"
crashplan_enable="YES
...

Create User for ssh-access
Create a new user, note that the user needs to be in group wheel

Code:

root@crashplan_1:/ # adduser
Username: crashplan
Full name:
Uid (Leave empty for default): 1001
Login group [crashplan]:
Login group is crashplan. Invite crashplan into other groups? []: wheel
Login class [default]:
Shell (sh csh tcsh nologin) [sh]: tcsh
Home directory [/home/crashplan]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password: (ENTER YOUR PASSWORD)
Enter password again: (ENTER YOUR PASSWORD)
Lock out the account after creation? [no]:
Username   : crashplan
Password   : *****
Full Name  :
Uid        : 1001
Class      :
Groups     : crashplan wheel
Home       : /home/crashplan
Home Mode  :
Shell      : /bin/tcsh
Locked     : no
OK? (yes/no): yes
pw: mkdir(/home/crashplan): No such file or directory
adduser: INFO: Successfully added (crashplan) to the user database.
Add another user? (yes/no): no
Goodbye!

Start the sshd manually this one time:

Code:

root@crashplan_1:/ # service sshd start
...

Now, let's create a tunnel. This will redirect my laptop port 4200 to port 4243 on the crashplan jail. The console will stay open, you may close it when you are done with the step 7.

Use PuTTY, a telnet and SSH application for Windows and Linux, to forward port 4200 on the local computer to port 4243 on the remote computer.

Port Forwarding with PuTTY
1. Open PuTTY.
2. Go to the settings for Connection > SSH > Tunnels.
3. Enter these settings:
Source port: 4200
Destination: localhost:4243
4. Click Add.
5. Go to the Session settings.
6. Enter the appropriate connection information in Host Name (or IP address) field.
(Optional) To save these connection options for later reuse, enter a label in Saved Sessions and click Save
7. Click Open to open the SSH tunnel.
8. Sign in to the remote computer using your SSH credentials.

Step 5 : Verify Crashplan is running and listening

Code:

[root@freenas] ~# jexec crashplan_1 sockstat -4
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
crashplan sshd      4149  5  tcp4   192.168.1.103:22      192.168.1.83:53226
root     sshd       4147  5  tcp4   192.168.1.103:22      192.168.1.83:53226
root     java       3952  56 tcp4   127.0.0.1:4243        *:*
root     java       3952  57 tcp4   *:4242                *:*
root     java       3951  56 tcp4   127.0.0.1:4243        *:*
...

Step 6: Mount storage directories
From the FreeNAS GUI you must configure the storage for the jail so that you can back up the filesystem. This can be done if you go to:

Code:

Jails -> crashplan_1 -> Storage -> Add Storage

Step 7 : Configure Crashplan UI to connect to remote host (through ssh-tunnel)

Configure the desktop (my laptop) to use the ssh tunnel by editing the ui.properties file. Make a backup of the file, if you use the Crashplan on the desktop machine too. This change will divert the connections to port 4200, so it will not work without the ssh-tunnel and it will not connec to local crashplan any more.

You may find the file in the following location:

Code:

Linux (if installed as root): /usr/local/crashplan/conf/ui.properties
Mac: /Applications/CrashPlan.app/Contents/Resources/Java/conf/ui.properties
Solaris (if installed as root): /opt/sfw/crashplan/conf/ui.properties
Windows: C:\Program Files\CrashPlan\conf\ui.properties

Change the service port to 4200 by removing un-commenting (removing #) the #servicePort=4200 line, which we will use to tunnel to the remote connection.

Code:

servicePort=4200

Restart the Crashplan service by typing services.msc into the Start Menu search box.

Locate the Crashplan service and click restart.

Step 8: Connect with Crashplan UI and update.

Launch the modified Crashplan UI on the desktop (my laptop). Ssh-tunnel must be open. Immediatly after you login the UI will exit and the crasplan plugin will start the update process. This process in incremental, that means that the crasplan plugin will restart itself many times, until reaching the latest version.

*Mine did not do this action. I was able to log in and configure the headless server. When configuring your headless server, be sure to configure the directory where you want to store your backups.

To change your backup directory, in Crashplan, go to Settings>General Tab and click the Configure button to the right of "Inbound backup from other computers" near the bottom, you can change the default backup location.

This is where things got interesting for me.
unknown to me, during this time, my FreeNAS plugin decided it was going to update itself. The update may take some time, so you may want to wait a while before you continue.

1. From the FreeNAS GUI Restart the CrashPlan Jail

After doing this you can see how the version numbers increase with

Code:

root@crashplan_1:/ ee /usr/pbi/crashplan-amd64/share/crashplan/log/app.log

Once the update is complete, make sure the client (desktop application) is the same version as the plugin, because even though they are supposed to auto update to the latest version, if there is a difference it will not work.

Step 9: Copy the authentication token required after the update

Follow the instruction in this Step 1
You can find the plugin token here on the FreeNAS box

Code:

root@crashplan_1:/ cat /var/lib/crashplan/.ui_info

Step 10: Connect with Crashplan UI...Finally!

your settings should be reverted in order to continue. Step 9 should have you reset this.
you may need to:

Restart the Crashplan service by typing services.msc into the Start Menu search box.
Locate the Crashplan service and click restart.

Now launch the UI client, login and configure your backups.
You may close the ssh-tunnel at this point when the Crashplan UI is closed.

Common Problems

After update to 3.6.3 (happens automatically, pushed from Crashplan) the service fails to start. Thanks to mstinaff thread: http://forums.freenas.org/index.php?threads/crashplan-3-6-3.18416/ we now have a working solution to this.

You need to manually update a file. First make a backup:

Code:

root@crashplan_1:/ # cd /usr/pbi/crashplan-amd64/share/crashplan/bin
root@crashplan_1:/usr/pbi/crashplan-amd64/share/crashplan/bin # cp -v run.conf run.conf.bkp

The original file for

Code:

/usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf

looks like this:

Code:

SRV_JAVA_OPTS="-Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx1024m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"
GUI_JAVA_OPTS="-Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"

You will need to update it to look like this:

Code:

SRV_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx1024m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"
GUI_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false

Finally, you need to stop and start the jail. Without doing this the plugin will not start correctly.

 

[fracai]

I love this post, not only because of the utility, but because now I don't have to slog through the steps for my own write up!

The only addition I have is that some backup issues were fixed for me by manually resetting the cache. I think there might be one or two other troubleshooting threads that I've seen that included some useful information.

This needs to be stickied as soon as possible. I cannot compliment this post enough!

 

[ChiknNutz]

This is good info, but how the heck is a new person coming into FreeNAS ever supposed to figure this out on their own? The way FreeNAS is advertised all over the place, is it is super easy to set up and has all these plugins just waiting for you to use them. This lengthy guide is an example of how this is not really the case.

 

[depasseg]

This is good info, but how the heck is a new person coming into FreeNAS ever supposed to figure this out on their own? The way FreeNAS is advertised all over the place, is it is super easy to set up and has all these plugins just waiting for you to use them. This lengthy guide is an example of how this is not really the case.

Crashplan is unique in its ability to make the simple very difficult.

 

[fracai]

Crashplan is unique in its ability to make the simple very difficult.

No kidding. I cannot understand how they can make the remote gui connection so complicated.

 

[ChiknNutz]

And this is one of the main features I am looking forward to once the new gear arrives (since it turns out my old setup just wouldn't cut it). Now it appears I need at least a Master's in IT to get Crashplan running 8-s

 

[cyberjock]

This is good info, but how the heck is a new person coming into FreeNAS ever supposed to figure this out on their own? The way FreeNAS is advertised all over the place, is it is super easy to set up and has all these plugins just waiting for you to use them. This lengthy guide is an example of how this is not really the case.

Considering how nearly impossible most of the forum users would be if they were told to make a FreeBSD system, keep it up to date, create a working zpool using recommended best practices, then get CIFS and NFS working and stable, FreeNAS is so turnkey it's laughable. If you've never had to setup CIFS or NFS yourself by editing conf files, you have no way of even gauging how incredibly difficult it can be.

FreeNAS is advertised as easy (and relative to doing it yourself, it is easy beyond understanding). Even some FreeBSD experts said that if they were to create a server using even 1/2 the features that FreeNAS has, what you can do in 15 minutes on FreeNAS would take a BSD expert several days to implement.

Also, jails are considered an advanced function. It's not supposed to be easy, and there is no way to make it easier. It's BSD in the raw- its the CLI. Either accept it for what is (hard and a PITA to learn if you've never done CLI before) or simply don't use it. I realize you aren't complaining, I'm just mentioning it. ;)

 

[ayysir]

did these steps but none have worked...
dont see .ui_info file in /var/lib/crashplan/ on my freenas....

 

[ChiknNutz]

I am curious why go to the effort of using Crashplan when you can just use any number of built-in or free third-party applications for backups (i.e. Windows backup and recovery, Aeomi Backupper, Comodo, etc.)? Since all of these (Crashplan included) is installed on the client computer and can be set to backup to your NAS, what is it about Crashplan that folks are willing to go to the effort and suffer thru the various headaches I've noticed?

 

[pirateghost]

I am curious why go to the effort of using Crashplan when you can just use any number of built-in or free third-party applications for backups (i.e. Windows backup and recovery, Aeomi Backupper, Comodo, etc.)? Since all of these (Crashplan included) is installed on the client computer and can be set to backup to your NAS, what is it about Crashplan that folks are willing to go to the effort and suffer thru the various headaches I've noticed?

I use crashplan to back up files to their online service. I don't use crashplan to back up to my NAS. You can't back up client backups to their cloud service so you pick one or the other. Either way, the backup and restore process is extremely easy.

I don't really understand why so many people have so many issues. It took me about 20 minutes to get everything configured and running and it takes me about 5 minutes to update clients when the server updates. I have torn down and rebuilt my crashplan plugin many times trying to duplicate some of the issues people have here. I haven't been successful in duplicating the issues yet.

 

[ChiknNutz]

I am not currently using it. I installed the plugin and the app on one computer, but haven't went thru the effort of getting them talking yet. I have noticed enough complaints that I was wondering if it was worth the effort, but then there could be a gazillion folks that have it working w/o incident and it's only the few that have issues we hear about.

 

[pirateghost]

I am not currently using it. I installed the plugin and the app on one computer, but haven't went thru the effort of getting them talking yet. I have noticed enough complaints that I was wondering if it was worth the effort, but then there could be a gazillion folks that have it working w/o incident and it's only the few that have issues we hear about.

That's pretty much the case I think

 

[Fish]

I couldn't get the port forwarding part going. Can anyone write the instruction for the current version of Putty?

 

[pirateghost]

I couldn't get the port forwarding part going. Can anyone write the instruction for the current version of Putty?

Putty hasn't changed in years....

 

[James Random]

did these steps but none have worked...
dont see .ui_info file in /var/lib/crashplan/ on my freenas....

Same here. (Among many other problems ... :()

 

[nello]

Has anyone considered updating this guide for either:

• An installation in a new FreeNAS 9.10 Jails? Does anything change?
• Migrating a CrashPlan installation from a 9.3.x Jail installation to a 9.10 Jail? I'm most interested in whether users can retain their archives vs. starting a new archive and losing all their history. (Yes, I'm assuming that the archives are saved outside of the Jail.)

I'd like to upgrade to 9.10, but I'd rather not be the first person to set up CrashPlan on a 9.10 Jail. :)

Update 2016.05.02
I changed references to 10.x -> 9.10. Sorry for the confusion.

 

[depasseg]

I'd like to upgrade to 10.x, but I'd rather not be the first person to set up CrashPlan on a 10.x Jail. :)

10.x is months away from being released. I would wait.

 

[nello]

10.x is months away from being released. I would wait.

Yes, my mistake. Sorry.

I meant FreeNAS 9.10:

https://forums.freenas.org/index.php?threads/freenas-9-10-release-now-available.42223/

Jails:
FreeBSD Jails now default to a FreeBSD 10.3-RC2 based template.
Old jails, or systems on which jails have been installed, will still
default to the previous FreeBSD 9.3 based template. Only those machines
using jails for the first time (or deleting and recreating their jails
dataset) will use the new template.​

 

[stualden]

In case you haven't already seen it, see this parallel thread related to this topic:

https://forums.freenas.org/index.php?threads/what-to-do-about-crashplan-on-9-10.42889/

The conclusion I (and others) came to was that using using the CrashPlan plugin is no longer worth the trouble. With 9.10 I can no longer get the plugin to work, and certainly all bets are off for 10.x. To get automatic CP updates and avoid all the monkeying around with a headless arrangement, it seems a lot easier either to run Linux CrashPlan on Linux running in jail (e.g., VirtualBox) or run it anywhere you like outside of FreeNAS. Any of these approaches is very likely to remain workable post-10.x.

If you do migrate, you should know that "adoption" of one computer by another with a different filesystem does not necessarily trigger a brand-new backup - see this:

https://support.code42.com/CrashPla...h_A_Different_Operating_System_Or_File_System

I was able to move from FreeNAS (being backed up to another computer) to a Windows box (accessing the FreeNAS CIFS shares and backing them up to that same other computer) with no troubles and no re-do of the existing backup.

 

[picklefish]

In case you haven't already seen it, see this parallel thread related to this topic:

The conclusion I (and others) came to was that using using the CrashPlan plugin is no longer worth the trouble.

Thanks just wasted a couple hours trying to get it working following the guides haha. Dang it.