SOLVED Upgrade Procedure for Crashplan 4.4.1 on older FreeNAS 9.2.1.7

[jadz]

I have not upgraded to FreeNAS 9.3 because of initial hardware compatibility issues with my HP ProLiant server. As such I am still running FreeNAS 9.2.1.7. As many others experienced recently, CrashPlan failed on my jail/pluin install. I was not able to get the original installation happy, so I attempted to setup a clean jail. These are the steps I followed.

I am no expert in FreeBSD, FreeNAS or CrashPlan. But I tried to piece together the information found in various threads in this forum to make a procedure that works to get backups working for me. Please feel free to provide corrections to the information here.

1. Install the CrashPlan Plugin (this both creates a jail and installs the plugin).
   
   
2. Enable ssh (optional, you can also use the web-based terminal access or jump through the primary FreeNAS shell - see the help documentation for more info about accessing your jails).
   1. vi /etc/rc.conf
      
      
   2. change sshd_enable=”NO” to sshd_enable=”YES”
      
      
   3. vi /etc/ssh/sshd_config to uncomment PermitRootLogin setting and change value to yes
      
      
   4. set password for root user in jail (open terminal using web-interface then just run the passwd command to set the password)
3. Upgrade default mis-configured pkg software (required due to older jail template see https://forums.freenas.org/index.php?threads/keeping-the-jails-up-to-date.20062/ )
   1. portsnap fetch extract (this downloads and extracts the ports, and takes a while)
      
      
   2. cd /usr/ports/ports-mgmt/pkg
      
      
   3. make deinstall reinstall
4. Install Bash and link it where the CrashPlan upgrader is expecting
   1. pkg install bash
      
      
   2. ln -siv /usr/local/bin/bash /bin/bash
5. Edit run.conf options for more memory and to validate the java.nio.channels.spi.SelectorProvider setting (the initial install may have this correctly configured, as part of the upgrades that CrashPlan does these values will be broken, you will come back and check this file during the upgrade steps).
   1. vi /usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf
      
      
   2. SRV_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx2048m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false -Djava.net.preferIPv4Stack=true"GUI_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"

1. Restart Crashplan
   1. /usr/local/share/crashplan/bin/CrashPlanEngine stop
      
      
   2. cpuset -l 0 /usr/local/share/crashplan/bin/CrashPlanEngine start
2. Optional - Check that base version is working:
   1. SSH to Jail
      
      
   2. Forward port 4200 to 127.0.0.1:4243 (this only works on older CrashPlan versions, like the 3.6.3 that comes with the FreeNAS 9.2.1.7 plugin).
      
      
   3. Install Crashplan 3.6.3 on the Windows Client
      
      
   4. Open Crashplan client and validate you are able to connect to crashplan 3.6.3 on the server.
3. Restart crashplan service again (which should kick off the upgrade)
   1. /usr/local/share/crashplan/bin/CrashPlanEngine stop
      
      
   2. cpuset -l 0 /usr/local/share/crashplan/bin/CrashPlanEngine start
4. View upgrade status
   1. tail -f /usr/pbi/crashplan-amd64/share/crashplan/log/history.log.0
      
      
   2. I 10/15/15 11:32AM CrashPlan stopped, version 3.6.4, GUID 711641008093265921
      
      
   3. I 10/15/15 11:32AM CrashPlan started, version 3.6.4, GUID 711641008093265921
      
      
   4. etc…
5. The first upgrade removes “-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider “ from the run.conf so we need to update that
   1. vi /usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf
      
      
   2. SRV_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx2048m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false -Djava.net.preferIPv4Stack=true"GUI_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"
      
      
   3. /usr/local/share/crashplan/bin/CrashPlanEngine stop
      
      
   4. cpuset -l 0 /usr/local/share/crashplan/bin/CrashPlanEngine start
6. Repeat the last step ensuring that run.conf has the values highlighted in RED until all the upgrades are complete.
   
   
7. tail -f /usr/pbi/crashplan-amd64/share/crashplan/log/service.log.0
   
   
8. Download Crashplan 4.4.1 (or the version that matches what is now running on the server, as listed in /usr/pbi/crashplan-amd64/share/crashplan/log/history.log.0
   1. Install for all users
      
      
   2. Edit C:\ProgramData\CrashPlan\.ui_info and have it match /var/lib/crashplan/.ui_info from the server, except with the IP of the server filled in instead of 127.0.0.1
9. Update the CrashPlan instance to connect to 0.0.0.0
   1. vi /usr/pbi/crashplan-amd64/share/crashplan/conf/my.service.xml
      
      
   2. <serviceUIConfig>
      
      
   3. <serviceHost>0.0.0.0</serviceHost>
10. Ensure that the service will start on jail reboot
    1. vi /etc/rc.conf and add crashplan_enable=”YES”
11. Add Storage (do this in the Crashplan jail to mount the FreeNAS storage)
    1. mkdir /mnt/xxxx
       
       
    2. mkdir /mnt/xxxx2
       
       
    3. Then through the web-ui of the FreeNAS box add storage

 

[danb35]

Thanks for taking the time to write this up. Note that enabling SSH in the jail is optional--you can just ssh into the server itself, jls, jexec <JID number of your crashplan jail> tcsh.

 

[guglez]

Thanks for taking the time to write this up. Note that enabling SSH in the jail is optional--you can just ssh into the server itself, jls, jexec <JID number of your crashplan jail> tcsh.

Are you sure? Portforwarding will not work properly with out ssh in jail.

 

[danb35]

You're right--I hadn't noticed those steps in your writeup. The port forwarding isn't necessary either but that is the way Code42 tells you to run headless.

 

[joerawr]

For those not using port forwarding not much is different from the OP. I'm going to post my solution here, as it was similar, but I also pulled things from a few other solutions:

Troubleshooting:
to find the crashplan jail:
# jls
to connect to the jail, substitute 2 for your jails number:
# jexec 2 csh

From this point, all of the below is in the crashplan jail (jexec [jailnumber] csh).
Ports and IP:
# sockstat -l | grep java

Who's connected:
# netstat -an | grep 42

Pids and process:
# ps auwx | grep java
and/or
# ps auwx | grep -i crash

Crashplan version:
# cat /usr/pbi/crashplan-amd64/share/crashplan/log/history.log.0 | grep version

My System:
FreeNAS 9.2.1.7
I'm NOT using ssh port forwarding.
I was getting the bash errors.
I was getting the pkg: Unable to find catalogs errors when trying to upgrade bash.
I was already upgraded to 4.4.1 .


Upgrade pkg in order to upgrade Bash:
# fetch 'http://pkg.freebsd.org/freebsd:9:x86:64/latest/Latest/pkg.txz'
# pkg add -f pkg.txz

# pkg update
(Yay! It works now!)

# pkg install bash
This step installed many dependencies in my case. I said Yes to all, and all seems fine. I probably should have run a full "pkg upgrade".

Soft link bash, to where Crashplan 'thinks' it should be.
# ln -siv /usr/local/bin/bash /bin/bash

Verify your crashplan jail IP, and ports the cli way:
# sockstat -l | grep tcp4
root java 7759 102 tcp4 192.168.1.100:4268 *:*
root java 7758 89 tcp4 192.168.1.100:4269 *:*

So, 192.168.1.100, 4268 and 4269.

# service crashplan stop
and to be sure...
# pkill -9 java

Verify port numbers, and the 0.0.0.0s are in all the right places:

# cat /usr/pbi/crashplan-amd64/share/crashplan/conf/my.service.xml | grep 0.0.0.0
<location>0.0.0.0:4268</location>
<serviceHost>0.0.0.0</serviceHost>
Not sure if <location> is required, but it's working for me.

# cat /var/lib/crashplan/.ui_info ; echo
4269,e6767137-xxxx-xxxx-b64c-37cb512121ca,0.0.0.0

Update your windows .ui_info to match this, however, if you are NOT using port forwarding, in the Windows file replace the 0.0.0.0 with the ip of your crashplan jail, like this:
4269,e6767137-xxxx-xxxx-b64c-37cb512121ca,192.168.1.100

# egrep "Port|location" /usr/pbi/crashplan-amd64/share/crashplan/conf/my.service.xml
<location>0.0.0.0:4268</location>
<servicePort>4269</servicePort>

Triple check that run.conf has increased Xmx and the java.nio.channels.spi.SelectorProvider section:
# vi /usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf

To avoid out of memory errors, I'm using 3072m. More about this here: https://forums.freenas.org/index.php?threads/crashplan-4-4-1-outofmemoryerror.38704/ :

SRV_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx3072m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false -Djava.net.preferIPv4Stack=true"
GUI_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"

Make sure that pastes as two lines, meaning SRV_JAVA_OPTS= is one long line and GUI_JAVA_OPTS= is one long line.

Speed Fix:
http://networkrockstar.ca/2013/09/speeding-up-crashplan-backups/
vi /usr/pbi/crashplan-amd64/share/crashplan/conf/my.service.xml

<dataDeDupAutoMaxFileSize>1073741824</dataDeDupAutoMaxFileSize>
<dataDeDupAutoMaxFileSizeForWan>0</dataDeDupAutoMaxFileSizeForWan>
Change both to 1

And change these from 64k to 1M
<socketSendBufferSize>1048576</socketSendBufferSize>
<socketReceiveBufferSize>1048576</socketReceiveBufferSize>


# service crashplan start
( the cpuset -l 0 line is in my crashplan startup script already.)

re-verify the key hasn't changed:
# cat /var/lib/crashplan/.ui_info ; echo

If it matches your Windows .ui_info, wait a minute or so, then fire up the Crashplan client on your Windows PC. Hope that helps.

I'm finally maxing out my 10Mb upload!!

Note:
I repeatedly see this error in my logs, with no harm being noticed.:

[11.03.15 12:13:48.713 WARN 71_NIWatcher de42.net.NetworkInterfaceWatcher] SocketException getting network interfaces! java.net.SocketException: Cannot assign requested address, com.code42.exception.DebugException: SocketException getting network interfaces! java.net.SocketException: Cannot assign requested address
STACKTRACE:: com.code42.exception.DebugException: SocketException getting network interfaces! java.net.SocketException: Cannot assign requested address
at com.code42.net.NetworkInterfaceWatcher.populateAddresses(NetworkInterfaceWatcher.java:112)
at com.code42.net.NetworkInterfaceWatcher.doWork(NetworkInterfaceWatcher.java:136)
at com.code42.utils.AWorker.run(AWorker.java:148)
at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketException: Cannot assign requested address
at java.net.NetworkInterface.getAll(Native Method)
at java.net.NetworkInterface.getNetworkInterfaces(Unknown Source)
at com.code42.net.NetworkInterfaceWatcher.populateAddresses(NetworkInterfaceWatcher.java:97)
... 3 more