SOLVED CrashPlan 3.6.3

[mstinaff]

I am running FreeNAS 9.2.0 with the CrashPlan plugin 3.5.3_1. I launched a remote head to this engine today and it seems to have triggered an upgrade of the engine from 3.5.3 to 3.6.3 (released today I believe)

Now the engine fails to start. From the engine_output.log

Code:

[02.12.14 16:56:51.842 INFO    main                root                                    ] END Loading Configuration         
jtux Loaded.                                                                                                                     
Exiting!!! java.lang.RuntimeException: Unexpected IO Exception constructing selector engine - e=java.io.IOException: Function not implemented                                                                                   
[02.12.14 16:56:53.267 INFO    main                root                                    ] *****  STOPPING  ***** 

Any ideas? Other logs to check? If I re install the plugin is there any way to ensure that it doesn't try to upgrade itself again?

Many Thanks

 

[edvservice]

I have the same problem.... new Installation of Java 1.7.0.45 dont't work.

 

[mstinaff]

I think I have a solution, can someone else confirm?

edit /usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf

after the auto update to 3.6.3 it looked like this:

SRV_JAVA_OPTS="-Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"
GUI_JAVA_OPTS="-Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"

Add the bolded text to make it read like this (which matches the run.conf included in the plugin originally):

SRV_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"
GUI_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"

After I did this I was able to restart the engine and connect with a remote head. Backup is running again for me.


Update 2014-03-05:
As reported by David E you may need to stop the jail and edit the file from the FreeNAS shell.

Also I have tested this fix on a fresh plugin install that auto-updated itself last week. And the engine starts and runs fine. There is an error at the top of the engine_output.log though

Java HotSpot(TM) Client VM warning: Can't detect initial thread stack location - find_vma failed

But the only effect seems to be that running service crashplan status will always report that the engine is stopped, even if it is running. Some preliminary searching seems to indicate that Java is having and access issue with the /proc folder in the jail, but I haven't dug any deeper than that.

 

[mordysh]

Thanks.

I don't use the pbi (I have problem with hebrew files). I use the previous method from http://forums.freenas.org/index.php...an-in-freenas-9-1-jail-tested-on-32bit.14099/ and I had he same error. I tried your fix and now it's working again.

mordy.

 

[mstinaff]

Thank you for confirming mordy.

Now a forum noob question. How do I change the tag from problem to solved?

 

[Klox]

Is there a way to force the upgrade from 3.5.3 to 3.6.3? Mine hasn't performed the upgrade yet and now I'm getting a prompt for an "Archive Password Required" even though I already have one set. When I enter in a new one I get "CrashPlan has been disconnected from the backup engine." and the UI terminates.

 

[mstinaff]

what is the version of the CrashPlan client you are using as the gui? I was connecting with a 3.5.3 client when my plugin auto updated. I have since installed a fresh 3.5.3_1 PBI to try reproducing/testing the fix (and the jail I finally found the fix in had a lot of other trial/error cruft I wanted to leave behind.)

So now I am connecting my 3.6.3 client as the head to the 3.5.3 plugin/engine and I am seeing the same prompt as you, except I didn't set an archive password before so the prompt just tells me I have to create one. I tried creating one and nothing changed, prompt still comes up and backup keeps running.

I may try uninstalling the 3.6.3 client from the head machine and reinstalling the old 3.5.3 for the gui (available here) and see if connecting with that triggers it.

 

[hobodave]

I was able to resolve these problems by manually upgrading the CrashPlan used by the plugin to version 3.6.3.

 

[TheLostBryan]

I am still having an issue with CrashPlan. In troubleshooting the issue. I installed a fresh jail of 3.5.3_1 PBI. Tail was running on /var/log/crashplan/engine_error.log and /var/log/crashplan/engine_output.log and I connected with a 3.5.3 remote head. I was greeted with the account login prompt. After entering my credentials, the remote head showed a message about upgrading. The crashplan service restarted, and failed to load.

engine_output.log

Java HotSpot(TM) Client VM warning: Can't detect initial thread stack location - find_vma failed
[02.27.14 10:14:21.133 INFO main root ] Locale changed to English
[02.27.14 10:14:21.135 INFO main root ] *************************************************************
[02.27.14 10:14:21.135 INFO main root ] *************************************************************
[02.27.14 10:14:21.135 INFO main root ] STARTED CrashPlanService
[02.27.14 10:14:21.137 INFO main root ] CPVERSION = 3.6.3 - 1388556000363 (2014-01-01T06:00:00:363+0000)
[02.27.14 10:14:21.137 INFO main root ] LOCALE = English
[02.27.14 10:14:21.138 INFO main root ] ARGS = [ ]
[02.27.14 10:14:21.138 INFO main root ] *************************************************************
[02.27.14 10:14:21.313 INFO main root ] Adding shutdown hook.
[02.27.14 10:14:21.325 INFO main root ] BEGIN Loading Configuration
[02.27.14 10:14:21.391 INFO main root ] BEGIN Copy Custom
[02.27.14 10:14:21.393 INFO main root ] Directories: [.Custom, custom, /usr/pbi/crashplan-amd64/share/crashplan/conf/.Custom, /usr/pbi/crashplan-amd64/share/crashplan/conf/custom]
[02.27.14 10:14:21.394 INFO main root ] NOT waiting for custom skin to appear
[02.27.14 10:14:21.394 INFO main root ] NO customizations found.
[02.27.14 10:14:21.394 INFO main root ] END Copy Custom
[02.27.14 10:14:21.422 INFO main root ] Loading from default: /usr/pbi/crashplan-amd64/share/crashplan/conf/default.service.xml
[02.27.14 10:14:21.550 INFO main root ] Loading from my xml file=conf/my.service.xml
[02.27.14 10:14:21.602 INFO main root ] Loading ServiceConfig, newInstall=true, version=4, configDateMs=null, installVersion=1364274000353
[02.27.14 10:14:21.608 INFO main root ] OS = Linux
[02.27.14 10:14:21.712 INFO main root ] AuthorityLocation@10050169[ location=central.crashplan.com:443, hideAddress=false ]
[02.27.14 10:14:21.717 INFO main root ] Checking Java memory heap max.
[02.27.14 10:14:21.720 INFO main root ] Previous Java memory max heap size was 1024
[02.27.14 10:14:21.723 INFO main root ] END Loading Configuration
jtux Loaded.
Exiting!!! java.lang.RuntimeException: Unexpected IO Exception constructing selector engine - e=java.io.IOException: Function not implemented
[02.27.14 10:14:22.619 INFO main root ] ***** STOPPING *****
[02.27.14 10:14:22.620 INFO Thread-0 root ] Stopping service...
[02.27.14 10:14:22.636 INFO Thread-0 root ] DownloadManager stopped.
[02.27.14 10:14:22.640 INFO Thread-0 root ] SystemWatcher stopped.
[02.27.14 10:14:22.640 INFO Thread-0 root ] Service stopped

engine_error.log

java.lang.RuntimeException: Unexpected IO Exception constructing selector engine - e=java.io.IOException: Function not implemented
at com.code42.nio.SelectorEngine.<init>(SelectorEngine.java:75)
at com.code42.nio.net.MultiSelectorEngine.<init>(MultiSelectorEngine.java:49)
at com.code42.nio.net.Factory.<init>(Factory.java:100)
at com.code42.messaging.nio.MessageProvider.<init>(MessageProvider.java:89)
at com.backup42.service.CPService.initPeerControllers(CPService.java:687)
at com.backup42.service.CPService.start(CPService.java:452)
at com.backup42.service.CPService.main(CPService.java:1848)
Caused by: java.io.IOException: Function not implemented
at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
at sun.nio.ch.EPollArrayWrapper.<init>(Unknown Source)
at sun.nio.ch.EPollSelectorImpl.<init>(Unknown Source)
at sun.nio.ch.EPollSelectorProvider.openSelector(Unknown Source)
at java.nio.channels.Selector.open(Unknown Source)
at com.code42.nio.SelectorEngine.<init>(SelectorEngine.java:73)
... 6 more
I'm not sure where to go from here.

 

[Darren David]

I am still having an issue with CrashPlan. In troubleshooting the issue. I installed a fresh jail of 3.5.3_1 PBI. Tail was running on /var/log/crashplan/engine_error.log and /var/log/crashplan/engine_output.log and I connected with a 3.5.3 remote head. I was greeted with the account login prompt. After entering my credentials, the remote head showed a message about upgrading. The crashplan service restarted, and failed to load.

+1 here. I'm on FreeBSD 10.0 but having the exact same issue, and the previously listed fix didn't help. 3.5.3_1 works fine, and fails after upgrade to 3.6.3. I'm assuming any fix is applicable across both platforms...

 

[TheLostBryan]

I neglected to mention that in my post. I did try mstinaff's fix, with no success. I am working through the manual install process linked above, but cannot run Java 1.7 after completing installation. Getting "error while loading shared libraries: libjli.so: cannot open shared object file: No such file or directory".

The plugin updated itself into oblivion before the initial backup had completed, so I am making an alternate 3rd backup before proceeding further. As the data backup saying goes, "If it's not in three places, it doesn't exist".

 

[Darren David]

+1 here. I'm on FreeBSD 10.0 but having the exact same issue, and the previously listed fix didn't help. 3.5.3_1 works fine, and fails after upgrade to 3.6.3. I'm assuming any fix is applicable across both platforms...

Fixed for me - I used the fix posted above (essentially copying the run.conf from v3.5.3_1), but I had to increase the max heap size from "-Xmx512m" to "-Xmx2048m" in SRV_JAVA_OPTS because my backup set was so large. I read somewhere that you should have around the same max heap size as your backup set (although I found that at 4096m it wouldn't start at all). Worth experimenting with smaller increases in max heap size to find what works for you.

Hope that helps someone else...

Darren

 

[kavefish]

Hi all,

I started noticing problems with CrashPlan tonight, most likely after the head machine tried to update the server inside the plugin jail. I implemented mstinaff's solution and CrashPlan is stable again. However I am now seeing the following text show up in my console every couple of minutes:

Code:

Feb 28 00:06:35 nas kernel: linux: pid 37580 (java): syscall inotify_init not implemented

It doesn't seem to be affecting the functionality of CrashPlan.

Thanks to all for posting the troubleshooting and solution! I was pulling my hair out for a couple of hours before I saw this thread.

Edit: it looks like the inotify_init is a warning because the real-time backup feature isn't supported yet.

 

[Seany]

I think I have a solution, can someone else confirm?

My install also attempted an upgrade, but this fix did not work for me.

 

[Karnage]

The fix worked for me.

Couldnt edit the file from within the jail though; it kept reverting to what it already was. Had to edit via SSH.

 

[David E]

Worked for me too, but I had weird problems with it not sticking. I think you want to shut down the jail, edit the file from the nas machine, then start it up. Thanks mstinaff!

 

[shawndewet]

I found I had to stop and restart the jail (after applying the run.conf fix) for the crashplan plugin to remain in a started state.
However, now when I try to PuTTY in to the jail in order to do the ssh tunnel from 4200 to 4243 I get a error about the connection timing out. ai jai jai. now what?

 

[David E]

Your ssh connection? Check if sshd is running.

 

[Vipthomps]

you are all genius.

Worked for me plugin was 3.6.3 oddly desktop was 3.5 ...might have been the MacBook push the upgrade... anyway i digress. I stopped the jail, applied the update to run.conf, and was able to start the service and connect the GUI after i updated that to 3.6.3. Service wouldn't stay started so I also needed to restart the jail as mentioned above. Then the backup started right away. Phew! start to finish only about 30 minutes thanks to you all :D

 

[shawndewet]

Your ssh connection? Check if sshd is running.

Yup, all services started and processes running in the crashplan jail. Even rebooted the server (due to a power failure). Still connection timeout when trying to PuTTY in to the crashplan jail. Perhaps I'll start afresh (remove jail and crashplan and try again) when I have time this weekend.