SOLVED CrashPlan Plugin Won't Start

[nello]

After making some changes to a working CrashPlan installation (details here), I wasn't able to start the CrashPlan Service so I decided to go back to square one.

I deleted the CrashPlan Plugin and re-installed it. Now when I try to start the service, I get the message:

Crashplan data did not validate, configure it first.

Oddly, I did NOT get a prompt to accept the terms and conditions on the re-install. And the Jail Storage was set up once the plugin was installed.

All of this seems to indicate that deleting the Plugin doesn't delete everything; something is left behind that makes the reinstall skip the acceptance and Storage setup.

How do I get the CrashPlan Plugin installed and started again?

 

[SweetAndLow]

The storage mounts reappears because you didn't remove it before deleting the jail. To completely start over i suggest removing the storage under the jails settings, then delete the plugin, then reboot and finally reinstall the plugin.

 

[nello]

To completely start over i suggest removing the storage under the jails settings, then delete the plugin, then reboot and finally reinstall the plugin.

As suggested, I:

1. Removed CrashPlan's Jail Storage;
2. Stopped the CrashPlan Jail;
3. Deleted the CrashPlan Plugin;
4. Rebooted FreeNAS server; and
5. Installed CrashPlan Plugin.

After booting up:

1. CrashPlan Jail Storage does NOT appear;
2. Terms of Service DO appear, and after accepting them;
3. CrashPlan Plugin Starts.

Thank you.

 

[nello]

I'm back to my old problem:

https://forums.freenas.org/index.php?threads/optimizing-crashplan.24108/#post-148666


The service appears to start:

But this error appears in the FreeNAS Console:

Oct 22 10:15:04 OctoberGroupNAS kernel: linux: pid 9086 (java): syscall epoll_create not implemented


And apparently the CrashPlan Plugin is NOT listening on port 4242:

root@crashplan_1:/ # sockstat -4
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root python2.7 4459 3 tcp4 10.10.49.11:12346 *:*
root sshd 4244 5 tcp4 *:22 *:*
root syslogd 4110 7 udp4 *:514 *:*
? ? ? ? tcp4 10.10.49.11:12346 10.10.49.10:18105
? ? ? ? tcp4 10.10.49.11:12346 10.10.49.10:21551
? ? ? ? tcp4 10.10.49.11:12346 10.10.49.10:19176
root@crashplan_1:/ #


I seem to be having this problem reported back in March, 2014:

https://forums.freenas.org/index.php?threads/crashplan-plugin.15366/page-5#post-105788

root@crashplan_1:/ # service crashplan status
/usr/local/etc/rc.d/crashplan: WARNING: no shebang line in /usr/bin/cpuset
crashplan is not running.
root@crashplan_1:/ #


I tried following the thread for a solution but didn't get very far:

root@crashplan_1:/ # /usr/bin/cpuset -l 0 /usr/local/share/crashplan/bin/CrashPlanEngine restart
Restarting CrashPlan Engine ... Stopping CrashPlan Engine ... /usr/pbi/crashplan-amd64/share/crashplan/bin/CrashPlanEngine: line 145
: kill: (16711) - No such process
OK
Starting CrashPlan Engine ... Using standard startup
OK
root@crashplan_1:/ #


crashPlan@crashplan_1:~ % ls /proc

14551145771457817104173274110424442484459curproc

crashPlan@crashplan_1:~ %



crashPlan@crashplan_1:~ % sockstat -4

USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS

crashPlan sshd 14577 5 tcp4 10.10.49.11:22 10.10.49.134:52648

root sshd 14551 5 tcp4 10.10.49.11:22 10.10.49.134:52648

root python2.7 4459 3 tcp4 10.10.49.11:12346 *:*

root sshd 4244 5 tcp4 *:22 *:*

root syslogd 4110 7 udp4 *:514 *:*

crashPlan@crashplan_1:~ %



Here's the CrashPlan log:

crashPlan@crashplan_1:~ % cat /var/log/crashplan/engine_output.log

Java HotSpot(TM) Client VM warning: Can't detect initial thread stack location - find_vma failed

[10.22.14 10:54:14.114 INFO main root ] Locale changed to English

[10.22.14 10:54:14.115 INFO main root ] *************************************************************

[10.22.14 10:54:14.115 INFO main root ] *************************************************************

[10.22.14 10:54:14.115 INFO main root ] STARTED CrashPlanService

[10.22.14 10:54:14.117 INFO main root ] CPVERSION = 3.6.4 - 1388642400364 (2014-01-02T06:00:00:364+0000)

[10.22.14 10:54:14.117 INFO main root ] LOCALE = English

[10.22.14 10:54:14.118 INFO main root ] ARGS = [ ]

[10.22.14 10:54:14.119 INFO main root ] *************************************************************

[10.22.14 10:54:14.268 INFO main root ] Adding shutdown hook.

[10.22.14 10:54:14.277 INFO main root ] BEGIN Loading Configuration

[10.22.14 10:54:14.332 INFO main root ] BEGIN Copy Custom

[10.22.14 10:54:14.333 INFO main root ] Directories: [.Custom, custom, /usr/pbi/crashplan-amd64/share/crashplan/conf/.Custom, /usr/pbi/crashplan-amd64/share/crashplan/conf/custom]

[10.22.14 10:54:14.333 INFO main root ] NOT waiting for custom skin to appear

[10.22.14 10:54:14.333 INFO main root ] NO customizations found.

[10.22.14 10:54:14.333 INFO main root ] END Copy Custom

[10.22.14 10:54:14.338 INFO main root ] Loading from default: /usr/pbi/crashplan-amd64/share/crashplan/conf/default.service.xml

[10.22.14 10:54:14.451 INFO main root ] Loading from my xml file=conf/my.service.xml

[10.22.14 10:54:14.528 INFO main root ] Loading ServiceConfig, newInstall=false, version=6, configDateMs=null, installVersion=1388556100363

[10.22.14 10:54:14.529 INFO main root ] OS = Linux

[10.22.14 10:54:14.794 INFO main root ] AuthorityLocation@5099378[ location=central.crashplan.com:443, hideAddress=false ]

[10.22.14 10:54:14.796 INFO main root ] Checking Java memory heap max.

[10.22.14 10:54:14.799 INFO main root ] Previous Java memory max heap size was 1024

[10.22.14 10:54:14.802 INFO main root ] END Loading Configuration

jtux Loaded.

Exiting!!! java.lang.RuntimeException: Unexpected IO Exception constructing selector engine - e=java.io.IOException: Function not implemented

[10.22.14 10:54:15.690 INFO main root ] ***** STOPPING *****

[10.22.14 10:54:15.691 INFO Thread-0 root ] Stopping service...

[10.22.14 10:54:15.706 INFO Thread-0 root ] DownloadManager stopped.

[10.22.14 10:54:15.709 INFO Thread-0 root ] SystemWatcher stopped.

[10.22.14 10:54:15.710 INFO Thread-0 root ] Service stopped

crashPlan@crashplan_1:~ %


And here is the conf file:

crashPlan@crashplan_1:~ % cat /usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf

SRV_JAVA_OPTS="-Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx1024m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"

GUI_JAVA_OPTS="-Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"crashPlan@crashplan_1:~ %

 

[mjws00]

I keep watching you swim up stream. We have a few crashplan threads keep floating up, due to something they have done recently. I'd be looking at the run.conf fix mentioned and likely java. Plugins move a little slow so I'm not sure if you're going to get stuck installing in a jail. Maybe a little more control and flexibility would be useful? There may be a larger pool of FreeBSD users encountering the same thing? Consider we are emulating linux and then running a java app on an unsupported platform, while trying to ball it up in a 1 click plugin. Lots of moving parts.

Wish I could offer you more than a good luck.

 

[nello]

I keep watching you swim up stream.

Thank you. As in real life, I'm not much of a swimmer and I'm really in over my head on this one. :)

I'd be looking at the run.conf fix mentioned ….

Yes, I found that thread too.

To me the interesting thing about that thread is that it's titled ‟CrashPlan 3.6.3” whereas my CrashPlan log above says that the Plugin is trying to run 3.6.4

[10.22.14 10:54:14.117 INFO main root ] CPVERSION = 3.6.4 - 1388642400364 (2014-01-02T06:00:00:364+0000)

In the CrashPlan 3.6.3 thread, there is a post that says that run.conf had changed and the problem appeared following a CrashPlan update. So, I'm thinking that somehow there's been upgrade from CrashPlan 3.6.3 to 3.6.4 and again the run.conf retrograded to an old, incorrect version.

 

[nello]

I updated this file:

/usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf

And added the following as documented in CrashPlan 3.6.3:

• SRV_JAVA_OPTS: Added as first parameter
  -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider
  
  
• GUI_JAVA_OPTS: Added as first parameter
  -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider

Then I rebooted.

CrashPlan Plugin now starts without errors.

 

[James Random]

Thank you. Thank you!

I just received an email from Crashplan saying that no backups had been made for 3 days. Checked the logs and found that it had been updated automatically and refused to start since then. Have been browsing the forums for a while to try and find the fix.

Adding those options in run.conf seems to have fixed it.

 

[joerawr]

I updated this file:

/usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf

And added the following as documented in CrashPlan 3.6.3:

• SRV_JAVA_OPTS: Added as first parameter
  -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider
  
  
• GUI_JAVA_OPTS: Added as first parameter
  -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider

Then I rebooted.

CrashPlan Plugin now starts without errors.

Thank Nello,

That was the fix for me too. same issue, no backups for 4 days, and the plugin wouldn't start.

I'll add for those having similar issue: Open the crashplan jail, click the shell button, and then you can navigate to the /usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf file.

 

[Derek Dysart]

I've updated the run.conf and it seems to be start, but if I tail the engine_output.log, it seems like the service is in a loop of starting, running for maybe 90 seconds, then stopping and starting. This would be consistent with the client app just exiting every so often.

Java HotSpot(TM) Client VM warning: Can't detect initial thread stack location - find_vma failed
[10.24.14 14:45:58.098 INFO main root ] Locale changed to English
[10.24.14 14:45:58.098 INFO main root ] *************************************************************
[10.24.14 14:45:58.098 INFO main root ] *************************************************************
[10.24.14 14:45:58.099 INFO main root ] STARTED CrashPlanService
[10.24.14 14:45:58.099 INFO main root ] CPVERSION = 3.6.4 - 1388642400364 (2014-01-02T06:00:00:364+0000)
[10.24.14 14:45:58.099 INFO main root ] LOCALE = English
[10.24.14 14:45:58.100 INFO main root ] ARGS = [ ]
[10.24.14 14:45:58.100 INFO main root ] *************************************************************
[10.24.14 14:45:58.175 INFO main root ] Adding shutdown hook.
[10.24.14 14:45:58.180 INFO main root ] BEGIN Loading Configuration
[10.24.14 14:45:58.208 INFO main root ] BEGIN Copy Custom
[10.24.14 14:45:58.208 INFO main root ] Directories: [.Custom, custom, /usr/pbi/crashplan-amd64/share/crashplan/conf/.Custom, /usr/pbi/crashplan-amd64/share/crashplan/conf/custom]
[10.24.14 14:45:58.208 INFO main root ] NOT waiting for custom skin to appear
[10.24.14 14:45:58.208 INFO main root ] NO customizations found.
[10.24.14 14:45:58.208 INFO main root ] END Copy Custom
[10.24.14 14:45:58.211 INFO main root ] Loading from default: /usr/pbi/crashplan-amd64/share/crashplan/conf/default.service.xml
[10.24.14 14:45:58.265 INFO main root ] Loading from my xml file=conf/my.service.xml
[10.24.14 14:45:58.346 INFO main root ] Loading ServiceConfig, newInstall=false, version=6, configDateMs=1413605356354, installVersion=1388556100363
[10.24.14 14:45:58.347 INFO main root ] OS = Linux
[10.24.14 14:45:58.525 INFO main root ] AuthorityLocation@7016932[ location=central.crashplan.com:443, hideAddress=false ]
[10.24.14 14:45:58.528 INFO main root ] Checking Java memory heap max.
[10.24.14 14:45:58.531 INFO main root ] Previous Java memory max heap size was 1024
[10.24.14 14:45:58.532 INFO main root ] END Loading Configuration
jtux Loaded.
[10.24.14 14:47:02.836 INFO Thread-0 root ] Stopping service...
[10.24.14 14:47:03.062 INFO Thread-0 root ] Selector shutting down...
[10.24.14 14:47:03.062 INFO Thread-0 root ] Selector shutting down...
[10.24.14 14:47:03.062 INFO Thread-0 root ] Selector shutting down...
[10.24.14 14:47:03.062 INFO Thread-0 root ] Selector shutting down...
[10.24.14 14:47:03.062 INFO Thread-0 root ] [Factory 25361446]stop() Stopping
[10.24.14 14:47:03.062 INFO Thread-0 root ] UISession stopped.
[10.24.14 14:47:03.063 INFO Thread-0 root ] PG::DefaultGroup Stopping RemotePeerList. Stopping workers and removing all peers. #peers=4, location=[616268919149166593@0.0.0.0:4242]
[10.24.14 14:47:03.063 INFO Sel-UI-A root ] SelectorEngine stopped.
[10.24.14 14:47:03.063 INFO Sel-UI-C root ] SelectorEngine stopped.
[10.24.14 14:47:03.063 INFO Sel-UI-R root ] SelectorEngine stopped.
[10.24.14 14:47:03.063 INFO Sel-UI-W root ] SelectorEngine stopped.
[10.24.14 14:47:03.272 INFO Thread-0 root ] PeerConnector closed.
[10.24.14 14:47:03.273 INFO Thread-0 root ] Selector shutting down...
[10.24.14 14:47:03.273 INFO Thread-0 root ] Selector shutting down...
[10.24.14 14:47:03.273 INFO Thread-0 root ] Selector shutting down...
[10.24.14 14:47:03.273 INFO Thread-0 root ] Selector shutting down...
[10.24.14 14:47:03.273 INFO Thread-0 root ] [Factory 1828625]stop() Stopping
[10.24.14 14:47:03.274 INFO W29927483_PeerTimeou root ] PeerConnector.TimeoutWorker stopped.
[10.24.14 14:47:03.274 INFO Sel-Peer-R root ] SelectorEngine stopped.
[10.24.14 14:47:03.274 INFO Sel-Peer-W root ] SelectorEngine stopped.
[10.24.14 14:47:03.274 INFO Sel-Peer-C root ] SelectorEngine stopped.
[10.24.14 14:47:03.275 INFO Sel-Peer-A root ] SelectorEngine stopped.
[10.24.14 14:47:03.382 INFO Thread-0 root ] PeerController stopped.
[10.24.14 14:47:04.595 INFO Thread-0 root ] Backup torn down.
[10.24.14 14:47:04.596 INFO Thread-0 root ] DownloadManager stopped.
[10.24.14 14:47:04.596 INFO Thread-0 root ] SystemWatcher stopped.
[10.24.14 14:47:04.596 INFO Thread-0 root ] Service stopped​

In the engine_error.log is see the following:

/etc/host.conf: line 2: bad command `hosts'
/etc/host.conf: line 3: bad command `dns'
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].
log4j:ERROR Attempted to append to closed appender named [null].​

The little big of digging I could do on this led me to a Twitter post from Crashplan that it's safe to ignore these errors. I was in the same boat as others and backups weren't running for quite some time, but the above run.conf fix seemed to work and they started back up. I'm now stuck with 4.6G left and it seems to be stalled again (or at least stuck in a loop of trying and failing).

 

[diskdiddler]

I just noticed my crashplan plugin has "died" after a recent stop and start (NO OTHER CHANGES) would I be right in thinking something has been patched or changed somewhere?

 

[Jase]

I also had this issue. Can confirm that following the instructions for 3.6.3 (joerwar post) worked fine and the service started :)

It's worth noting that you need to make the change outside the jail.

 

[grimer]

My crashplan stopped working 7 days ago. I didn't make any changes to the jail/plugin. Presumably it has automatically updated? I have opened a shell into the jail and I can navigate to the relevant folder containing run.conf.

Could somebody please post step-by-step instructions on how to edit this file to fix the problem?

Thanks,

Rob

 

[Matthew Davis]

I too am sitting at 7 days since a last good backup. I can connect to the instance via the headless display. And all I see is

I 10/27/14 09:53PM CrashPlan started, version 3.6.4, GUID xxxx
I 10/27/14 09:54PM Backup scheduled to always run

I won't even connect to a remote computer (on the local LAN) to do a backup.

Last edit: my apologies and sorry for the posting. I didn't realize I haven't made any changes to my backup content in 7 days. And the remote connection issue was a firewall issue on the remote server. Changes made earlier in the thread are working just fine for me with 3.6.4 code.

 

[SweetAndLow]

I would also like to add this has been fixed with the most recent version of the port. So whenever the PBI gets updated with the latest port this issue shouldn't happen with furutre Crashplan updates.

update port bug:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194465

Update PBI bug:
https://bugs.freenas.org/issues/6324

 

[shawndewet]

Yup, joerwar's post worked for me too (had to restart my crashplan jail though).

 

[Derek Dysart]

Maybe I'll start a new thread since my issue isn't that Crashplan won't start (it will thanks to the fix noted here), it just keeps restarting.

 

[Derek Dysart]

New year, and I finally dug into this. For the prosperity of anyone who stumbles across this later having pulled all your hair out, my issue was due to Crashplan crashing from out-of-memory conditions, as explained here. The fix was to increase the server memory to 2GB by changing the SRV_JAVA_OPTS parameter -Xmx1024m to -Xmx2048m

 

[nello]

… my issue was due to Crashplan crashing from out-of-memory conditions, as explained here.

Your link refers to changing the memory allocation on the client, not the server.

Derek, can you expand a bit on whether you are backing to or from FreeNAS? And, did you change the memory allocation to the FreeNAS plugin or to another computer?

Thank you.

- nello

 

[Derek Dysart]

Sorry, I should have been more clear. I am backing up from FreeNAS to CrashPlan Central. Once I added the extra parameter noted earlier in the thread, CrashPlan started working, but I could see it constantly restarting. Turned out after I fixed one issue, I ran into another.

I changed the -Xm in the SRV_JAVA_OPTS on FreeNas.