Crashplan plugin 3.6.3 service won't start after system reboot

[bcknapp]

Hi, I'm very new to the FreeNAS system and this is my first post in the community. The hardware details of my setup are listed below. I installed the Crashplan plugin following some of the instructions in these videos: ,
I edited /usr/pbi/crashplan-amd64/share/crashplan/conf/my.service.xml line 24 and changed <servicehost> to 0.0.0.0 to listen to any IP address. I then edited C:\Program Files\CrashPlan\conf\ui.properties to point the Crashplan client to point to the IP address of the jail in which crashplan is installed. Also, not sure if it matters but I also did the following command as recommended by the guide I followed: /usr/bin/cpuset -l 0 /usr/local/share/crashplan/bin/CrashPlanEngine restart. This configuration worked initially and I was able to sign in, choose the folders for backup (after adding jail storage points), and backup progress was working great... until my first reboot. After reboot, in the Plugins > Installed section, the crashplan Service status button is turned off and will not turn on when I click on it. I used the shell for the jail in the WebGUI and attempted to restart the crashplan service using "service crashplan stop && service crashplan start" but no luck. I am unable to connect via the Crashplan client on my PC. One thing I noticed is that when I run the command sockstat -l I get a very short list with no mention of Java when compared to running the command shortly after the successful installation.

Code:

USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root python2.7 5701 3 tcp4 192.168.1.2:12355 *:*
root cron 5473 4 dgram -> /var/run/logpriv
root syslogd 5405 4 dgram /var/run/log
root syslogd 5405 5 dgram /var/run/logpriv
root syslogd 5405 6 udp6 *:514 *:*
root syslogd 5405 7 udp4 *:514 *:*

I have deleted the plugin (apparently this deletes the jail as well) and have redone this entire process now 3 times, and everytime I reboot the freeNAS system, I get the exact same problem.

Being incredibly new to the community I have not been able to find anything that seems to relate to this problem either in these forums or on Google. If anyone could point me in the right direction, I would greatly appreciate it!

Brian

 

[dlavigne]

Does the service start if you type service crashplan onestart ? If not, post the error message.

 

[Spacemarine]

I just did a fresh install of Freenas 9.3 and the Crashplan 3.6.3 plugin. I followed the guide in the first video (thanks for that!) and everything works perfectly. Didn't even need the second video.

I also restarted the Freenas a few times and even installed some updates and restarted again, everything just works. No problems at all!

Although I can't help you directly, maybe you should attempt a fresh install of 9.3? Unfortunately you didn't write if you already did that....

 

[bcknapp]

dlavigne:

I tried your suggestion and this is what I got:

root@crashplan_1:/ # service crashplan onestart
/usr/local/etc/rc.d/crashplan: WARNING: no shebang line in /usr/bin/cpuset
CrashPlan is already running with pid 35259 35260 35261 35262 35263 35264 35265
35266 35267 35269 35270 35271 35272 35273 35274 35275 35276 35277 35278 35279 35
280 35281 35282 35283 35284 35285 35286 35287 35288 35289 35290 35291 35292 3529
3 35294 35295 37995

This is odd because the button is still red/off in the plugins section of the freenas web interface, and when I click on it it says "some error has occurred" at the top of the screen.


Spacemarine:

I just did a fresh install of Freenas (after some delay due to a failed and RMAd power supply!) and have set up no other plugins or much else besides setting my CIFS shares and creating datasets. I installed the Crashplan plugin, accepted the license agreement, and repeated the steps I described from my original post.

Having spent many hours looking through the freenas forums for solutions I have noticed many people finding a solution in adding some information to the run.conf file, specifically adding -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider to the beginning, but oddly, my run.conf file appears to already have these lines in them.



Notably, when I run: root@crashplan_1:/ # service crashplan status
I get

root@crashplan_1:/ # service crashplan status
/usr/local/etc/rc.d/crashplan: WARNING: no shebang line in /usr/bin/cpuset
crashplan is not running.

I was able to connect with the crashplan client from my desktop once, but upon closing the client and trying again to reconnect, it times out with no connection. Is it possible that my windows client (version 3.6.4) is somehow conflicting with the service 3.6.3 version?

Also, I have noticed some users saying that their service is automatically updating to 3.6.4, is there a way I can tell what version of the service is running?

One other thing to note, after a reboot of the freenas system the console prints out this information:

Jan 8 21:45:00 freenas manage.py: [freeadmin.navtree:545] Couldn't retrieve http://192.168.1.4/plugins/crashplan/1/_s/treemenu: timed out
Jan 8 21:46:34 freenas manage.py: [plugins.utils:92] Couldn't retrieve http://192.168.1.4/plugins/crashplan/1/_s/status: No JSON object could be decoded

the IP of my freenas jail is 192.168.1.2. Not sure if that is an issue.

Thanks for the responses, I appreciate the help!

Thank you,
Brian

 

[danb35]

I'm having the same problem. When I attempt to start the crashplan service in the jail, I get the following in the system log:

Code:

Jan 14 11:54:49 freenas2 linux: pid 98999 (java): syscall epoll_create not implemented

Searching on that error message, I found Bug #6324, which suggested editing /usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf to make sure it includes "-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider". I've done this without apparent effect.

In my case, I installed the crashplan plugin while running FreeNAS 9.3-20141231. It ran for maybe a week without difficulty, uploading ~ 1.8 TB to crashplan central. I rebooted the server to upgrade to 9.3-20150109 yesterday, and since then crashplan won't start.

 

[Derek Dysart]

I don't know if this will help, but I'll offer it. I did notice that the backup service upgraded itself again this week. My server version is now reporting version 3.7.0 (this is reported in the client, so since you can't get the service running, I get it's sort of hard to find it out). When this occured, my run.conf reverted back to one that didn't have "-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider" in it. It also changed the memory setting for the server back to use only 1G. This was right around the time I upgraded FreeNAS as well, but I think that's just coincidence. A friend of mine reported the same issue on his FreeNAS system.

In order to edit these files, you need to stop the jail, then edit the file from the host system (it will be somewhere like /mnt/<zpool name>/jails/crashplan_1/usr/pbi/crashplan-amd64/share/crashplan/bin). In my case, I also needed to expand the memory allocation of the process to prevent out of memory errors (this is for a known issue with Crashplan if backing up large amounts of data), so I changed the parameter "-Xmx1024m" to "-Xmx2048m". Restart the jail, then see if you're client connects.

If it helps, my run.conf currently is (my edits in bold):

SRV_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx2048m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -networkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"
GUI_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile.encoding=UTF-8 -Dapp=CrashPlanDesktop -DappBaseName=CrashPlan -Xms20m -Xmx512m -Djava.net.preferIPv4Stack=true -Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=false"​

 

[depasseg]

Sounds similar to this post: https://forums.freenas.org/index.php?threads/crashplan-plugin.15366/page-5#post-105945

 

[danb35]

Thanks for the suggestion. Launching the client does now indicate a version of 3.7.0, even though it says it can't connect to the backup engine. As you suggested, I stopped the jail and brought up the run.conf file in nano. The "-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider" parameter was absent, so I added that and changed the memory setting, restarted the jail, then did "jexec 5 csh" and ran sockstat -l. No change, still no java processes listening. The client still says it's unable to connect, too. However, I don't see the "epoll_create not implemented" error in the console when I try to start crashplan--I guess that may be an improvement.

Running the "cpuset" command noted above also didn't get it running.

 

[Derek Dysart]

Ugh, I've been there and understand the frustration - my install was dormant and crashing over and over again (see the memory setting above) until recently.

You might try looking in the service log to see if there are any errors that give more hints in there. In the jail, it's going to be /usr/pbi/crashplan-amd64/share/crashplan/log/service.log.0.

 

[danb35]

I hate it when I make typos. I had an extra m in the memory limit option which was preventing java from starting. I fixed that, did "service crashplan start", and it now seems to be running--"sockstat -l" shows the processes listening, and the client can connect. Thanks for the help!

 

[Pasquale61]

Mine stopped working all of a sudden too a couple of days ago. I have stayed up to date on version 9.3 and have not had any problems until this and have not changed anything.

I noticed something strange, and maybe I'm just not doing this right. I tried Derek's suggestion to add the items in bold above to my run.conf. I did this with the jail off as advised. The strange thing is that the memory setting goes back to -Xmx1024m after I start the jail, but the other two entries stay intact. And, for some reason it "appears" to not start, even though I think it really is running:

root@crashplan_1:/home/crashplan # service crashplan onestart
/usr/local/etc/rc.d/crashplan: WARNING: no shebang line in /usr/bin/cpuset
Starting CrashPlan Engine ... Using standard startup
OK
root@crashplan_1:/home/crashplan # service crashplan status
/usr/local/etc/rc.d/crashplan: WARNING: no shebang line in /usr/bin/cpuset
crashplan is not running.
root@crashplan_1:/home/crashplan

The FreeNAS GUI shows that CrashPlan is stopped, but when I connect with the headless Crashplan client, it says it is up, and my CrashPlan clients are backing up to it. (Although it seems to be synchronizing blocks with CrashPlan Central again...) I think this has something to do with the latest 3.7.0 update.

Any ideas on what I should do? Is there something I am supposed to do with the FreeNAS plugin when CrashPlan updates their client?

Thanks!

 

[Spacemarine]

@Pasquale61:

Mine also stopped working two days ago. Upon issuing the same commands inside the jail, I get the same output. So I guess I have the same problem.

Found any solution yet?

 

[Victor866]

Same issue here :(

 

[Hisma]

crashplan no longer working for me :((

this is obviously affecting a large group of users, so it is something common to the crashplan installation, not our individual systems.

 

[Pasquale61]

@Pasquale61:

Mine also stopped working two days ago. Upon issuing the same commands inside the jail, I get the same output. So I guess I have the same problem.

Found any solution yet?

No, I really have not found a real solution yet, but it is backing up even though the FreeNAS GUI says the plugin is stopped. I followed Derek's recommendations earlier in this thread (modifying run.conf) and it seems to have at least gotten my backups going again. There's obviously something still broken and I really don't know enough about FreeBSD, PBI's and jails to fix this permanently. I come from a Linux world and am still trying to figure out all the differences. I'm really bummed out about this because overall I really love my FreeNAS setup, but a working CrashPlan setup is critical for me, as I'm sure it is for others here.

I am seriously considering installing Linux (Ubuntu) in a VirtualBox jail and going that route for CrashPlan. This way I can upgrade the CrashPlan software myself when new versions are pushed out and I don't have to worry as much about other dependencies, like java versions, kernel versions, etc. At least I can find plenty of support to keep that configuration current. I'm not sure if this will open another can of worms, but at this point I'm not sure what choice I have other than to give this a try.

 

[Pasquale61]

Someone opened a bug ticket regarding this problem a few days ago: https://bugs.freenas.org/issues/7507

 

[berd1921]

This is from user
Toddebner on this thread https://forums.freenas.org/index.ph...ing-upgrade-then-no-longer-works-3-6-4.24227/
"
Went into crashplan jail, started shell. edited the run.conf file

edit /usr/pbi/crashplan-amd64/share/crashplan/bin/run.conf

Added -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider after the " and before the -Dfile...... in both the SRV_JAVA.. and GUI_JAVA... lines

SRV_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile............"
GUI_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile............"
"

Steps for me that got me back up and running:

#1 Download crashplan 3.7 desktop client from crashplan website then install.
#2 check initial setup and fix if necessary http://www.reddit.com/r/freenas/comments/20vbyd/crashplan_installation/
#3 edit run.conf with instructions above https://forums.freenas.org/index.ph...ing-upgrade-then-no-longer-works-3-6-4.24227/
#4 increase memory allocation to current max of 3gb

 

[Pasquale61]

@berd1921
The Java mods to the run.conf is all I had to do, but the problem for me is that the FreeNAS GUI didn't show that it was running, even after rebooting the entire FreeNAS server several times. However, after today's 9.3 update and reboot, the GUI now shows that it is up. Go figure... I didn't see anything in the release notes about this, so maybe CrashPlan just had to catch up on the resync or something.

 

[Hisma]

thanks guys.

This is exactly what I had to do when the plugin auto-updated to 3.6.3. The run.conf file seems to get reset after each update. Adding the java lines as have been stated above fixed it.

The only thing I did differently was increase the memory allocation as was suggested as well. I also updated the desktop client to 3.7. That was all I had to do. No need to update to 9.3, it still shows the plugin as up on 9.2.1.6.

 

[Spacemarine]

SRV_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile............"
GUI_JAVA_OPTS="-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Dfile............"

These lines are everything I changed and it totally did the trick for me. Funny thing: when I first edited run.conf and restarted crashplan, it didn't work and I couldn't see my changes in the file, so I thought I forgot to save. But looking at the history, I can see what went on: on 1/15/2015 crashplan updated itself from 3.6.3 to 3.6.4 and stopped working. Today when I tried to start crashplan again, it immediately updated itself to 3.7 and deleted my changes to run.conf and stopped working again. Now that I repeated the changes, it works perfectly again.