Optimizing CrashPlan

Status
Not open for further replies.

nello

Patron
Joined
Dec 30, 2012
Messages
351

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
I'm fairly certain that article is only referring to the server sending the data. Since crashplan does dedup locally then sends it over the network it has to hold that dedup data in memory which requires higher memory footprint with more data. Modify the receiving crashplan server probably won't make a difference. What issues are you having? The only thing I can think of is speed, time for dedup and compression time. These are all going to be modified on the client side(Sender).
 

nello

Patron
Joined
Dec 30, 2012
Messages
351
I'm fairly certain that article is only referring to the server sending the data.… Modify the receiving crashplan server probably won't make a difference.

Yes, that's what I suspected. But, it was the only tip I found that might be relevant to my archives.


What issues are you having? The only thing I can think of is speed, time for dedup and compression time. These are all going to be modified on the client side(Sender).

I'm not aware of any problems per se but it seems slow.

My initial backup of 2.5TB on iMac over a (wired) gigabit LAN is hasn't completed after four days despite configuring my iMac to never sleep. I think part of the problem is that CrashPlan doesn't run continuously. During the evaluation period I think it runs every 15 minutes. But once the evaluation period has expired, the free version runs only once a day by default. After backing up some large chunk, it seems to think it's done its work for the day. I keep kicking it off manually and it does some more work. Now, after four days it still hasn't finished the initial 2.5TB backup.

Maybe there's some interference because I'm continuing to backup to my old destination at the same time as using the Plugin. My old backup configuration mounts FreeNAS as a AFP share point.

Running CrashPlan on my iMac and writing to FreeNAS as a mounted AFP drive seems to run a lot faster than running CrashPlan as a Plugin on my FreeNAS box.
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
Even with the free version once you kick off a backup it will run continuously untill all the files are backed up. The time limit is just how often it will check for changes and if it finds changes it will re-run the backup. Now as far as the issues you are seeing where crashplan doesn't run continuously, crashplan will appear to back up in stages. This is because it does all the duplication and compression work before sending it over the wire. It doesn't send and dedup in parallel. So it will do some computation on your data on the client(Mac) and then send that data to freenas. If you are just watching network activity you will see the send portion of the work then when network activity dies what it is doing is all computation on your Mac computer before it sends the data again. Check out your system monitor and there should be a java process sucking up ~100% cpu on your mac if it is still running properly.

What you gain from crashplan that you don't get with AFP backup is deduplication(space efficiency), automated backup, encryption, compression, version control. Just using freeNAS features you can get compression and version's but it's not in a nice little package like crashplan provides. If you are looking to just backup a single computer to your FreeNAS maybe crashplan is a little to complicated and I say with the idea that there could be a much simpler way to do it if you don't need all the crashplan features. Since you have a Mac did you evaluate just using time machine over AFP? I think this works on freenas, I have seen people complaining but I don't own a mac so haven't paid much attention to it.
 

nello

Patron
Joined
Dec 30, 2012
Messages
351
What you gain from crashplan that you don't get with AFP backup is deduplication(space efficiency), automated backup, encryption, compression, version control. Just using freeNAS features you can get compression and version's but it's not in a nice little package like crashplan provides.

Guess I wasn't clear.

I ran CrashPlan on the iMac. The CrashPlan Destination was FreeNAS mounted as an AFP Share.


If you are looking to just backup a single computer to your FreeNAS maybe crashplan is a little to complicated and I say with the idea that there could be a much simpler way to do it if you don't need all the crashplan features. Since you have a Mac did you evaluate just using time machine over AFP? I think this works on freenas, I have seen people complaining but I don't own a mac so haven't paid much attention to it.

I had two goals.

First, I wanted to backup multiple computers, my own and extended family members.

As I recall, when first did this, FreeNAS allowed only one AFP to be designated as a Time Machine (TM) destination. And, as I recall, I needed to have a separate FreeNAS user for each computer, otherwise the TM files would become commingled. In other words, if I someone in the family decided they no longer wanted to use my service, I couldn't delete their files unless they had their own username. There was also a problem with quotas. TM doesn't support quotas so unless each person has their own FreeNAS quota-enabled Dataset, they could end up hogging space from other users. It was a long time ago, and I may not have the specifics correct. Nevertheless, I managed to cobble together a Rube Goldberg configuration that seemed scalable and maintainable (for my needs).

Then came the testing and I found that it just wasn't stable. Every few weeks TM reported that it could no longer work with the existing dataset and had to start over with a new initial backup, discarding all the history of incremental changes until then. The third time this happened, I decided that I just didn't trust TM with my backups.

My second goal was to support remote backups.

Several people told me that opening my network firewall to AFP traffic for TM was a bad idea. This was the final nail in TM's coffin. I mean think about it. What good is an automated backup unless it works automatically as you travel? Yes, I could have fiddled with a VPN or SSH connection to my network and run TM over a secure connection. But that's not exactly automatic. Not to mention, I was trying to provide a service for my mother. I really didn't want to mess with VPN or SSH on her computer.

The bottom line is that TM is a single-user solution for people who are willing to take their backup drives with them as they travel. Kind of dumb to travel with both your original and backup disks. But, it's dead simple. And it's better than nothing. Best of all, the Apple Store supports it and they'll help you with the restore. Several extended family members continue to use TM instead of my CrashPlan service on FreeNAS.
 

nello

Patron
Joined
Dec 30, 2012
Messages
351
I'm fairly certain that article is only referring to the server sending the data. Since crashplan does dedup locally then sends it over the network it has to hold that dedup data in memory which requires higher memory footprint with more data. Modify the receiving crashplan server probably won't make a difference.

Code42 Confirms that adjusting the Java memory size is relevant only on the client side, not the FreeNAS Plugin side:

MikeW (CODE42 Support)

Oct 16 11:38 AM

Hello Nello,

Since CrashPlan does all of the heavy lifting on the client side, the memory requirements would need to be adjusted on the clients.

MikeW
Tier 1 Customer Champion
Code42 Support
 

nello

Patron
Joined
Dec 30, 2012
Messages
351
Summary

I'm looking for specific instructions on using Rsync to copy CrashPlan archives from one Dataset to another.


Background

As I mentioned above, I'm moving from using:
  1. CrashPlan using FreeNAS as a Destination mounted as an AFP Share, to
  2. CrashPlan Plugin running in a FreeNAS Jail.
I'm pretty cautious, so I created separate Datasets for (1) and (2), above.

Now that I've configured (2), and made sure that it's working, it's time for me to copy the archive from the Dataset I used in (1) to the one I have for (2).

According to CrashPlan support, this can be done with Rsync:

https://support.code42.com/CrashPla...ing_A_Backup_Archive_To_A_New_Folder_Or_Drive


Questions

I've never used Rsync.
  • Do I need to turn on the Rsync service? In the Jail?
  • Do I need to do this at a command line? In the Jail?
  • Is there a GUI interface for what I need to do?
Sorry to be such a Noob. Thank you for your indulgence.

- nello
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
No gui interface is available. I would mount the second dataset in the jail then use rsync in the jail to move the data from one mount point to the other. You don't need the rsync service for this.
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
If you are just doing this for testing, why not create a ZFS snapshot, then clone that snapshot, and then mount the clone like sweetnlow mentioned above. This way the copy would take a couple seconds and you would have your data. If the test was successful, then change the mount point for the CP jail and restart CP.
 

nello

Patron
Joined
Dec 30, 2012
Messages
351
… create a ZFS snapshot, then clone that snapshot…. This way the copy would take a couple seconds and you would have your data. If the test was successful, then change the mount point for the CP jail and restart CP.

Brilliant!

Now, I'm trying to do what you suggest within the GUI.

I have a single ZFS Volume, /mnt/red, shown below:

OctoberGroupNAS_-_FreeNAS-9_2_1_5-RELEASE-x64__80c1d35_.png

  • crashPlanBackups
    The old CrashPlan Destination built up over the last year using FreeNAS as a AFP Share

  • crashPlanPlugin
    The new CrashPlan Destination created in the past few weeks while testing the CrashPlan Plugin.
What I want to do to crashPlanBackups is
  • ZFS Snapshot
    OctoberGroupNAS ZFS Snapshot.png



  • Clone
    OctoberGroupNAS  Clone.png

Which yields a third Dataset, crashPlanOctoberGroupNAS
OctoberGroupNAS zfs volumes.png


And now, update the CrashPlan Plugin Jail to point to the cloned Dataset:
OctoberGroupNAS Update Jail.png
 
Last edited:

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
After you take the snapshot, go to the ZFS Snapshots and select the snapshot you just made, then click "Clone" icon on the right side of the row. Name the Clone.

upload_2014-10-20_17-53-3.png





Now you can test the clone.d volume When done, destroy the cloned volume.


upload_2014-10-20_17-56-5.png
 

nello

Patron
Joined
Dec 30, 2012
Messages
351
I can't get the CrashPlan Plugin to start. Yes, I've tried going back to the old Jail Storage settings as well as rebooting FreeNAS.

In all the configurations I've tried, the Jail reports that CrashPlan is running:
Jail.png



But the Plugins report that the CrashPlan is off. Even if I turn it on, once I click on another FreeNAS window, e.g., Jails again (above), this Plugin shows the service off when I return to it.
Plugins.png


Why are the Jails and Plugins reporting a different status for CrashPlan?

How to I get CrashPlan to run again?!?
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
I don't know exactly, but it sounds like the jail might be stuck, so the plugin can't start another instance. I'd look for ways to restart (or kill) hung up jails and then try to start the plugin.
 

nello

Patron
Joined
Dec 30, 2012
Messages
351
I decided to go back to square one. I deleted the CrashPlan Plugin and re-installed it.

Now when I try to start the service, I get the message:

Crashplan data did not validate, configure it first.

Oddly, I did NOT get a prompt to accept the terms and conditions on the re-install. And the Jail Storage was set up once the plugin was installed.

All of this seems to indicate that deleting the Plugin doesn't delete everything; something is left behind that makes the reinstall skip the acceptance and Storage setup.

Apparently, this is the code that generates the error:

https://bugs.freenas.org/projects/f...shplan/resources/crashplanUI/freenas/views.py
 
Last edited:

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
Try deleting the Plugin and then deleting the Jail (from the shell type jls (to get a listing) and then jail -r Num (where Num is the ID number of the Crashplan jail). Then try installing the plugin.
 

nello

Patron
Joined
Dec 30, 2012
Messages
351
Try deleting the Plugin and then deleting the Jail (from the shell type jls (to get a listing) and then jail -r Num (where Num is the ID number of the Crashplan jail). Then try installing the plugin.

Sorry, after deleting the CrashPlan Plugin, I don't see any evidence of a Jail

jls.png


No Entry Found.png
\
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
Reboot?
 

nello

Patron
Joined
Dec 30, 2012
Messages
351

nello

Patron
Joined
Dec 30, 2012
Messages
351
Last edited:
Status
Not open for further replies.
Top