Should I run CrashPlan in a WM?

[MortenSJ]

Hi guys

I know there is a CrashPlan plugin available in FreeNAS, but I've not yet had any luck installing it properly and it also seems to be a very old version.

Would it be better and easier to make a VM with Ubuntu or Windows, and run it in there? Or is there a different alternative or method to backup up my files to a cloud service?

 

[zoomzoom]

CrashPlan doens't function properly in FreeNAS 9.10/11, so unless you're running 9.3, I'd recommend waiting until bug #23779 is addressed.

• You can create a standard jail and install the linux_base-f10 port, however it has known vulnerabilities from mid 2016 that will not be patched, by following this thread
  • I don't recommend this due to the unpatched vulnerabilities in linux_base-f10
  
  
• Updating the CrashPlan version in the plugin is easy enough to do, with this thread laying out how to do so.
  • Dismiss all the SSH steps, as they're not necessary if you have SSH setup on FreeNAS... simply issue jexec <jailname>

 

[MortenSJ]

I actually tried to follow that guide last time, but i just couldn't get it to work :/

I would be up for giving it another try, but is there any cons in terms of running Ubuntu w/ CrashPlan in a Virtual Machine?

 

[zoomzoom]

A VM is not the same as a jail, and depending on your hardware, is not something I personally would recommend for CrashPlan. A VM requires dedicated resources to be set aside for it, versus a jail which does not (i.e. you take away RAM from FreeNAS to run a VM, and since FreeNAS runs from RAM, unless you have 32GB+, I personally wouldn't go that route as it's likely the performance on the FreeNAS box will take a hit).

The bug report linked to earlier, which it appears you didn't read, will eventually be fixed, and likely sooner rather than later, so I personally have chosen to wait for that bug to be addressed. You could always research if there's another linux base you could use from the ports collection, however I haven't done so since I haven't had the time.

• You can also create your own jail template based upon linux, and not freebsd, by following the instructions set forth in the 9.10 manual for Managing Templates

 

[MortenSJ]

Okay. I might have to figure out the CrashPlan jail thing or wait for that bug to be fixed :) Thank you for your help!

 

[scrappy]

I have CrashPlan running in its own dedicated Linux VM on FreeNAS Corral, but future release FN11 or 9.3 should be a similar setup. If you have the system resources available, a VM works great for CrashPlan.

 

[zoomzoom]

I have CrashPlan running in its own dedicated Linux VM on FreeNAS Corral, but future release FN11 or 9.3 should be a similar setup. If you have the system resources available, a VM works great for CrashPlan.

There's no point to running Corral as it's dead in the water and development for it has ended for the time being... There's a thread in the announcements that explains why. It's recommended to downgrade to 9.10 or upgrade to 11-RC. The features in Corral will be implemented into FreeNAS 11, however it's unlikely development will ever begin again on Corral. It's been downgraded and classified as experimental and unstable.

Due to FreeNAS running from RAM, running a VM simply for CrashPlan is inefficient and will cause more problems than it solves. A VM requires a user to specify ahead of time resource consumption, regardless if the VM is using those resources or not.

• For example, say the FreeNAS box runs on 16GB of RAM and an 8 core 2.4gHz CPU.
  • If you allocate the VM 4GB of RAM, that drops the RAM available to the host[FreeNAS] to 12GB, regardless if the VM is using all 4GB or not.
  • The same goes for the CPU... Say you allocate 6gHz to the VM (2.4 x 8 = 19.2gHz), that leaves 13.2gHz for the host [FreeNAS].
• FreeNAS runs from RAM, not from a storage device, which is why it's always recommended to max out the RAM prior to making other changes for performance. To create a VM simply to run CrashPlan in is a waste of resources.
• See this post below from @danb35

As I've mentioned several times, the bug that's preventing CrashPlan from running on FreeNAS 9.10 & 11 will likely be fixed this week or next, as the bug is likely affecting other plugins as well since it appears to be due to broken symlinks and shebangs.

Sent from my Nexus 6 using Tapatalk

 

[SweetAndLow]

I run a Linux vm with crashplan on my desktop and it works great. My desktop almost never gets turned off.

Sent from my Nexus 5X using Tapatalk

 

[scrappy]

There's no point to running Coral as it's dead in the water and development for it has ended for the time being... There's a thread in the announcements that explains why. It's recommended to downgrade to 9.10 or upgrade to 11-RC. The features in Coral will be implemented into FreeNAS 11, however it's unlikely development will ever begin again on Coral. It's been downgraded and classified as experimental and unstable.

Due to FreeNAS running from RAM, running a VM simply for CrashPlan is inefficient and will cause more problems than it solves. A VM requires a user to specify ahead of time resource consumption, regardless if the VM is using those resources or not.

• For example, say the FreeNAS box runs on 16GB of RAM and an 8 core 2.4gHz CPU.
  • If you allocate the VM 4GB of RAM, that drops the RAM available to the host[FreeNAS] to 12GB, regardless if the VM is using all 4GB or not.
  • The same goes for the CPU... Say you allocate 6gHz to the VM (2.4 x 8 = 19.2gHz), that leaves 13.2gHz for the host [FreeNAS].
• FreeNAS runs from RAM, not from a storage device, which is why it's always recommended to max out the RAM prior to making other changes for performance. To create a VM simply to run CrashPlan in is a waste of resources.

As I've mentioned several times, the bug that's preventing CrashPlan from running on FreeNAS 9.10 & 11 will likely be fixed this week or next, as the bug is likely affecting other plugins as well since it appears to be due to broken symlinks and shebangs.

Sent from my Nexus 6 using Tapatalk

Yes, I am well aware (painfully so) of Corral being dead. I am waiting for FreeNAS 11 STABLE to release before moving forward. Even then, I will probably wait for a point upgrade before switching. In the meantime I am stuck with Corral. Everything you say makes sense to a point, but for certain users such as myself with plenty of system resources available (dual Xeon 24 core, 72GB RAM etc.) a VM running Crashplan will make a negligible performance impact to the host machine. Most hypervisors, even type 2 like Bhyve, are typically ~90-95% efficient vs bare metal and Crashplan running in a Jail still needs the same resources as Crashplan in a VM. If you allocate an appropriate amount of system resources for your VM for CP to run properly, the cost of using a VM under Bhyve is well worth it to me considering the issues I have read about from other users running Crashplan in an environment which it was never intended for (FreeBSD)

 

[zoomzoom]

@scrappy Great points =]

 

[MortenSJ]

Thanks everyone for your input and help! I'll wait till the update, and hopefully someone will be kind enough to make a comprehensive guide/tutorial.

Off Topic question. Would you guys recommend upgrading to RC 11

 

[zoomzoom]

@MortenSJ I just discovered there is a CrashPlan port in the ports collection @ /usr/ports/sysutils/linux-crashplan, which utilizes linuxbase-c6-6.8_12

In a Standard Jail:

1. Issue: portsnap fetch extract
   
2. PreReqs:
   • Download jdk-8u131-linux-i586.tar.gz on PC, transfer to Jail, and place it in /usr/ports/distfiles
   • Add the following to /etc/fstab
     
     Code:

     linprocfs   /compat/linux/proc	  linprocfs	   rw			  0	   0
     tmpfs	   /compat/linux/dev/shm   tmpfs		   rw,mode=1777	0	   0
     

3. Issue: cd /usr/ports/sysutils/linux-crashplan
4. Issue: make install clean

Beyond that, I'm not sure, so some research will need to be done on the FreeBSD forum or on Google.

 

[danb35]

• If you allocate the VM 4GB of RAM, that drops the RAM available to the host[FreeNAS] to 12GB, regardless if the VM is using all 4GB or not.

• The same goes for the CPU... Say you allocate 6gHz to the VM (2.4 x 8 = 19.2gHz), that leaves 13.2gHz for the host [FreeNAS].

Neither of these is correct. The VM doesn't hog memory or cores that the guest isn't using (which is also why it's perfectly valid to allocate more cores than are physically present in the machine). Even VirtualBox doesn't do this, and it's widely regarded as a pretty crappy hypervisor.

As to the subject of the thread, I've long since switched to running CrashPlan in a Ubuntu VM--originally under VirtualBox (using the late and lamented VirtualBox jail), and later under Proxmox on a separate host. That has CrashPlan running on an OS it's actually supposed to run on, and completely avoids the issues with the headless configuration that came up so frequently on FreeNAS (I just VNC to the VM). I'd expect it would work similarly well in a bhyve VM.

 

[zoomzoom]

@danb35 I didn't realize that, as I thought most distros/hypervisors didn't allow for dynamic allocation of resources. I use Hyper-V and ESXi and every distro I've ever installed has never allowed for dynamic allocation when its booted in a guest VM

• Ubuntu, Arch, OpenSUSE, Sophos UTM, TrueOS don't allow for dynamic allocation of RAM in Hyper-V, but do for CPU
• On ESXi, Ubuntu and Sophos UTM don't allow for dynamic allocation of either RAM

I've never tried VirtualBox or running a VM on FreeNAS and simply assumed the hypervisor would operate similarly.

• Do you by chance know what within a hypervisor determines whether it allows for dynamic allocation, as I just installed Ubuntu 17.04 in an ESXi, FreeNAS, and Hyper-V VM, and the only one dynamic allocation of RAM occurred in was within the FreeNAS VM.

 

[danb35]

@danb35 I didn't realize that, as I thought most distros/hypervisors didn't allow for dynamic allocation of resources.

Well, now you have me second-guessing myself. I remember seeing one of the devs here saying more or less what I stated, and I'm pretty sure it's consistent with what I've seen under VirtualBox (and under Proxmox, for that matter), but I don't know that I can point to chapter and verse. I may need to do some digging.

 

[toadman]

@danb35 I didn't realize that, as I thought most distros/hypervisors didn't allow for dynamic allocation of resources. I use Hyper-V and ESXi and every distro I've ever installed has never allowed for dynamic allocation when its booted in a guest VM

• Ubuntu, Arch, OpenSUSE, Sophos UTM, TrueOS don't allow for dynamic allocation of RAM in Hyper-V, but do for CPU
• On ESXi, Ubuntu and Sophos UTM don't allow for dynamic allocation of either
• On Free

I've never tried VirtualBox or running a VM on FreeNAS and simply assumed the hypervisor would operate similarly.

• Do you by chance know what within a hypervisor determines whether it allows for dynamic allocation, as I just installed Ubuntu 17.04 in an ESXi, FreeNAS, and Hyper-V VM, and the only one dynamic allocation of RAM occurred in was within the FreeNAS VM.

I'm curious why you are saying ESXi does not allow for dynamic allocation of cpu or memory. It will do both when over committed. Maybe we are using the terms slightly differently?

 

[zoomzoom]

I'm curious why you are saying ESXi does not allow for dynamic allocation of cpu or memory. It will do both when over committed. Maybe we are using the terms slightly differently?

After double checking, I was wrong about the CPU usage on ESXi, as that is dynamically allocated, but RAM is not (specific to the two VMs I have setup). I have Ubuntu 16.10 and Sophos UTM 9.4 installed on ESXi, and ESXi does not allow either to have dynamic allocation for RAM, however I've also never over committed the RAM on either. I'll have to test that and report back =]

• It could be I don't have a specific setting set for Ubuntu, but Sophos is the main VM that's always powered on in ESXi and as soon as Ubuntu boots, the 6GB of RAM I have assigned to it is auto reserved and allocated to Sophos, even though most days Sophos rarely goes above 33% utilization of it's 6GB.
  
  
• This is also how Hyper-V works, with dynamic allocation for CPU, but RAM is fully allocated as soon as the VM boots.
  • Specifically to Hyper-V, I do know Ubuntu does not support dynamic allocation of RAM at all, something I only learned a few months ago when I was looking for a specific PowerShell command and happened across a Hyper-V PowerShell wiki on how to setup a VM via cli.

 

[toadman]

Yea, VMs will get the memory they request as long as the host is not over committed. But it's not "reserved" unless the settings request a reservation. If memory needs to be reserved then there has to be enough non-reserved memory left on the host to allow the VM to power on or to migrate to the host. (On my hosts my main freenas vm and the backup freenas vm on a different host are the only VMs where I set a reservation. I reserve all of their requested memory so I know there is no way they will ever be memory starved and performance won't tank because the ARC is swapping or something.)

When host memory is over committed, the host will use resource shares and the size of the working set to dynamically determine what VM gets what memory. So a VM that has settings for 8GB that is only using 2GB will only get the 2GB (and maybe less if there is still memory contention, with the rest needing to swap).

 

[rwslippey]

Neither of these is correct. The VM doesn't hog memory or cores that the guest isn't using (which is also why it's perfectly valid to allocate more cores than are physically present in the machine). Even VirtualBox doesn't do this, and it's widely regarded as a pretty crappy hypervisor.

As to the subject of the thread, I've long since switched to running CrashPlan in a Ubuntu VM--originally under VirtualBox (using the late and lamented VirtualBox jail), and later under Proxmox on a separate host. That has CrashPlan running on an OS it's actually supposed to run on, and completely avoids the issues with the headless configuration that came up so frequently on FreeNAS (I just VNC to the VM). I'd expect it would work similarly well in a bhyve VM.

I know this is an older thread, but I had a question regarding the performance hit for your network. I have my Freenas box and a VMware server I use for other things sitting next to it. How badly does running the backup through your network effect performance of the overall network? Thinking maybe it's worth adding a dedicated NIC to prevent issues with network speed?
Given that crashplan home is done, I'm thinking of going to pro but I really don't want to deal with the headless configurations headaches anymore.

Also do you use CIFS or NFS?

 

[danb35]

How badly does running the backup through your network effect performance of the overall network?

Not in the least. I have a gigabit LAN, and only 20 Mbit upload speed--there's no way Crashplan is going to come anywhere close to saturating my LAN.

Also do you use CIFS or NFS?

NFS.