VM upload speed increase after a restart.

[lmannyr]

FN is X10SL7-F Xeon 32GB RAM 10 4TB Reds in RAIDZ2
ubuntu 16.04 VM 4GB RAM 20G drive.

Cable ISP 75/12 I regularly see greater than 500/400 (download/upload)

Crashplan is running in ubuntu.

Backing up for the first time 6TB.


Crashplan had been uploading all day at about 10Mbps. I restart the ubuntu VM, load up CrashPlan, and now its uploading at 600+Mbps. CrashPlan and FN reports the same speed.

This is repeatable.

Why does this happen? And is there anything I can do to let Crashplan increase its speed without having to restart the VM.

 

[c32767a]

So the Ubuntu image in the Jail is reading data from your FN volume and then uploading it to Crashplan via your internet connection?

And your Internet connection is 75 Mb/s down and 12Mb/s up?

And you're seeing a 600Mb/s upload?

If the units of measure are all megabits per second, the 600Mb/s upload seems unlikely to be real. Even if your cable modem is a DOCSIS 3.1 modem, it's highly unlikely a 75/12 tier would allow bursts, let alone sustained upload speeds at 600Mb/s. It's far more likely your cable provider is doing DOCSIS 3.0 or lower, and in that case it's more or less technically impossible to see real world upstream speeds of 600Mb/s.


It's far more likely that the 10Mb/s upload rate is the real rate at which you're sending data to Crashplan's servers.

I would bet that the 600Mb/s you are seeing is from Crashplan traversing all the files on your volume that have already been successfully backed up. I would speculate it hashes them and compares the hash to what's already in the cloud. When it hits files that have not yet been transferred, it will slow back down to your ISP's provisioned uplink and you will see the 10Mb/s speed you reported.

 

[SweetAndLow]

huh? you have a 12mb/s upload and you are seeing a 10mb/s upload rate from crashplan. Seems like you are getting exactly what you pay for.

 

[lmannyr]

So the Ubuntu image in the Jail is reading data from your FN volume and then uploading it to Crashplan via your internet connection?

YES.

And your Internet connection is 75 Mb/s down and 12Mb/s up?

YES.

And you're seeing a 600Mb/s upload?

YES, after a restart of the ubuntu VM.

If the units of measure are all megabits per second, the 600Mb/s upload seems unlikely to be real. Even if your cable modem is a DOCSIS 3.1 modem, it's highly unlikely a 75/12 tier would allow bursts, let alone sustained upload speeds at 600Mb/s. It's far more likely your cable provider is doing DOCSIS 3.0 or lower, and in that case it's more or less technically impossible to see real world upstream speeds of 600Mb/s.

Modem: Netgear CM600. DOCSIS3.0

It's far more likely that the 10Mb/s upload rate is the real rate at which you're sending data to Crashplan's servers.

I would bet that the 600Mb/s you are seeing is from Crashplan traversing all the files on your volume that have already been successfully backed up. I would speculate it hashes them and compares the hash to what's already in the cloud. When it hits files that have not yet been transferred, it will slow back down to your ISP's provisioned uplink and you will see the 10Mb/s speed you reported.

That kind of makes sense and what I thought too. Except, the "TX" on FreeNAS "Reporting/Network" tab shows these high upload speeds as well. Seems it burst up to 600-700 and will stay steady at 400ish for 6-8 hours at a time. Yesterday it stuck at 350-450 for 12+ hours. These numbers are coming from the FreeNAS "Reporting/Network" tab graphs. CrashPlan shows the same numbers though. If the FreeNAS "Reporting/Network" graphs did not show the high upload numbers when compared to CrashPlans Upload speeds I would agree 100%.

 

[lmannyr]

huh? you have a 12mb/s upload and you are seeing a 10mb/s upload rate from crashplan. Seems like you are getting exactly what you pay for.

NOPE. Reread the post.

 

[SweetAndLow]

NOPE. Reread the post.

Nope I'm right, what is the upload of the Ubuntu machine? The upload your seeing in the freenas graphs is worthless. It's telling you how much data the crashplan application is reading. And did you know that crashplan reads all your data so it can back it up? So actually your not even getting gigabit speeds.
Crashplan is just reading and checking for changes every time you reboot your machine it has to start over, lol. Just let it run and be happy you have a backup.

Sent from my Nexus 5X using Tapatalk

 

[lmannyr]

Nope I'm right, what is the upload of the Ubuntu machine?

I dont' know how to find that info on unbuntu. Will look around.

The upload your seeing in the freenas graphs is worthless. It's telling you how much data the crashplan application is reading.

Seriously?

And did you know that crashplan reads all your data so it can back it up? So actually your not even getting gigabit speeds.

I havn't seen any gigabit speeds.

Crashplan is just reading and checking for changes every time you reboot your machine it has to start over, lol. Just let it run and be happy you have a backup.

No doubt I'm happy to have an offsite backup. Cheap too! I see Crashplan "sync" after a restart. After the "sync" and it start to "backup" is when those crazy high speeds show for hours on end.

Thanks for the insights.

 

[c32767a]

YES.
That kind of makes sense and what I thought too. Except, the "TX" on FreeNAS "Reporting/Network" tab shows these high upload speeds as well. Seems it burst up to 600-700 and will stay steady at 400ish for 6-8 hours at a time. Yesterday it stuck at 350-450 for 12+ hours. These numbers are coming from the FreeNAS "Reporting/Network" tab graphs. CrashPlan shows the same numbers though. If the FreeNAS "Reporting/Network" graphs did not show the high upload numbers when compared to CrashPlans Upload speeds I would agree 100%.

I still think those high numbers are reporting artifacts. The cable companies don't usually make mistakes with things that involve money like bandwidth caps, so I think it's highly unlikely more than 12Mb/s can flow upstream from your network.

Does your router have any reporting capabilities? you might want to look there and see what it's telling you.

Also, do you have any other crashplan devices on your local network?

 

[lmannyr]

I still think those high numbers are reporting artifacts. The cable companies don't usually make mistakes with things that involve money like bandwidth caps, so I think it's highly unlikely more than 12Mb/s can flow upstream from your network.

Makes sense.

Does your router have any reporting capabilities? you might want to look there and see what it's telling you.

I have an Apple Airport Extreme which does not have any reporting. Im in the hardware planning stages of building a pfSense box though. Parts are in a shopping cart...just need to hit the Green button...

Also, do you have any other crashplan devices on your local network?

Nope.

 

[gpsguy]

If you run a speed test like: http://www.speedtest.net/ what's your upload speed?

I realize it's just a point in time, but at least it's an easy test you can perform.

 

[lmannyr]

If you run a speed test like: http://www.speedtest.net/ what's your upload speed?

I realize it's just a point in time, but at least it's an easy test you can perform.

I get anywhere from a fraction of advertised to 10% above advertised. I'll do a speedtest when it's supposedly running at way above advertised.

Does file size matter? I've noticed these high numbers on VERY large files (1tb images)

 

[c32767a]

I get anywhere from a fraction of advertised to 10% above advertised. I'll do a speedtest when it's supposedly running at way above advertised.

Does file size matter? I've noticed these high numbers on VERY large files (1tb images)

So if you have a large file, say 1tb, and an upload rate of say 10Mb/s, in round numbers, it's going to take a couple hundred hours to upload. ( if we ignore base 8 vs base 10) 10Mb/s is basically 1 megabyte per second, so you can send 3600 megabytes in an hour.

At 3.6 gigs per hour, a 1tb file should take 300ish hours to transfer.

Crashplan can handle restarting the upload of a file. What is likely happening is crashplan is restarting your 1tb file and the widget that shows transfer rate is improperly calculating transfer rate by dividing the total file size by the amount of the file that's been successfully uploaded in total, rather than the amount that's been sent in the current session. If you watch the counter, I bet the longer it runs, the more accurate it gets.