CIFS File Copy 75x Faster on Win vs OS X

[iamanthony]

When copying a file on my CIFS share from one directory to another, I get transfer speeds of 514.6 Mbps from my Windows 8.1 box, 95.34 Mbps from my Ubuntu 14.10 box, and 6.7 Mbps from my OSX 10.10 Yosemite box. Here's the commands I used to copy a 620 MB file from the folder source to destination with all three clients:

OS X 10.10 (copy duration of 1:20)

Code:

rsync -a --progress --stats --human-readable /Volumes/shared/source/ /Volumes/shared/destination/
sent 619.66M bytes  received 70 bytes  6.70M bytes/sec

Ubuntu 14.10 (copy duration of 12 seconds)

Code:

sudo rsync -a --progress --stats --human-readable /media/shared/source/ /media/shared/destination/
sent 619.73M bytes  received 38 bytes  45.91M bytes/sec

Windows 8.1 (duration of about 1 second)

Code:

T:\>robocopy T:\source T:\destination
Speed :  514604994 Bytes/sec. (514.6 Mbps)

Any ideal why the OSX client would be so much slower? Does anyone know what version of the SMB protocol OSX 10.10 uses?

 

[enemy85]

514 MB/s???? Do you have a 10gb connection between the 2 machines or what?

 

[iamanthony]

Just to be clear, I get 514 Mbps, not MBps. Because the file is being copied from the share to the share, I suspect that some facet of the SMB protocol is able to tell the server to copy the file without reading it to the local machine and writing it back to the server. I suspect that this data never actually traverses the network.

Some iSCSI SANS have a similar feature. I have two ESXi hosts, each connected to different LUNs on my SAN, but if I vMotion a VM from one LUN to another, the hosts are able to tell the SAN to move the file without reading & writing, so I get blindingly fast copies, several Gbps easy.

 

[SweetAndLow]

Try testing without rsync. Rsync is usually very cpu intensive and clients might be bottle necked. Just use cp -R or something similar.

 

[iamanthony]

Good idea, but its just about as slow with cp -r. I ran the following command from my OS X box and it took about 1:09 to complete. I was only using rsync to get a little more verbose output about the copy.

Code:

cp -r /Volumes/shared/source/ /Volumes/shared/destination/

 

[anodos]

Windows 8.1 is using server-side copy and so no data is passed through client.

As mentioned above, don't use rsync. For benchmarking you should probably try coping to/from client. Mount the samba share and use 'time cp -R foo'. This will actually time how long the copy operation takes. Clear your arc between tests. If large discrepancy persists, post your smb4.conf file as well as relevant entries from log.smbd.

 

[Mlovelace]

Good idea, but its just about as slow with cp -r. I ran the following command from my OS X box and it took about 1:09 to complete. I was only using rsync to get a little more verbose output about the copy.

Code:

cp -r /Volumes/shared/source/ /Volumes/shared/destination/

Robocopy is multi-threaded by default (8 threads to be exact) and will give you faster transfer speeds then a simple copy command. Use the basic cp command on the windows client as well if you are trying to compare.

 

[SweetAndLow]

1. Are all computers using gigabit lan connections?
2. Can you test network speed using iperf?
3. Try eliminating the local disk on the client by using dd if=/dev/zero of=/share/mount/location bs=4k count=10k

 

[cyberjock]

Robocopy is multi-threaded by default (8 threads to be exact) and will give you faster transfer speeds then a simple copy command. Use the basic cp command on the windows client as well if you are trying to compare.

Yes, except you create a whole new slew of problems. If you don't have the ability to run exactly 8 threads on 8 cores then you'll potentially have other problems with CPU contention and such. In general, its just a very poor way to measure performance.

My problem is with the Robocopy output...

Code:

T:\>robocopy T:\source T:\destination
Speed :  514604994 Bytes/sec. (514.6 Mbps)

Is it doing 514MB/sec or 514Mbps? The left says 514MB/sec (it says Bytes/sec, not bits/sec) and the right says Megabits/sec. So either someone is translating improperly or Robocopy is translating improperly. Just another reason why robocopy shouldn't be used. :p

 

[anodos]

Is it doing 514MB/sec or 514Mbps? The left says 514MB/sec (it says Bytes/sec, not bits/sec) and the right says Megabits/sec. So either someone is translating improperly or Robocopy is translating improperly. Just another reason why robocopy shouldn't be used. :p

Actually, for robocopy Mbps is million bytes per second because we all know that microsoft defines megabytes using base 2.