Slow moving data to another folder in the same dataset

[id0827502]

Hi guys,

I've got some data that I need to move from one folder to another on the same dataset. When I go to do that however, it goes incredibly slow (1.2MB/s) which...kinda surprises me.

I'm of the impression that if you're moving data in the same zpool but DIFFERENT datasets then you will experience delays, however if the data is in the SAME dataset then it should perform as if you're just moving the data locally.

I've done some basic checks and everything seems to be in order.

Quick dd test:

[root@rammus] /mnt/rammus_shell/media# dd if=/dev/zero of=tmp.dat bs=2048k count=50k
51200+0 records in
51200+0 records out
107374182400 bytes transferred in 88.374571 secs (1214989576 bytes/sec)

I've also confirmed that it's the same dataset I'm working with (both the source and destination folder is in the media dataset):

[root@rammus] ~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rammus_shell 710G 6.43T 268K /mnt/rammus_shell
rammus_shell/.system 5.60M 6.43T 268K /mnt/rammus_shell/.system
rammus_shell/.system/cores 358K 6.43T 230K /mnt/rammus_shell/.system/cores
rammus_shell/.system/samba4 709K 6.43T 556K /mnt/rammus_shell/.system/samba4
rammus_shell/.system/syslog 4.29M 6.43T 882K /mnt/rammus_shell/.system/syslog
rammus_shell/jails 8.03G 6.43T 575K /mnt/rammus_shell/jails
rammus_shell/jails/.warden-template-pluginjail 807M 6.43T 807M /mnt/rammus_shell/jails/.warden-template-pluginjail
rammus_shell/jails/.warden-template-portjail 2.17G 6.43T 2.17G /mnt/rammus_shell/jails/.warden-template-portjail
rammus_shell/jails/download_jail 4.62G 6.43T 4.63G /mnt/rammus_shell/jails/download_jail
rammus_shell/jails/minecraft 412M 6.43T 2.47G /mnt/rammus_shell/jails/minecraft
rammus_shell/media 702G 6.43T 698G /mnt/rammus_shell/media

This is using a simple cp command, and also over FTP.

I'd be very appreciative to be pointed in the right direction and/or given some things to try.

 

[cyberjock]

Your dd test is invalid. You have compression enabled.

 

[id0827502]

Your dd test is invalid. You have compression enabled.

Is there a way I can do a dd test with compression enabled and get accurate results or do I have to turn it off?

 

[cyberjock]

You have to turn it off.

 

[id0827502]

You have to turn it off.

OK, I've turned it off and I get a much lower, but probably far more accurate result:

[root@rammus] /mnt/rammus_shell/media# dd if=/dev/zero of=tmp.dat bs=2048k count=50k
51200+0 records in
51200+0 records out
107374182400 bytes transferred in 317.480002 secs (338207703 bytes/sec)

This still seems normal to me. Any suggestions?

 

[panz]

Maybe I didn't understand the whole operation. When I want to MOVE data I use cut & paste with Samba: the move task is immediate :)

 

[id0827502]

Maybe I didn't understand the whole operation. When I want to MOVE data I use cut & paste with Samba: the move task is immediate :)

I tried the exact same thing with FTP but it still writes at 1.2MB/s and I'm not sure why...

 

[panz]

Please, help me understand the problem. Did you try to cut and paste with Samba? This should result in an IMMEDIATE data moving.

 

[id0827502]

Please, help me understand the problem. Did you try to cut and paste with Samba? This should result in an IMMEDIATE data moving.

No, I tried to cut and paste with FTP. I'm not sure what the difference would be however I tested your theory using CIFS and again it transferred at 1.2MB/s.

 

[panz]

You're doing something wrong, because cut and paste with Samba (NOT copy and paste!) should not move the data (aka the bits), but only update the pointers to those bits.

 

[id0827502]

I found an interesting serverfault article that advises of the following:

On a file "Move" (AKA Cut and Paste) between the same share it will simply change the index on the file server (pretty much instant).


Unfortunately when you try to move between two different shares (Even on the same file server) or do a "copy" operation it does pull the data through the client PC (slow).


On any "move" operation is does copy each file before deleting the source file, so at least that's good =)


Easy to test all of this by connecting to a share with a large folder and trying it out.

Which leads me to believe that when I copy/paste from one location to another on my NAS, it's pulling through my client PC and THEN copying to the specified location which would explain the slow speeds (going through my wireless connection). I just tested the above advice about how if you MOVE data (i.e a cut/paste) then the move will be instant, which is 100% true.

Interesting! Does anyone know why this is or if it can be turned off or is it the nature of the beast?

 

[id0827502]

You're doing something wrong, because cut and paste with Samba (NOT copy and paste!) should not move the data (aka the bits), but only update the pointers to those bits.

So if I cut/paste, it should still leave the source data in its orignal location? Sorry if I misunderstood.

EDIT: I understand now. I misread. Thank you for your help!

 

[rs225]

Maybe his FTP server does sync on each write? Try setting sync=disabled for the dataset and see if problem vanishes.

 

[cyberjock]

Maybe his FTP server does sync on each write? Try setting sync=disabled for the dataset and see if problem vanishes.

Note: this should be done for testing only and NEVER on a production system.

After testing set sync=standard again.

 

[camilo suarez]

just wondering, if i cut and paste a file that is stored in the Transmission Jail to another folder outside the jail, the data is going to be moved? or just the pointers are going to be updated?

 

[cyberjock]

Whether its "in a jail or not" is inconsequential. The real factor is where your file system boundaries are. Datasets are considered filesystem boundaries.