NFS does *not* support server-side copy with ZFS? :(

[winnielinnie]

I'm not going to cry to coreutils about it though.

I almost want to, because it's as if the developers are still in the mentality of a pre-ZFS world with slow CPUs.

If I'm understanding correctly: inline compression, fast CPUs, and CoW filesystems (such as ZFS and Btrfs) make "preserving sparseness" pointless in these use-cases.

But like the situation with mainline Linux kernel development, I wouldn't be surprised if mentioning ZFS will make them scoff at you in disgust. I am but a mere, feeble end-user.

Not very from the point of final result, but reading/writing holes would be a waste of time.

A round-trip over the network is an even bigger waste of time. I'd rather just let my CPU with LZ4 inline compression do its magic, while leverage the speed of bypassing the network round-trip with a server-side copy instead. (Especially when there aren't really "sparse" files in the first place.)

Besides, up until these last few posts, this wasn't even about preserving the sparseness of a very small file that is "apparently" large. It dealt with MKV and MP4 videos (that contain the same video stream), with different containers. Neither are "highly sparse" files, and it's likely the way an MKV header is saved that seems to fool cp's "crude heuristic". (This can be true for other filetypes, but I don't want to keep repeating this over and over for different filetypes.)

 

[Ericloewe]

The lseek/copy_file_range loop would be ugly, but a functional solution for cp. @mav@, seeking to holes should be a "fast" operation (in the sense that it's metadata-only and the relevant metadata is probably sitting in ARC already), right?

 

[mav@]

@Ericloewe Right. It may require some metadata reads for large file, and it will be a random disk I/O, but that is incomparably faster than reading actual data.

 

[ChrisRJ]

Are there some benchmarks available? Or is it sufficiently clear under what circumstances there is how big an advantage from "analyzing" the file in the first place?

I am bringing this up, because I have seen so many cases (different details though), where comparable thinking was done, only to discover that performing such a check prior to every transaction would actually slow down the system overall.

 

[braiam]

Is there any issue tracking this? Should it be created?

 

[anodos]

Is there any issue tracking this? Should it be created?

IIRC this is simply an issue with core-utils / how users are copying files. copy_file_range() works as designed. As I mentioned earlier in this discussion:

We don't fork coreutils, and we don't offer support for operating as an NFS, SMB, etc client and so there's not really anything for us to fix here.

 

[Ericloewe]

Is there any issue tracking this? Should it be created?

It's more of an upstream GNU cp thing.

 

[braiam]

@winnielinnie there's a patch that is less pessimistic about using copy_file_range() proposed at coreutils bug tracker, you could try it out. In my case, it seems to always use it for files I can verify.

 

[winnielinnie]

@winnielinnie there's a patch that is less pessimistic about using copy_file_range() proposed at coreutils bug tracker, you could try it out. In my case, it seems to always use it for files I can verify.

Going to try out the patched version when I have the chance!

Big thanks, @braiam! I'm not in any mailing list (let alone GNU's), so I greatly appreciate you creating a bug report and referencing them back to this forum thread.

In regards to the discussion in that email thread: For those of us using ZFS as the underlying filesystem, I don't see the big deal with maintaining "sparseness", since inline compression handles storage space seamlessly.

 

[braiam]

@winnielinnie you don't need to join, simply sending a mail to the bug address that is listed here is enough. https://debbugs.gnu.org/cgi/bugreport.cgi?bug=60416

 

[winnielinnie]

Well good news, just tested it and it works as expected! (I didn't do as many different tests with different files, but it utilizes server-side copy where previously it did not.)

Looks like they already pushed it through. Should be hitting downstream soon enough.