SOLVED CIFS/SAMBA/SMB with vfs_fruit not supporting Mac OS X resource forks beyond 128KiB

[Stux]

So,

I'm trying to setup an SMB share so that Mac OS X users can use it. We have legacy content with resource forks exceeding 128KiB.

It is the sole share on the FreeNAS box.

I'm trying to rsync the legacy content from a Mac OS X Server to the share, but its not copying the resource forks on files greater than 128KiB.

I'm using Carbon Copy Cloner 4.1.10 (latest version) on the Mac OS X Server (10.9.5)

The problem is illustrated with these two log lines


2F0A4289| 09/28 14:47:51 DEBUG: Max xattr size for the destination filesystem is 131072 bytes


oh-oh.

and


2F0A4289| 09/28 14:47:51 The "com.apple.ResourceFork" extended attribute on "/Volumes/shared/<redacted>" is too large (151488 bytes) to be preserved on the destination.


And yes, if I check the file on the share, it has a Resource Fork of 0 bytes, where it should be more like 150KB.

Now, I've added vfs_fruit to the cifs share "shared" already. Which is I believe supposed to fix this.

And I've also then begun trying to add various auxillary parameters, based on the vfs_fruit manpage

"fruit:resource file", although it is supposed to be the default, is supposed to work around this limitation in freebsd's zfs x_attr support.

Nothing seems to help.

Any ideas?

 

[Stux]

It doesn't work with a bare finder copy either.

 

[Stux]

I've prepared a zip file with two files with resource forks. One is an old ReadMe document with 8KB resource fork for an embedded PICT resource. The other is an old system extension with 151KB of resource fork.

If you decompress this zip on an OSX box, it should work fine. The ReadMe should open in TextEdit and display the PICT resource. The extension should have an icon. The included PNGs show how things should look.

Now, copy this directory to a FreeNAS SMB share. It should basically appear the same on the share. but doesn't. So it can't round trip.

Also, you can't decompress the .zip on the share (at least I can't), presumably because OSX fails to create the resource fork.

Mac OS X 10.11.6: Unable to expand "resourceForkTest.zip" into "shared". (Error 2 - No Such file or directory.) [for searchers]

Will submit a bug if this is still an issue when I update to U1 (waiting for something to finish).

 

[Stux]

I've prepared a zip file with two files with resource forks. One is an old ReadMe document with 8KB resource fork for an embedded PICT resource. The other is an old system extension with 151KB of resource fork.

If you decompress this zip on an OSX box, it should work fine. The ReadMe should open in TextEdit and display the PICT resource. The extension should have an icon. The included PNGs show how things should look.

Now, copy this directory to a FreeNAS SMB share. It should basically appear the same on the share. but doesn't. So it can't round trip.

Also, you can't decompress the .zip on the share (at least I can't), presumably because OSX fails to create the resource fork.

View attachment 13896
Mac OS X 10.11.6: Unable to expand "resourceForkTest.zip" into "shared". (Error 2 - No Such file or directory.) [for searchers]

Will submit a bug if this is still an issue when I update to U1 (waiting for something to finish).

I checked an SMB share exported from an OSX box, using the above zip file. It decompresses correctly, but also shows the No Such file or directory error. Will have to investigate that separately, but it shows OSX Server can certainly handle resource forks.

Second time I tested it it worked without an error.

 

[Stux]

Just created a vanilla Windows dataset, added a CIFS share pointing to it. Added the "catia" and "vfs_fruit" additions.

(thus it has default permissions)

Connected, tested. No resource fork support.

This was to ensure that it wasn't something to do with the Windows ACL stuff I've already setup on the first windows dataset.

 

[Stux]

Updating to U1 seems to resolve the issue.

Double confirmed by rebooting to 9.10.1 and confirming it didn't work, then rebooting back to 9.10.1-U1 and confirmed it worked again.

I noticed a failure getting the smb4.conf in the U1 boot crawl. Its after dmesg, does anyone know where the boot log is?

 

[Stux]

So I worked out how to fix the OSX can't unzip zips on a samba share error.

The basic issue is that an OSX zip includes a __MACOSX directory, which incudes AppleDouble encoded resource forks for the various files in the zip. OSX expands this in place, then consumes the __MACOSX directory, and sets the resource forks extended attributes on the files.

Then vfs_fruit stores the extended attributes... in AppleDouble encoded files... the same ones that just got consumed, but now in the parent directory.

That's not the problem.

The problem is that by default vfs_fruit vetoes the appledouble files. This is great, but unfortunately, means when you expand the zip, it disallows the creation of the AppleDouble directories, and then the zip expansion fails.

By adding this aux param to the share:
fruit:veto_appledouble=no

fruit will no longer veto the appledouble files, and zip expansion works as expected. The apple doubles are consumed, deleted, and appied to the expanded items. and then fruit recreates them. All works great in the GUI.

They are dot prefixed, so windows users shouldn't see them if you have "Show Hidden Files" unchecked, and neither will Mac or Unix users (unless they look) because they're dot prefixed. But it will be another detritus strewn about a share by mac clients (actually by vfs_fruit).

The problem is that file utilities, especially rsync type utils, will see them.

Using a mac based resource fork safe cloner on this would be a bad idea. It'd see the apple double files, delete them, then wonder where the resource forks went.

Hmmm. Ideally, vfs_fruit would be able to use the extended attributes.

So, at the end of the day, I think its better to have transparent resource fork handling, at the expense of vetoing appledouble. Which means files which look like AppleDouble can't be created/manipulated on the share.

You can test the failure mode quite easily form OSX's terminal


$ mkdir touch_test
$ cd touch_test
$ touch ._u_cant_touch_this
touch: ._u_cant_touch_this: Permission denied
$ touch .u_cant_touch_this
$ touch u_cant_touch_this
$ ls -alt
total 64
drwxrwxrwx 1 stux staff 16384 28 Sep 18:24 .
-rw-r--r-- 1 stux staff 0 28 Sep 18:24 u_cant_touch_this
-rw-rw-rw- 1 stux staff 0 28 Sep 18:24 .u_cant_touch_this
drwxrwxrwx 1 stux staff 16384 28 Sep 18:23 ..
$


for bonus points, here's a nasty way to write some data to the resource fork:


$ echo 'resource fork!' > u_cant_touch_this/..namedfork/rsrc
$ ls -l u_cant_touch_this/..namedfork/rsrc
-rwx------ 1 stux staff 15 28 Sep 18:32 u_cant_touch_this/..namedfork/rsrc
$


15 bytes of resource fork data, which is actually stored in appledouble behind the scenes. This was all on the FreeNAS SMB share.

 

[nojohnny101]

I've never seen a thread so long with one person talking to himself :D

Thanks for sharing! Good troubleshooting!

 

[anodos]

So I worked out how to fix the OSX can't unzip zips on a samba share error.

The basic issue is that an OSX zip includes a __MACOSX directory, which incudes AppleDouble encoded resource forks for the various files in the zip. OSX expands this in place, then consumes the __MACOSX directory, and sets the resource forks extended attributes on the files.

Then vfs_fruit stores the extended attributes... in AppleDouble encoded files... the same ones that just got consumed, but now in the parent directory.

That's not the problem.

The problem is that by default vfs_fruit vetoes the appledouble files. This is great, but unfortunately, means when you expand the zip, it disallows the creation of the AppleDouble directories, and then the zip expansion fails.

By adding this aux param to the share:
fruit:veto_appledouble=no

fruit will no longer veto the appledouble files, and zip expansion works as expected. The apple doubles are consumed, deleted, and appied to the expanded items. and then fruit recreates them. All works great in the GUI.

They are dot prefixed, so windows users shouldn't see them if you have "Show Hidden Files" unchecked, and neither will Mac or Unix users (unless they look) because they're dot prefixed. But it will be another detritus strewn about a share by mac clients (actually by vfs_fruit).

The problem is that file utilities, especially rsync type utils, will see them.

Using a mac based resource fork safe cloner on this would be a bad idea. It'd see the apple double files, delete them, then wonder where the resource forks went.

Hmmm. Ideally, vfs_fruit would be able to use the extended attributes.

So, at the end of the day, I think its better to have transparent resource fork handling, at the expense of vetoing appledouble. Which means files which look like AppleDouble can't be created/manipulated on the share.

You can test the failure mode quite easily form OSX's terminal


$ mkdir touch_test
$ cd touch_test
$ touch ._u_cant_touch_this
touch: ._u_cant_touch_this: Permission denied
$ touch .u_cant_touch_this
$ touch u_cant_touch_this
$ ls -alt
total 64
drwxrwxrwx 1 stux staff 16384 28 Sep 18:24 .
-rw-r--r-- 1 stux staff 0 28 Sep 18:24 u_cant_touch_this
-rw-rw-rw- 1 stux staff 0 28 Sep 18:24 .u_cant_touch_this
drwxrwxrwx 1 stux staff 16384 28 Sep 18:23 ..
$


for bonus points, here's a nasty way to write some data to the resource fork:


$ echo 'resource fork!' > u_cant_touch_this/..namedfork/rsrc
$ ls -l u_cant_touch_this/..namedfork/rsrc
-rwx------ 1 stux staff 15 28 Sep 18:32 u_cant_touch_this/..namedfork/rsrc
$


15 bytes of resource fork data, which is actually stored in appledouble behind the scenes. This was all on the FreeNAS SMB share.

Thanks for posting this here. I don't use OSX myself and so don't have much experience troubleshooting Mac problems. If you need help with more technical issues regarding Samba, the best place to get support is the Samba mailing list. If you find something you consider to be a legitimate bug (or something that you think the GUI can handle better) be sure to file a bug report on the FreeNAS bug tracker.

 

[ovizii]

so what is your end result, aka what auxiliary parameters and VFS objects did you end up using in the end?

 

[Stux]

It's on the back burner at the moment.

Right now I have an issue with permissions I'm trying to solve.

I'm planning on investigating inherit_acls and the like.

But to get resource forks working properly, update to u1 or later, and enable vfs_fruit and the modules the manual says to use with it.

 

[Stux]

Yeah, I saw that. Its part of my investigations for sure. Just literally no time for playing with it at the moment.

 

[anodos]

Yeah, I saw that. Its part of my investigations for sure. Just literally no time for playing with it at the moment.

What's ironic is that I once saw a presentation by a samba dev about how CIFS / SMB1 was such a mess because the specs were closed / poorly documented resulting in lots of weirdly-behaving implementations of them, but how much better SMB2 is because of how open MS is being. Thanks Apple!

 

[tobiasbp]

The problem is that by default vfs_fruit vetoes the appledouble files. This is great, but unfortunately, means when you expand the zip, it disallows the creation of the AppleDouble directories, and then the zip expansion fails.

By adding this aux param to the share:
fruit:veto_appledouble=no

fruit will no longer veto the appledouble files, and zip expansion works as expected. The apple doubles are consumed, deleted, and appied to the expanded items. and then fruit recreates them. All works great in the GUI.

This worked for me in FreeNAS 11.1. OS X users can now unzip files in the SAMBA share.

My impression is, that this is only an issue with .zip file created with OS X native zipping tool. Is that correct?

 

[anodos]

@Stux,

FYI, FreeNAS 11.1-U1 now supports large resource forks.

 

[Stux]

@Stux,

FYI, FreeNAS 11.1-U1 now supports large resource forks.

Is that via large xattrs?

 

[anodos]

Is that via large xattrs?

Samba had a 64KB size limit in it's xattr code. @BaT wrote a patch that fixed the issue.

 

[Stux]

Sounds like I need to retest the fidelity of a samba FreeNAS store again :)

 

[anodos]

It might be worth trying the following options if you're going to use vfs_fruit:

Code:

fruit:resource = stream
fruit:metadata = stream

This is of course if you're not using AFP as well as Samba. If you are using AFP, then you might want Samba to use netatalk compatible xattrs for AFP_Info and AFP_Resource.

 

[tobiasbp]

It might be worth trying the following options if you're going to use vfs_fruit:

Code:

fruit:resource = stream
fruit:metadata = stream

I understand this as: You no longer need to use fruit:veto_appledouble=no to be able to unzip files in place. You should now use fruit:resource = stream and fruit:metadata = stream instead. And, since fruit:veto_appledouble=no may have side effects (According to the man page), it is not the preferred way of handling this problem.

I have switched to using fruit:resource = stream and fruit:metadata = stream on a box running FreeNAS 11.1-U1. Unzipping from OS X works as expected.

I have tried it on FreeNAS 11.1. Using the resource and metadata options results in an unbearable amount of "beach balling" (slowness) when accessing the file server from a machine running Mac OS 10.11.6.

I'm going back to fruit:veto_appledouble=no, which works fine.

I have not updated the zpool to the latest version. Could it be relevant to this issue?

This is of course if you're not using AFP as well as Samba. If you are using AFP, then you might want Samba to use netatalk compatible xattrs for AFP_Info and AFP_Resource.

I understand this as: Unless you are sharing the same data with AFP.