Problems with rsync

[jonandermb]

I'm trying to backup my Freenas Folders. Now this is my setup:
Freenas (Latest) serving shares to Mac users via AFP
To backup, i plug a USB drive to a Mac machine, this disk is formatted with Mac OS Plus filesystem
The files on the AFP share have their attributes plus extended attributes.

When rsyncing via AFP, meaning, pulling from afp share to the local usb disk on the Mac machine, the performance is quite slow and i get the feeling, some files are copied over again.
Now, rsync on the Mac machines is version 2.x, quite old.
I submit the -E flag when pulling the files, but this seems to be a Mac-specific thing, patched by Apple on the rsync version they ship. When pulling via AFP, the server recognizes this flag and sens over the xattr and they get properly stored on the external USB disk backup.
Now, rsync 3.x implements the -X flag to deal with extended attributes, so I installed it via macports on the Mac machine and tried to rsync via ssh but the remote server (freenas) does not seem to like the idea very much and fails (you get a null pointer exception usually after a couple of transfers)
Of course, if you try to rsync via ssh submitting the Mac-olny -E flag, freenas's rsync respondas that the option is unrecognized. The file transfers via ssh, without extended attributes modifiers is quite fast, however.
Can anyone shed a bit of light on this issue and help me troubleshoot how to rsync via ssh keeping xattr and also why rsync keeps pulling some files, even though they were not modified when over AFP?
BTW: This server has aroung 300k files+folders being served via AFP.
Thanks

 

[saurav]

About rsync seemingly pulling files unnecessarily, try running it with -nir flags. It will not actually pull anything but will give you an itemized list of files it would pull if you remove the -n (dry run) flag, along with the reason for pulling each file.

 

[jonandermb]

Sorry, I just figured everything out. I take your advice about dry-running. Much appreciated :)

I also had a conversation over reddit about this, maybe someone is interested about it.
I just went with the approach of dumping the extended attributes completelly and rsync via ssh.
Thanks for your help.

 

[Robert Trevellyan]

I just went with the approach of dumping the extended attributes completelly and rsync via ssh.

If you stopped caring about xattrs, you might enjoy DropSync 3, available from the Mac App Store (no affiliation). It makes rsync much easier on OS X.

 

[jonandermb]

After some days, I noticed that Mac sucks with rsync. When checking times, the time on the server and the one on the external usb differs and makes rsync think the file has changed, thus transferring the whole thing again, even though they are identical.
This calls for a backup filesystem change, but, keeping as much data as possible.
I'm open to any suggestion: NTFS, FAT32, ext4..... what do you think the best option would be?

 

[saurav]

Macs don't like NTFS much. They might mount it read-only unless you buy special software for NTFS. FAT32 is more acceptable, but it can't hold files larger than 4GB. I don't know ext4 is even an option.

You can try exFAT. You can share it with Windows but partition/format in Windows and then use on Mac.