Netatalk 2.2 for FreeNAS 8.0.1 Beta 4 - Lion AFP & Time Machine

[garth]

NOTE: This should only be attempted by those who are comfortable mucking about with their FreeNAS installation. This is at your own risk. There is no warranty or support for the below. I am not part of the FreeNAS team. This is only for FreeNAS 8.0.1 Beta 4. These will not work on FreeNAS 7 and I have no plans to try to compile there. If someone is interested in doing this for FreeNAS 7 they can pull a snapshot of Frank Lahm's master branch from Sourceforge git and try themselves.

UPDATE: Updated tar ball link below with other binaries included, atalkd, cnid_dbd, chid_metad, papd. Late night.

UPDATE 2: Initial release is for x86 systems.

UPDATE 3: Tarball for amd64 now available. Instructions below should be the same. (2011-08-02)

UPDATE 4: How to integrate the contents of the afp_voluuid.conf file with the adisk.service file. At the bottom. (2011-08-04)

---

I have compiled Netatalk 2.2 from Frank Lahm's master tag after he merged knu's fixes for FreeBSD. Those fixes can be summed up here:

https://github.com/franklahm/Netatalk/pull/1/files

Compiled versions can be found here:

x86 - http://dl.dropbox.com/u/32896380/netatalk-2.2.tar.gz

amd64 - http://dl.dropbox.com/u/32896380/netatalk-2.2-amd64.tar.gz

Notes:
I could not get the configure script to recognize Avahi on my FreeBSD 8.2 build box. So this afpd binary will not automatically register with Avahi. There is an additional file you will need to edit for Avahi and copy into place.

Instructions:
ssh into your FreeNAS box
nas# /usr/local/etc/rc.d/netatalk stop
nas# /usr/local/etc/rc.d/avahi-daemon stop

In order to make your changes survive a reboot you must mount the filesystem read/write:

nas# mount -uw /

scp netatalk-2.2.tar.gz to your FreeNAS installation.

untar and you will find the following files:

netatalk-2.2
->afpd binary
->atalkd binary
->cnid_metad binary
->cnid_dbd binary
->papd binary
->uams folder
->avahi-services folder

Make a backup of your existing Netatalk

nas# mv /usr/local/sbin/afpd /usr/local/sbin/afpd-2.1.5
nas# mv /usr/local/sbin/atalkd /usr/local/sbin/atalkd-2.1.5
nas# mv /usr/local/sbin/cnid_metad /usr/local/sbin/cnid_metad-2.1.5
nas# mv /usr/local/sbin/cnid_dbd /usr/local/sbin/cnid_dbd-2.1.5
nas# mv /usr/local/sbin/papd /usr/local/sbin/papd-2.1.5
nas# mv /usr/local/libexec/netatalk-uams /usr/local/libexec/netatalk-uams-2.1.5

cd to the netatalk-2.2 directory that was untarred.

nas# cp ./afpd /usr/local/sbin/
nas# cp ./atalkd /usr/local/sbin/
nas# cp ./cnid_metad /usr/local/sbin/
nas# cp ./cnid_dbd /usr/local/sbin/
nas# cp ./papd /usr/local/sbin/
nas# cp -rp ./uams /usr/local/libexec/netatalk-uams

Create a new directory for the signature file

nas#: mkdir /usr/local/etc/netatalk

Edit the adisk.service file in avahi-services. Replace the bits with curly brackets with your installation information. This is necessary since this afpd binary is not avahi aware. After editing, copy the file into place:

nas# cp ./avahi-services/adisk.service /etc/local/avahi/services

The following will make this change permanent after reboot
nas# cp /etc/local/avahi/services/adisk.service /conf/base/etc/local/avahi/services/

---

That should do it. Start Netatalk and Avahi

nas# /usr/local/etc/rc.d/netatalk start
nas# /usr/local/etc/rc.d/avahi-daemon start

AFP shares should mount as normal. You must still connect to the share you want to use for Time Machine before you can select it in the Time Machine system preferences in Lion.

I have tested this afpd binary with both Snow Leopard 10.6.8 and Lion 10.7. It resumed using existing sparsebundle backups. If this doesn't work for you, it should be straightforward to revert to Netatalk 2.1.5, assuming you followed the backup instructions.

Hopefully by the Fall, Netatalk 2.2 will be in FreeBSD ports and will make its way to FreeNAS 8.x.

---------------

UPDATE 4:

Update for complete Avahi integration:

This would be alot easier with a newer version of Avahi on FreeNAS and also having Netatalk compiled with Avahi support, but for now, for total OS X nirvana, the remaining steps can be followed:

For those that started using the tarballs earlier this week, the creation of /usr/local/etc/netatalk/ is an important step as it lets Netatalk generate and store UUIDs for volumes you have marked for Time Machine in the GUI or command line (indicated by options:tm in AppleVolumes.default).

After starting Netatalk with that directory created, you should connect from your Mac to your Time Machine share(s). You can then disconnect.

If you then look in the /usr/local/etc/netatalk directory you will find two files

nas#: ls -l /usr/local/etc/netatalk

afp_signature.conf
afp_voluuid.conf

The second one is of interest and will help us better define the adisk.service file - which is key to getting these shares visible to Time Machine without first mounting them.

So look inside your afp_voluuid.conf and you will see something like

nas#: cat /usr/local/etc/netatalk/afp_voluuid.conf

"TMShareName1" 185CCAF0-BED9-11E0-962B-0800200C9A66
"TMShareName2" 286A9850-BED9-11E0-962B-0800200C9A66

You may have more lines, you may just have one. Either way both values on each line are important

It is also important now to make sure that this directory survives a reboot.

nas#: cp -rp /usr/local/etc/netatalk /conf/base/etc/local/

Now we can more properly edit the adisk.services file. If Avahi was integrated with Netatalk, this would all happen automagically. When Netatalk 2.2 becomes part of FreeNAS and has the link to Avahi, this entire file will be generated automagically.

So using adisk.service in the tarball, you should concentrate in the first <section> node and the <txt-record> with the uuids as the part to edit.

This should be filled out using the values from afp_voluuid.conf.

Note: You still need to add the MAC Address and server name to the first <txt-record> line.

Note: These lines are different from what you will find in adisk.service in the tarballs.

Code:

<txt-record>dk0=adVF=0xa1,adVN=TMShareName1,adVU=185CCAF0-BED9-11E0-962B-0800200C9A66</txt-record>
<txt-record>dk1=adVF=0xa1,adVN=TMShareName2,adVU=286A9850-BED9-11E0-962B-0800200C9A66</txt-record>
<txt-record>dk2=adVN=AnotherNonTimeMachineShare,adVU=47C29F80-BEDA-11E0-962B-0800200C9A66</txt-record>

Now for the explanations:

- dk{n} should be sequential depending on the number of shares you have

- adVF=0xa1 is the magic bit that will make these shares show up in the Time Machine 'Select Disk' window in System Preferences->Time Machine on your Mac. Or when you need to restore.
This should only be on shares that are TimeMachine backups - i.e. have options:tm in AppleVolumes.default

- For non-Time Machine shares, any UUID will do - just make sure it is different. You can generate one here: http://www.famkruithof.net/uuid/uuidgen

- Typos are easy (trust me) - double check your work.

Now save adisk.services and restart Avahi

/usr/local/etc/rc.d/avahi-daemon restart

And then check things out on your Mac.

When you are satisfied, copy the adisk.service file to /base to make sure it survives a reboot:

nas#: cp /etc/local/avahi/services/adisk.service /conf/base/etc/local/avahi/services/

 

[Scrat]

No luck :(

Code:

# /usr/local/etc/rc.d/netatalk start
/usr/local/sbin/cnid_metad: 1: Syntax error: "(" unexpected
/usr/local/sbin/afpd: 1: Syntax error: "(" unexpected

Am I missing something?

Greets, Peter

 

[maxim.kazachek]

netatalk is FreeBSD ports collection since Saturday. I had tried to change files in FreeNAS ports collection and doing test build 7021 right now. Will see whether it will work or not

 

[garth]

Good luck Maxim - let us know how it goes - I don't think it will build without the patches from knu for FreeBSD. Let me know if you get it to recogize avahi.

 

[garth]

No luck :(

Code:

# /usr/local/etc/rc.d/netatalk start
/usr/local/sbin/cnid_metad: 1: Syntax error: "(" unexpected
/usr/local/sbin/afpd: 1: Syntax error: "(" unexpected

Am I missing something?

Greets, Peter

Make sure you grabbed the newer tarball with all the binaries, not just afpd.

 

[Scrat]

Make sure you grabbed the newer tarball with all the binaries, not just afpd.

Hi! I tried all from scratch, but get the same error. Do you have any further ideas?

Greets, Peter

P.S Yes, I'm running FN 8.0.1 b4

 

[garth]

Hi! I tried all from scratch, but get the same error. Do you have any further ideas?

Greets, Peter

P.S Yes, I'm running FN 8.0.1 b4

That error makes me wonder if it has something to do with where you are decompressing the files or how you are transferring the files to your freeness box. It's almost like an encoding or line feed problem. Maybe check the permissions and ownership of the files. They should match the old files.

 

[Scrat]

That error makes me wonder if it has something to do with where you are decompressing the files or how you are transferring the files to your freeness box. It's almost like an encoding or line feed problem. Maybe check the permissions and ownership of the files. They should match the old files.

Hmmm ... downloaded the archive again via another linux system (I have no Windows boxes here ;) ), scp'ed it again to my FreeNAS and untared it there. Then I checked the permissions and corrected some links like in the orignal folder (for safety, I'm sure it doesn't matter):

lrwxr-xr-x 1 root wheel 11 Aug 1 23:18 uams_clrtxt.so -> uams_pam.so
lrwxr-xr-x 1 root wheel 15 Aug 1 23:20 uams_dhx.so -> uams_dhx_pam.so
lrwxr-xr-x 1 root wheel 16 Aug 1 23:20 uams_dhx2.so -> uams_dhx2_pam.so

But I get the same error. :(

I guess something is missing on my FreeNAS, you've installed on your box.

So many people here blaming missing TimeMachine support, but nobody wants to try out this possible solution? :p

Greets, Peter

 

[garth]

These netatalk files are the only files I've changed on this installation.

What happens if you run

/usr/local/sbin/afpd -v

or

/usr/local/sbin/atalkd -v

Also, what does your afpd.conf look like

cat /etc/afpd.conf

 

[marko]

Have you compiled it on i386 or amd64 os?
I have the same problem as Scrat and I run 64bit

Code:

vmnas# /usr/local/sbin/atalkd -v
/usr/local/sbin/atalkd: Exec format error. Binary file not executable.
vmnas# /usr/local/sbin/afpd -v
/usr/local/sbin/afpd: Exec format error. Binary file not executable.

Any chance you could get a 64bit tar-ball?

Thanks,
Marko

 

[garth]

Aha - yes, my apologies - this is compiled for x86. I can put together a x64/amd build box and compile a 64-bit version, but it will take a couple days.

 

[pauldonovan]

Good luck Maxim - let us know how it goes - I don't think it will build without the patches from knu for FreeBSD. Let me know if you get it to recogize avahi.

The netatalk port Makefile lists patches for the commits that went into git master. Well, the filenames are the same as the files, I haven't actually tracked down the patch files themselves in CVS to read their contents. The port is in both CURRENT and STABLE as far as I can tell. If might be better to build using the port now that it seems to be fixed?

This is something I really need and I could help technically, but I won't have any free time until later in the month. Good luck!

Paul

 

[Cronock]

This thread has me excited that I may get my Time Machine working again. I'm also running 64bit, so I'll wait for at least one other person to have it sorted out first and then give it a shot. Any thoughts on when this might hit an official release of FreeNAS?

 

[garth]

Marko, Peter, Cronock and other FreeNAS amd64 users - I have compiled and uploaded a tarball with amd64 binaries. The link is up in my original post. These were compiled on a vmware virtual machine using the FreeBSD 8.2 AMD64 install. The VM CPU is Intel, so I can't be sure these will work with AMD processors.

I also don't have a 64-bit FreeNAS box, so you'll have to let me know if these work. These binaries were compiled the same way as the x86 versions.

-Garth

 

[garth]

The netatalk port Makefile lists patches for the commits that went into git master. Well, the filenames are the same as the files, I haven't actually tracked down the patch files themselves in CVS to read their contents. The port is in both CURRENT and STABLE as far as I can tell. If might be better to build using the port now that it seems to be fixed?

This is something I really need and I could help technically, but I won't have any free time until later in the month. Good luck!

Paul

I've been compiling applications on Linux for a long time, but I'm pretty new to the world of FreeBSD ports. I've been trying to use port upgrade on my build boxes to have it pull in the updated Avahi and Netatalk ports from FreeBSD ports but have been totally failing at that as well as finding decent instructions as to how portupgrade is supposed to work. If anyone has any pointers I can give it a try. They are just VMs so I don't care about wrecking them accidentally.

 

[pauldonovan]

I've not done it myself, but the documentation here: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/updating-upgrading.html might help. Also:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html
http://serveradmins.net/updating-the-freebsd-ports-tree-easily/

I agree though, getting your head around exactly how FreeBSD names branches and the methods of updating and rebuilding a system is difficult. I'm still learning. It doesn't help that there are a number of different ways to get updated code - portupgrade, portsnap, cvsup etc. I think portsnap might be the easiest, although I'm not clear on exactly how it chooses which branch to get stuff from - I think it defaults to current which in our case is what we want.

I wish the FreeNAS developers would do this for us, there are a number of tickets open about this

Paul

 

[pauldonovan]

I just tried your amd64 build and it works fine! Thanks very much. I'm using 8.01beta3 and have tested with OS X 10.6.7 and OS X 10.6.8. I only tested a Time Machine backup on the 10.6.8 machine but it continued OK.

I did notice in the FreeNAS logs that afpd was warning about a missing /usr/local/etc/netatalk directory so I created it myself. It looks like it stores a server signature file in there and a file containing GUIDs for advertising the TM share. I presume that's what it would use to advertise the share using Avahi.

I'll leave it running for a few days before I purchase Lion

Paul

 

[marko]

WORKS A TREAT!!!

Thanks very much Garth!

 

[Scrat]

Marko, Peter, Cronock and other FreeNAS amd64 users - I have compiled and uploaded a tarball with amd64 binaries. The link is up in my original post. These were compiled on a vmware virtual machine using the FreeBSD 8.2 AMD64 install. The VM CPU is Intel, so I can't be sure these will work with AMD processors.

I also don't have a 64-bit FreeNAS box, so you'll have to let me know if these work. These binaries were compiled the same way as the x86 versions.

-Garth

Garth, you're my hero! :)

After throwing your package in my FreeNAS I could select all my old TimeMachine-Backups (backed up with Snow Leopard 10.6.8) and finally I made my first TimeMachine-Backup with Lion 10.7.

My FreeNAS specs:

Version:
FreeNAS-8.0.1-BETA4-amd64

Platform and FreeBSD release:
FreeBSD 8.2-RELEASE-p2 amd64 FREENAS64

Hardware platform:
Pentium(R) Dual-Core CPU E5200 @ 2.50GHz

Many, many thanks

Peter

Edit: Of course many thanks for your very good documentation in your first post, it's just a cut&paste. :)

 

[wizpig64]

Regarding curly brace things in aidsk.service, is this correct?

* mac-address: gotten from 'ifconfig | grep ether' (this is the easy part)
* freenas-name: the machine's hostname as "myfreenas" or "myfreenas.local"?
* generate-a-uuid: any random string, or what?

Thanks.

May have figured it out with this: http://www.famkruithof.net/uuid/uuidgen

couldn't connect to time machine drive, bah. then again i'm connecting over the internets. Will try again when i get home and i'm on my local network.