macOS (10.12) Sierra

[adrianwi]

For anyone who might be worried whether their FreeNAS experience will be degraded, I installed the developer preview yesterday and was pleasantly surprised to find everything (AFP shares, TimeMachine and WUI) still works perfectly :D

 

[Ericloewe]

I heard rumors of Time Machine working over SMB (without hacks like setting up a disk image) - got anything interesting to share?

 

[adrianwi]

Mine are all setup with AFP shares, but if I get chance I'll create a new SMB share and try it.

So, I created a new dataset, configured cifs service and setup a cifs share. Not used it before so I might not have it right, but I could mount and write to it ok. When I selected a backup disk in TimeMachine the new share wasn't available as a option.

 

[fracai]

TimeMachine will be a lot more interesting with 10.13 when APFS is available with snapshots.

Will it require APFS at the destination?
Will APFS be documented enough or open source such that it could be implemented by FreeBSD?
Will APFS support sending snapshots and deltas to files?

 

[Ericloewe]

Will APFS be documented enough or open source such that it could be implemented by FreeBSD?

Well, it's a filesystem, not the network protocol. It's pretty clear they're going with SMB for networking.

They're probably going to implement Time Machine effectively like SMB Previous Versions. Even if it's gratuitously incompatible, it should be possible to replicate the functionality on FreeNAS, with a bit of work, that will depend on how good the documentation is.

Will APFS support sending snapshots and deltas to files?

I get the idea they're going for a more NTFS Shadow Copies approach rather than ZFS full CoW, so deltas are unlikely.

 

[fracai]

Well, it's a filesystem, not the network protocol. It's pretty clear they're going with SMB for networking.

What I meant was that it would be interesting if APFS itself becomes available for other platforms and becomes a target for sending deltas. (more later)

They're probably going to implement Time Machine effectively like SMB Previous Versions. Even if it's gratuitously incompatible, it should be possible to replicate the functionality on FreeNAS, with a bit of work, that will depend on how good the documentation is.

I may be misunderstanding how previous versions works, or what you mean, but it doesn't look at all like what I expect. My understanding is that previous versions are presented when you look at file and folder properties. TimeMachine is currently a separate interface with a fully navigable hierarchy. You can also browse the snapshot folders that TimeMachine currently uses to store the data and hardlinks (this will definitely change with APFS. directory hardlinks are going away for example). The example they've shown for working with APFS snapshots shows them mounted as read-only volumes.

To say it differently, I'm not interested in showing ZFS snapshots as if they were TimeMachine backups or Previous Versions (at least not in this discussion). I'm interested in storing TimeMachine snapshots on FreeNAS.

I get the idea they're going for a more NTFS Shadow Copies approach rather than ZFS full CoW, so deltas are unlikely.

Again, I disagree.

The APFS documentation specifically mentions copy-on-write, as does the APFS session video. Plus, Apple is not going to abandon TimeMachine backups on their AirPort devices or external drives. That's going to require sending snapshot deltas. It's possible that they'll keep this as part of a private API, but I'm betting there will be documented methods for getting a list of the blocks that are different between two snapshots. Even if FreeBSD doesn't or can't implement APFS I'm sure there will be a way to send deltas to a file. It would be kludgy, but that would enable sending backups to a ZFS volume without needing a disk image.

The session video is frustratingly light on details (API, capabilities), but it's better than nothing: https://developer.apple.com/videos/play/wwdc2016/701/

 

[fracai]

Additional APFS details: http://www.russbishop.net/apple-file-system

It is indeed currently missing snapshot deltas (this has to be coming in order to support TimeMachine) and data checksums (sounds like the developers disagree on the importance). I'll be hitting bugreporter.apple.com soon. And it's killing me that this is a developer preview for the next year and a half or so.

 

[anodos]

Well, it's a filesystem, not the network protocol. It's pretty clear they're going with SMB for networking.

They're probably going to implement Time Machine effectively like SMB Previous Versions. Even if it's gratuitously incompatible, it should be possible to replicate the functionality on FreeNAS, with a bit of work, that will depend on how good the documentation is.


I get the idea they're going for a more NTFS Shadow Copies approach rather than ZFS full CoW, so deltas are unlikely.

The pessimistic side of me thinks that Apple, coinciding with the new FS release, will start creating more apple-specific SMB2 extensions that samba will have problems dealing with (but somehow work on windows servers) causing Apple / samba compatibility to get in a bad state again just as things are starting to look good with vfs_fruit. Then the samba project will respond with more duct-tape, baling wire, and ram more things into xattrs and weird netatalk integration. This in turn will work kinda-sorta fine on Linux, but break in annoying and unpredictable ways on FreeBSD.

Nah. That'll never happen.

 

[Ericloewe]

The pessimistic side of me

I don't know, that sounds awfully realistic.

 

[anodos]

I don't know, that sounds awfully realistic.

I remember watching a YouTube presentation from a developer about smb2. He was excited that since MS opened up the protocol details that we wouldn't have problems with inconsistent client behavior like with SMB1. :D

 

[fracai]

A bit more info regarding APFS: http://dtrace.org/blogs/ahl/2016/06/19/apfs-part1/

It notably includes speculation and comments regarding how it'll interact with TimeMachine.

Given the way it will allow for clones at the file system and file level, I wouldn't be surprised if their solution allows sending snapshots that are "filtered" to remove blocks that are excluded from backup. Though that could be too costly to track.

Interesting times.

 

[anodos]

A bit more info regarding APFS: http://dtrace.org/blogs/ahl/2016/06/19/apfs-part1/

It notably includes speculation and comments regarding how it'll interact with TimeMachine.

Given the way it will allow for clones at the file system and file level, I wouldn't be surprised if their solution allows sending snapshots that are "filtered" to remove blocks that are excluded from backup. Though that could be too costly to track.

Interesting times.

Indeed. Very interesting.

I still have a hunch that Apple will either (1) end up introducing another custom smb2 protocol extension in its own server / client to handle backups over SMB (for performance reasons) or (2) tunnel whatever protocol is used for backups over SMB [like Apple does for Spotlight]. So I'm not holding my breath that "over SMB" will mean "over SMB to Samba" or even "over SMB to a Windows Server".

 

[dkusek]

Mine are all setup with AFP shares, but if I get chance I'll create a new SMB share and try it.

So, I created a new dataset, configured cifs service and setup a cifs share. Not used it before so I might not have it right, but I could mount and write to it ok. When I selected a backup disk in TimeMachine the new share wasn't available as a option.

We try NOT to use AFP in our environments with Macs as a lot of the time, we have Windows clients as well and need to make sure the share is accessible for all users. I was able to configure an CIFS/SMB share and get it working. Beware though, there is still the issue seen in 10.11.5 with SMB signing. I have not been able to get any type of performance over 50-60MB/s on a 10Gb Network WITHOUT disabling client side signing. Makes no difference what I do on the FreeNAS side of things. The only way to get performance back up to 600-900 MB/s is to disable client side signing from the specific mac users accessing the SMB share.

 

[Sebastian Schmid]

Anyone having issues with time machine on 10.12?
I can not backup anymore. With none of my updated macs.

Sebastian

 

[Ericloewe]

Major AFP update scheduled for monday, so try that update and if that doesn't work, we'll work from there.

 

[adrianwi]

I've got 2 macs running 10.12 now (one has been running the DP from the start) and both are backing up to FreeNAS using TimeMachine.

 

[Sebastian Schmid]

All right, update applied. Even deleted my user media to make the update work.

Same behavior prior and after the update.

I still can not backup to my old Time Machine dataset that contains backups which are 10.11
I can backup once, but only once to a newly created dataset. Works perfekt. But if I let my macs start to update the backup, TimeMachine will not go beyond preparing backup. It sits there. Hours and hours. Just the same as if I would try to backup to my old dataset.
It can see my macs login on freeness using my Time Machine user. But no backups. It keeps logging in and out. Just writing and reading a few kb.

Any clue?
Thanks, Sebastian

 

[Stux]

In the console app, search for backupd

This will show the time machine activity.

I have had tm get stuck like that before. Removing/re-adding the backup destination in the tm GUI resolved it, but it forces a full scan, which can take a long time.

 

[Sebastian Schmid]

Standard 16:05:08.224925 +0200 kernel AFP_VFS afpfs_vnop_ioctl: afpfs_FindForkRef failed -1

As soon as I try to resume an existing backup, these messages appear in the console, hundreds per second.
In the Time Machine gui on macOS you can see preparing backup. But the Backup will never start...

These messages disappear after I cancel the backup.

What can I Do?

Sebastian

 

[Stux]

Standard 16:05:08.224925 +0200 kernel AFP_VFS afpfs_vnop_ioctl: afpfs_FindForkRef failed -1

no entry for backupd, but that message is all over the place?

Is that on the Mac?