Looks like this thread might have gotten a little off track the last couple weeks... ;)
Here's the latest on my build:
All the jails are up and running. I've moved all the data over from my old NAS and have shut down all the services there. I've also taken the opportunity to "simplify" my sharing schemes. On the old NAS, the entire volume was shared via both NFS and CIFS. I retained an old school CIFS config where everything relied on UNIX permissions (so no ACLs). It worked, but it was clunky. I also had AFP turned on for Time Machine. This has proven to be somewhat limiting, especially now that 9.3 relies more on ACLs and extended attributes than it used to, at least for CIFS (which is more of a function of Samba than FreeNAS, but to-may-to, to-mah-to). The idea now is to have a separate dataset for each NAS Use Case (file sharing, VMPro VMWare backups, Time Machine, a scratch folder and Jails). This allows me to set permissions and ACL types separately, as well as set quotas for the various datasets.
The next step is to figure out a snapshot and backup policy. I had originally started out my old NAS with just a single volume and no datasets. This proved very limiting as far a snapshots went, and resulted in the volume filing up very quickly because even though the "file share" portion of the volume changed very little, the backups and scratch areas changed a lot, making snapshots very large. Once I figured this out, I created three child datasets containing various file shares and left everything else in the base volume. The datasets got snapshotted (once an hour, kept for two weeks and once a week, kept for 5 years) while the volume itself did not (most of what wasn't in a dataset were backups and scratch files). This is working rather well so far, but having six different snapshot policies is kind of a pain (too far in the other direction from "everything in one volume). This would become 8 snapshot tasks, now that I'm running jails. Which is a lot, especially if I want to replicate them. So, combining the snapshot policy I like with the dataset scheme from the paragraph above should give me the best of both worlds (I hope).
That takes care of snapshots, but what about backups? I still haven't decided exactly what I want to do here. What I'm leaning towards is a manual backup scheme using rsync. Since my backup device will be my old NAS, I don't want to leave it running 24/7 just so it can receive replication data from my main NAS. There's no real need to replicate every snapshot either. The hourly snapshots are more to recover from "oops" moments rather than recovering from a meltdown of the entire NAS. I would want to keep just the "long term" snapshots. I also need to make sure that, if I'm doing snapshots (as opposed to say, rsync) that I don't skip any snapshots since incremental snapshots require a complete chain of historical snapshots on the receiving device. tl;dr: Snapshots have a lot of constraints...
Another way to approach this would be to use rsync and do local snapshots on the backup device. This would be much more flexible as far as timing the backups goes. However, there could be filesystem level issues. I found this out when trying to rsync from my old NAS to the new one. I used the rsync service and exported my entire volume from the old NAS. I found that some files wouldn't come over (not a permissions issue as I was doing this as root), others would have their filenames garbled (which was very odd) and other times I ended up with entire directories that had screwy permissions on the new NAS (as in: no access for root! I ended up having to nuke the ACLs on these files and folders). Of course, I was transferring from Freeness 9.2.8 to 9.3 and there was probably all kinds of weirdness that took place over the last 4 years on my old NAS (which had started life running FreeNAS 8.x). Assuming I can iron out or work around any rsync issues, this will be my preferred method for backup: once a month turn on the backup NAS, run an rsync script, run a scrub, then turn it off until next month.
I do have one or two services that don't support SMTP AUTH and or TLS though, so it still comes in handy.
