Best practices for backup and recovery - advice needed.

[Thomas_VDB]

Hi all,

Since long I have a FreeNAS (microserver) running for multiple home users. I have also deployed a FreeNAS with same configuration in an SMB.
All data is on the network share. No local data on the client PC's/laptops.

For redundancy the data disk is mirrored, and for backup a cron job rsyncs the data each night to a 3rd internal disk.
However I am not fully happy with this backup solution : If a file gets corrupt, the file on the backup disk gets overwritten at night.

I am looking for a way to :
- Perform a full backup once to this internal disk, and from then do incremental or differential backups each night
- Keep only last 10 days or so.
- Easy recovery process, file based. So that a user can recover a file from a backup of 5 days ago without rolling back the entire data disk.
So I guess this means that copying snapshots is out?
- Backup and recovery preferrably via the GUI
- All managed and running automatically on the FreeNAS.
- Reporting?

Can anyone please tell me if this is configurable or external programs/plugins are needed?
Thx.

 

[Thomas_VDB]

As this question probably has been asked over and over again, I've spent a fair amount of googling and browsing the forums.
Now I can answer my own question. Or at least describe what my new backup plan is.

I see 3 levels of 'security' :
1. Lowest level (disk) : mirrored disks for redundancy. I know, it's not backup.
2. Image level (volume snapshot) : using cron job to sync/copy volume snapshots to a 2nd disk (internal) or other FreeNAS on the network.
3. File level : I want to be able to restore individual files/folders (because of user mistakes, deletions, getting corrupt).
For this, I was looking for a user-friendly solution where the users can restore the files themselves.
Somehow it took me a lot of time to find it, but the Windows built-in shadow copies (Previous versions) can work together with FreeNAS snapshots.

 

[nojohnny101]

Since long I have a FreeNAS (microserver) running for multiple home users. I have also deployed a FreeNAS with same configuration in an SMB.

Just for clarification purposes, you have two FreeNAS boxes correct?
I believe replication with snapshots can accomplish most of what you want, see below (YES indicate features of snapshots/replication):
YES - Perform a full backup once to this internal disk, and from then do incremental or differential backups each night
YES - Keep only last 10 days or so.
YES (I believe windows supports "shadow copies" in this way but if you have a mixed environment, restoring files requires cloning the appropriate snapshot, creating a share, then mounting it - Easy recovery process, file based. So that a user can recover a file from a backup of 5 days ago without rolling back the entire data disk.
YES - Backup and recovery preferrably via the GUI
YES - All managed and running automatically on the FreeNAS.
Not sure what you mean by this. FreeNAS (via GUI) will report the last snapshot that was replicated - Reporting?

 

[Ericloewe]

YES (I believe windows supports "shadow copies" in this way but if you have a mixed environment, restoring files requires cloning the appropriate snapshot, creating a share, then mounting it - Easy recovery process, file based. So that a user can recover a file from a backup of 5 days ago without rolling back the entire data disk.

Technically, the component is "previous versions", which on NTFS is backed by Shadow Copies, which are essentially a CoW thing for NTFS. On FreeNAS, they're backed by ZFS snapshots, so they work mostly the same.

Why mostly? As of now, only one snapshot task can be selected for Previous Versions. However, plans are in place to allow for arbitrary snapshots to be exported.

 

[Thomas_VDB]

Thanks for your reply! Happy that you confirmed with I figured out via the forums.
I only have one FreeNAS of my own. Just advised/installed the FreeNAS of some friends.
For reporting : I don't know what's possible, but it would be nice to receive a mail each night that the snapshot sync went successfully.

 

[Arwen]

One thing I started doing with my backup disk;

• Backup disk is ZFS single 8TB disk pool
• Backups use Rsync, which only copies changes
  
• After backup, I snapshot the backup ZFS dataset with that day's date

Thus, I have both full and incrementals, curtesy of ZFS and Rsync. When
the backup disk get's full, I'll delete some of the older snapshots. Bonus is
that it's faster than my prior scheme of new ZFS dataset per backup.

Note that while I could use ZFS send and receive, I am much more familar
with Rsync. (And happy with the current scheme.)

 

[michael.samer]

Hi
I'm currently in about the same situation: FreeNAS-11.1-U1, two big storages (same size); one is shared and use a dataset for each share (currently 5, all CIFS with VSS) with attending snapshots policies, the second shall be its backup. I used ZFS Replication, and it works and reports all syncs fine and running with last snapshot.
Question is: how do I backup all the snapshots of the Volume 1 (live data) to the volume 2 (Backup). Currently it seems only the datas in the snapshot is backuped, but not the snapshot structure. So in case of a failure I get the last working set, but all previous changes seems lost to me.
Cheers
Michael

 

[Scharbag]

One thing I started doing with my backup disk;

• Backup disk is ZFS single 8TB disk pool
• Backups use Rsync, which only copies changes
  
• After backup, I snapshot the backup ZFS dataset with that day's date

Thus, I have both full and incrementals, curtesy of ZFS and Rsync. When
the backup disk get's full, I'll delete some of the older snapshots. Bonus is
that it's faster than my prior scheme of new ZFS dataset per backup.

Note that while I could use ZFS send and receive, I am much more familar
with Rsync. (And happy with the current scheme.)

If you snapshot PRIOR to your RSYNC task, then you will will be able to revert to files that are overwritten by RSYNC.

And I am the same. I use RSYNC instead of ZFS replication due to familiarity and granularity.

Cheers,

 

[Arwen]

If you snapshot PRIOR to your RSYNC task, then you will will be able to revert to files that are overwritten by RSYNC.

And I am the same. I use RSYNC instead of ZFS replication due to familiarity and granularity.

Cheers,

Uh, I'd just have to revert to my prior snapshot...

In my case, I use the date as the snapshot name, ala @YYYYMMDD, so it makes more sense to me to use the date the backup & snapshot was taken.

 

[Scharbag]

Uh, I'd just have to revert to my prior snapshot...

In my case, I use the date as the snapshot name, ala @YYYYMMDD, so it makes more sense to me to use the date the backup & snapshot was taken.

Yeah, tomato tamoto. I have always prescribed to the theory of snapshotting before a planned change instead of after.