Backup on localhost, Snapshots/Replication or Rsync?

[Cephyid]

I have my system up and running and have been fiddling and experimenting with Snapshots and Replication as well as Rync. I'm really not sure what's best for me.

I have one volume with 4 datasets that I want to backup. For backup I have 2 volumes setup as well as an additional volume if needed. Rather than duplicating the datasets I'd rather just have one backup dataset and dump a folder in there with the source dataset name. Ie dataset Photos goes into a folder called photos on backup volume.

I've manually backed up data using CIFS shares just to make sure I have a backup but I really want regular, incremental backups that are easy to restore from.

1. A snapshot doesn't appear to take up any space until the second snapshot comes around with the difference. If I replicate it I really don't know what data I'm replicating or how I would restore data from it if something went wrong. I see the rollback snapshot button but if I press it will it show me some kind of useful information? The snapshot names just have a data and that's not telling me much when I do one every 15 minutes. In addition I'm trying to figure out if I can do a snapshot if the drive is more than 50% full. The destination drive has plenty of space but do I need twice the capacity on the source drive to do proper snapshots?

2. With rsync I can at least see a duplicate of the data. The problem here has probably been on my end where it has done crazy things like create nested folders in both the source and destination directory that are undeletable even after removing the rysnc task. I had to delete the datasets and start over and before getting into this again with no trailing slash on the source I just want to know which backup strategy is best for me.

Thanks and happy holidays

 

[SweetAndLow]

Snapshots and replication is the way to go if you can when doing a backup. It gives you a good consistent picture in time that doesn't change that you can backup.

Your idea of having a backup dataset on the dest is a good idea but you might end up with datasets under that if there are also datasets in the source. You mentioned just having folders on the dest but I don't think that is possible if the source is a dataset.

1. Snapshots track changes you have made so they don't need to store all the data just changes. And then when you replicate it only replicates the changes. This speeds things up and reduces bandwidth usage.

Rolling back a snapshot is different then restoring from a backup. Snapshots are not backup but replication is backup. You should try the rollback feature with some test data. It will allow you to make a dataset look exactly like it did when the snapshot was taken. You can also clone a snapshot into a new dataset and recover a single file if needed.

To recover using your backup would mean your original pool is completely trashed. To do this you would just replicate in the reverse order.

2. I don't understand because with replication you should see a copy of your data also.

 

[Cephyid]

Well I've identified the problem with replication. Let me see if I'm understanding this correctly.

1. I have a volume with 4 datasets. I create 4 different periodic snapshots. Therefore I can ONLY replicate to a dataset with the same name? I might be able to replicate the whole volume to another whole volume but I want 3 datasets on one drive and the 4th on another drive.
   
2. By creating a volume with simply 3 folders in it I seem to have thoroughly confused Freenas. I'm thinking I need to just delete the volume, start over, and create 3 datasets.

 

[Cephyid]

Once I've replicated the data, it's just a normal backup? The only kind of incremental backup is locally with snapshots. In the case of a hardware failure that would be lost since those can't be backed up?

 

[depasseg]

Well I've identified the problem with replication. Let me see if I'm understanding this correctly.

1. I have a volume with 4 datasets. I create 4 different periodic snapshots. Therefore I can ONLY replicate to a dataset with the same name? I might be able to replicate the whole volume to another whole volume but I want 3 datasets on one drive and the 4th on another drive.
   
2. By creating a volume with simply 3 folders in it I seem to have thoroughly confused Freenas. I'm thinking I need to just delete the volume, start over, and create 3 datasets.

Can you please retype this without using the word volume? I'm not sure what you mean. Also, "disks" are irrelevant. Pools and datasets are what need to be referred to (folders are an irrelevant sub part of a dataset).

And, no, the source and destination names don't need to be the same.

You need to pay attention to what you are taking snapshots of, and what is being replicated. Especially is you choose the recursive options.

Snapshots are backed up as long as you have a replication task. Think of it this way - you take snapshots in order to be able to go back to a previous point in time and you use replication to make a copy of the source data (including all the snapshot info) in case there is a hardware failure/theft/fire/etc.

 

[Cephyid]

Can you please retype this without using the word volume? I'm not sure what you mean. Also, "disks" are irrelevant. Pools and datasets are what need to be referred to (folders are an irrelevant sub part of a dataset).

And, no, the source and destination names don't need to be the same.

You need to pay attention to what you are taking snapshots of, and what is being replicated. Especially is you choose the recursive options.

Snapshots are backed up as long as you have a replication task. Think of it this way - you take snapshots in order to be able to go back to a previous point in time and you use replication to make a copy of the source data (including all the snapshot info) in case there is a hardware failure/theft/fire/etc.

I have one pool (2 drives in a mirror) with 4 datasets. These need to be backed up. I have periodic snapshots setup every 15 minutes.

For backup I have one pool with 1 drive in it to backup one of the datasets. I ideally want to also have this backup onto another pool with an external drive. I haven't tried another replication task for the same data but I'll get to that overnight.
I have another pool with an external drive that I am backing up the other 3 datasets. I have all of this working fine now but only from dataset to dataset. The pool with 3 datasets to another pool with no datasets didn't work. Neither did it when I had folders setup with the same name.

The backups appear to simply be straight duplicates of my current data. They were kinda weird at first since they used old manual snapshots but as the periodic snapshots kicked in everything normalized. I then see snapshots for both the backup and main data datasets but since I've been taking 15 min snapshots I didn't notice this before.

So I suppose I am setup to be able to recover data from either source at this point. I have the snapshots setup to expire in 2 weeks. I think what I would ideally want is for that to remain the case on the production machine but have them remain longer on the backup. Possible? 2 week snapshots in case I screw up but much longer backed up?

Lastly, two thumb drives with the config is good enough to make sure if I have a problem of some kind I can always boot back in, see my snapshots, and clone to a new drive?

 

[depasseg]

Can you please paste screenshots of each of your replication tasks as well as the output of 'zfs list' (using code tags)?

I'm doing what you are trying to do, so I'm guessing it's a configuration issue.

 

[Cephyid]

Code:

[root@freenas] ~# zfs list
NAME                                                             USED  AVAIL  REFER  MOUNTPOINT
Backup_Data                                                      216G   683G    96K  /mnt/Backup_Data
Backup_Data/Daniel                                              73.9G   683G  35.0G  /mnt/Backup_Data/Daniel
Backup_Data/Eliana                                               136G   683G   136G  /mnt/Backup_Data/Eliana
Backup_Data/Eliana y Daniel                                     6.21G   683G  6.21G  /mnt/Backup_Data/Eliana y Daniel
Backup_Photos                                                   1013G   785G    96K  /mnt/Backup_Photos
Backup_Photos/Photos                                            1013G   785G  1013G  /mnt/Backup_Photos/Photos
data_volume                                                     1.16T   607G    96K  /mnt/data_volume
data_volume/Daniel                                              35.0G   607G  35.0G  /mnt/data_volume/Daniel
data_volume/Eliana                                               136G   607G   136G  /mnt/data_volume/Eliana
data_volume/Eliana y Daniel                                     6.21G   607G  6.21G  /mnt/data_volume/Eliana y Daniel
data_volume/Photos                                              1013G   607G  1013G  /mnt/data_volume/Photos
freenas-boot                                                    1.16G  6.05G    31K  none
freenas-boot/ROOT                                               1.13G  6.05G    25K  none
freenas-boot/ROOT/FreeNAS-9.3-STABLE-201511280648               3.76M  6.05G   514M  /
freenas-boot/ROOT/FreeNAS-9.3-STABLE-201512121950               1.13G  6.05G   517M  /
freenas-boot/ROOT/Initial-Install                                  1K  6.05G   510M  legacy
freenas-boot/ROOT/Wizard-2015-12-02_18:31:05                       1K  6.05G   511M  legacy
freenas-boot/ROOT/default                                       2.00M  6.05G   512M  legacy
freenas-boot/grub                                               20.4M  6.05G  6.79M  legacy
videos_volume                                                   3.13T  5.64T   220K  /mnt/videos_volume
videos_volume/.system                                           3.64M  5.64T   104K  legacy
videos_volume/.system/configs-5ece5c906a8f4df886779fae5cade8a5  1.21M  5.64T  1.21M  legacy
videos_volume/.system/cores                                      796K  5.64T   796K  legacy
videos_volume/.system/rrd-5ece5c906a8f4df886779fae5cade8a5        96K  5.64T    96K  legacy
videos_volume/.system/samba4                                     732K  5.64T   732K  legacy
videos_volume/.system/syslog-5ece5c906a8f4df886779fae5cade8a5    756K  5.64T   756K  legacy
videos_volume/Applications                                       216G  5.64T   216G  /mnt/videos_volume/Applications
videos_volume/music                                             86.6G  5.64T  86.6G  /mnt/videos_volume/music
videos_volume/videos                                            2.84T  5.64T  2.84T  /mnt/videos_volume/videos

 

[depasseg]

Ok, so the problem is that you have 3 different replication tasks all pointing to the same remote volume/dataset. You need to create 3 datasets on "Backup_Data" and change each of your replication tasks accordingly. If it were me (and I've been there and done it), I would
1. change the 3 replication task destinations (this will cause replication to pause while we fix the destination)
2. destroy everything in Backup_Data (detach the dataset
3. Recreate the Backup_data and create 3 datasets (Daniel, Eliana, and eliana y Daniel)

Once those are created, and the next snapshot occurs, your data will begin replicating.

 

[Cephyid]

Wait. That's what I have setup. I know that the replication task points just to Backup_Data but the data is being replicated to the correct dataset. The space being used reflects that too. Is this just a case where it can handle Backup_Data and Backup_Data/name? Originally I tried to point them to folders instead of datasets and it did not like that so I created datasets and just matched the names and it seems to like that.

 

[depasseg]

Oh, OK. So it's working properly?

 

[Cephyid]

I think so. My question at this point is that IF I need to recover data how would it be done? I can see that I am getting snapshots taken on regular intervals, those same snapshots are getting backed up. What I don't know is if the backup snapshots expire in 2w and how I would keep longer snapshots as backup. Also if I just need to go back 2 hours to get a file do I need to clone the whole snapshot to another drive to pull that one file out? If for some reason one drive in my mirror goes down I can just get another drive and I'd probably be ok. If something terrible happened I could just copy data directly from the backup drive. If something even worse happened, the drives died, and my last data was bad, I could go back to a previous snapshot off the backup drive and hit the clone button?

 

[depasseg]

By default, the replication tasks will prune the snapshots on the destination end to follow the source. Meaning if you have your snapshots to expire in 2 weeks, then when the replication occurs, those snapshots that expired on your source (and were removed) will also be removed on the destination. There is a setting to enable/disable the removal of stale snapshots. My suggested way would be to create a couple more snapshot tasks (maybe once per week, keep for 12 weeks and another for once per month and keep for 12 months).

If you are sharing using CIFS, you can use the previous versions option in windows to get a previous version. Otherwise, yes you will need to clone, mount/share, get the file and then remove the clone.

I'd suggest installing freenas in a virtual machine and playing around with it so you understand it better.

If the source drive somehow got destroyed, you wouldn't need to clone. You could replciate them back to your repaired main pool, and while that is happening you can actually share your backup datasets. Replications are exactly that - copies of your source data. That's why I was trying to point out the differences between snapshots and replication.

 

[Cephyid]

By default, the replication tasks will prune the snapshots on the destination end to follow the source. Meaning if you have your snapshots to expire in 2 weeks, then when the replication occurs, those snapshots that expired on your source (and were removed) will also be removed on the destination. There is a setting to enable/disable the removal of stale snapshots. My suggested way would be to create a couple more snapshot tasks (maybe once per week, keep for 12 weeks and another for once per month and keep for 12 months).

So theoretically if I turned off replication then in 2 weeks my backup drives would be empty? Or is the confusion I'm having that the replication is a duplication of my data and because I'm taking snapshots those happen to get replicated too? I see " Replication Tasks: used to schedule the replication of snapshots to a remote system."

I'll follow your suggestion since I think that will do the trick to make sure I can pull old files if needed.

If you are sharing using CIFS, you can use the previous versions option in windows to get a previous version. Otherwise, yes you will need to clone, mount/share, get the file and then remove the clone.

10.4.3. Configuring Shadow Copies in the manual?

I'd suggest installing freenas in a virtual machine and playing around with it so you understand it better.

If the source drive somehow got destroyed, you wouldn't need to clone. You could replciate them back to your repaired main pool, and while that is happening you can actually share your backup datasets. Replications are exactly that - copies of your source data. That's why I was trying to point out the differences between snapshots and replication.

 

[depasseg]

the replication is a duplication of my data and because I'm taking snapshots those happen to get replicated too

Yep - This is exactly what's going on.

So theoretically if I turned off replication then in 2 weeks my backup drives would be empty?

No, the data would stay on there until the next time you enabled replication, then the replication task would prune the backup.

 

[Cephyid]

OK got it. Pretty snazzy setup really.

I just gotta setup the cifs previous versions and I think I'm all set. I'll do a few more backups to externals and sleep well.

Thanks for the help. I really appreciate it.

 

[Marcet]

I found this thread very interesting, so let me revive it with some questions :

1. Could you clarify the "Recursive Snapshot" ? As I understand it : It applies to the specified dataset and the nested datasets. Is that right ?
2. Would you recommend to always take "Recursive Snapshop" in case of future sub-datasets creation.
3. What is your typical periodic snapshot plan for backup ? : Lifetime, Interval
4. Begin, End : Snapshots are more useful when data is supposed to change. So I suppose, Begin and End should match working hours ?
5. How do you typically manage stale snapshots : delete selected in the replication task or manually ?
6. How to configure Begin and End of replication : Is it better after working hours ? Or it can be done all day ?
7. What if replication occurs when scrub is made ? Any scheduling issue there ?

A lot of questions, sorry. But I want to do it the right way.

 

[depasseg]

Could you clarify the "Recursive Snapshot" ? As I understand it : It applies to the specified dataset and the nested datasets. Is that right ?

So a snapshot of a dataset, without the recursive option, will only create a snapshot of the data that exists in that dataset, but not the subdatasets. If you wanted snapshots of sub-datasets, you would need to create those separately, or alternatively, use the recursive option. Recursive works if you are happy having the main and sub datasets on the same schedules.

Would you recommend to always take "Recursive Snapshop" in case of future sub-datasets creation.

I do it that way for my setup, but that is because I'm fine with all the snapshots being taken at the same time, and they don't need to be different frequencies.

What is your typical periodic snapshot plan for backup ? : Lifetime, Interval

Begin, End : Snapshots are more useful when data is supposed to change. So I suppose, Begin and End should match working hours ?

The only reason I see to limit the hours, is to limit the number of snapshots. Some I have set for business hours, others go all day.

How do you typically manage stale snapshots : delete selected in the replication task or manually ?

That is the "keep snapshots for" column shown above.

How to configure Begin and End of replication : Is it better after working hours ? Or it can be done all day ?

I let mine run all the time. It will only run when there is a snapshot and the overhead isn't too much for my server or switches. If it became an issue, I'd restrict it to a slow time frame.

What if replication occurs when scrub is made ? Any scheduling issue there ?

I haven't noticed anything. I guess there could be an issue, but my scrubs are middle of a weekend slow period, so any snapshots to be replicated should be small and quick.

 

[Marcet]

Thanks a lot for this very clear answer.

How do you typically manage stale snapshots : delete selected in the replication task or manually ?

That is the "keep snapshots for" column shown above.

I meant : do you check the box "Delete slate snapshots on the remote system" ?

 

[depasseg]

Ahh, yes, I use the delete stale snapshots option. If you didn't use that, then you would need to delete manually. Or just watch your pool completely fill up and go boom.