SOLVED Is a snapshot a complete backup?

[Borja]

I' planning to do backups to a remote FreeNAS system. So I read probably the best option is to create a replication task. But, at first, i created a snapshot and it only takes a small amount of Mb and in refer column some GBs but my volume has about 5TB space used.
So, this is great to rollback to an old version of data, but, what happened if NAS burns and all disk be destroyed? Can i recover data from remote snapshots?

 

[Borja]

Ok, I read again and they are not backups... so if array is damaged, i lost the data. What is the best way to perform a complete backup to other remote FreeNAS system then?

 

[danb35]

Ok, i read again and they are not backups

I think you read incorrectly--a snapshot replicated to another pool is a complete copy of the data referred to in that snapshot.

 

[Borja]

I think you read incorrectly--a snapshot replicated to another pool is a complete copy of the data referred to in that snapshot.

Ok, I read this reply: https://forums.freenas.org/index.php?threads/how-do-snapshots-work.10336/ and think snapshots in local and remote works the same way.

 

[Kevin Horton]

I use FreeNAS's recursive snapshot and replication functions to backup up my main server to a second server. The data on the second server matches the main server as far as I can tell. Every file I have looked for has always been there, and the space used is the same on both servers.

 

[Borja]

I use FreeNAS's recursive snapshot and replication functions to backup up my main server to a second server. The data on the second server matches the main server as far as I can tell. Every file I have looked for has always been there, and the space used is the same on both servers.

Thank you Kevin. Yes, i read documentation about how snapshot works and though that replication task only copy refer data of each snapshot.

So i dont clearly understand now... I keep snapshot for 2 week with 1 hour interval. What info is copied in remote with this configuration?

 

[Jailer]

Only the data that has changed since your last snapshot and only just the changes.

 

[danb35]

This is at least the second thread I've seen recently with this misconception. When you replicate a snapshot to another pool (whether on the same machine, on a different local machine, or on a machine on the other side of the world), you end up with all the data that existed in that "source" dataset at the time the snapshot was taken (including all its child datasets, if you set the snapshot to be recursive). And if you think about it for a minute, that's the only outcome that makes any sense. There's absolutely no purpose that would be served by doing it any other way. So, the snapshot itself may not be a backup, but the replicated copy of the snapshot certainly is.

 

[Borja]

This is at least the second thread I've seen recently with this misconception. When you replicate a snapshot to another pool (whether on the same machine, on a different local machine, or on a machine on the other side of the world), you end up with all the data that existed in that "source" dataset at the time the snapshot was taken (including all its child datasets, if you set the snapshot to be recursive). And if you think about it for a minute, that's the only outcome that makes any sense. There's absolutely no purpose that would be served by doing it any other way. So, the snapshot itself may not be a backup, but the replicated copy of the snapshot certainly is.

Thank you danb35 for your clarification. So, if i set periodic snapshot with 1 hour interval and keep for two weeks how data is replicated to the remote pool? It copies data with 2 weeks modified date or older and every snapshot with metadata since then?or how?

Thanks

 

[danb35]

how data is replicated to the remote pool?

I believe (but I'm not certain) that the oldest snapshot is replicated first, and then any subsequent ones. It has nothing to do with the modification date of any files; snapshots (and thus replication) are a block-level process.

 

[styno]

Exactly, it is replicating the difference in snapshots one by one, oldest first.
I suggest OP to read a bit more on COW-filesystems and how they implement snapshots.

 

[Borja]

I believe (but I'm not certain) that the oldest snapshot is replicated first, and then any subsequent ones. It has nothing to do with the modification date of any files; snapshots (and thus replication) are a block-level process.

This is my doubt. Now is replicating data, when its completed, the second snapshot will be only metadata (I suppose). But, because I only keep snapshots 2 weeks. I think data needs to be replicated some time... (I'm talking about remote)

 

[danb35]

I think data needs to be replicated some time.

Of course it does, and it is. When you first replicate, you transfer all the data (along with the relevant metadata). With any subsequent replication, you transfer any changes to the data, along with their relevant metadata.

 

[styno]

the second snapshot will be only metada (i suppose).

You can see a snapshot as a 'picture' of how the filesystem 'looked' like at that point in time, basically a bunch of pointers to the correct blocks, that's why a snapshot is instant.
ZFS being a COW based filesystem means that once you start altering the blocks (altering being writing different data to the blocks) the snapshot will contain both the pointers AND the original data (and will grow over time).

 

[Borja]

This is my doubt. Now is replicating data, when its completed, the second snapshot will be only metada (i suppose). But, because i only keep snapshots 2 weeks. I think data needs to be replicated some time...

Of course it does, and it is. When you first replicate, you transfer all the data (along with the relevant metadata). With any subsequent replication, you transfer any changes to the data, along with their relevant metadata.

This is great, very good backup system. would be great to have a way to extract concrete files instead of rollback or clone entire snapshot to be perfect, but its very good.

 

[danb35]

would be great to have a way to extract concrete files

There is such a way. Read my post up-thread, or @jgreco's post in the thread you linked.

 

[styno]

But you can access a snapshot directly and restore individual files!
Have a look in /mnt/yourpool/.zfs/snapshot.

 

[Kevin Horton]

Thank you Kevin. Yes, i read documentation about how snapshot works and though that replication task only copy refer data of each snapshot.

So i dont clearly understand now... I keep snapshot for 2 week with 1 hour interval. What info is copied in remote with this configuration?

Others have answered this question, but I'll illustrate with my setup and the outcome.

I've got several recursive snapshots set up at various intervals and retentions: hourly snapshots with 2 day retention, daily snapshots with a 30 day retention, and every four week snapshots with a 1 year retention.

The replication task to the backup server is set up to run from 1830 to 0630, so it occurs when the network that the backup server is on will be less busy. After the replication was set up, the first run took a long time, as a full copy of the data and all snapshots was copied over. Every day at 1830, when it starts back up again, it takes a few minutes, as all the snapshots from 0630 to 1830 are copied. The task runs again every hour, and copies the last hourly snapshots, which are usually very small, unless some large file has been added or modified.

If I look at the backup server, I see that it has the same long list of snapshots as the main server.

I tested what happens if communication between the two servers is lost for several days. In this case, the replication picks up where it left off, and sends over all the snapshots that were made in the period when communication was lost.

I've got also got a backup pool on my main server, and use replication to keep it matching the data on the main pool.

 

[Borja]

Thank you everyone, this is now more clear to me.

 

[Evertb1]

I' planning to do backups to a remote FreeNAS system. So I read probably the best option is to create a replication task. But, at first, i created a snapshot and it only takes a small amount of Mb and in refer column some GBs but my volume has about 5TB space used.
So, this is great to rollback to an old version of data, but, what happened if NAS burns and all disk be destroyed? Can i recover data from remote snapshots?

To take away any doubt you still have: You will find that a replication job to a remote server transfer all your data included in the job.. Past weekend I needed to do some work on my main file server. But as I did not know how long it would take, I used one of my lab computers as a temp file server (with Freenas of course). The replication worked without a flaw. After an overnight execution of the replication tasks (one for each dataset in my one and only volume) my temp server was ready for use (after I created the needed samba shares). I used some scripts to alter the mappings on the windows workstations to point to the temp server. And I was in business again. Your current snapshots will be also transferred to the receiving machine. And the replication job will create a new snapshot task on the receiving server as part of the process.