How does snapshot/replication work? Should I do more?

[Matt Tyree]

I set up a snapshot on my main NAS and a replication task to my secondary NAS. Are the snapshots really THIS small? Doesn't make much sense to me.
I did a manual snapshot first, am I reading this right? Only a 17.5GB snapshot for 2.5TB of data?


As I said, I am replicating these snapshots to my secondary NAS and the secondary shows these same snapshots, so that's working.

I just wonder, should I also be copying the actual files to the second NAS and not just the snapshots? I have a 4TB and 2TB volumes on the second NAS, so I have the space if the snapshots are that tiny.

Thanks!

 

[Robert Trevellyan]

No, you aren't reading it correctly, but that's mostly because that output is utterly cryptic. Take a look at the Storage tab to see how much space is actually in use on the secondary. Hint: it should match the primary.

When you replicate a snapshot, you are replicating the state of the entire filesystem at the moment the snapshot was taken. Incremental snapshot replication only transmits the changes since the previously replicated snapshot.

 

[depasseg]

Only a 17.5GB snapshot for 2.5TB of data?

Not quite. In this case, it's more like that snapshot contains 17.5GB of data that has been deleted or modified between the last snapshot and the next snapshot.

I just wonder, should I also be copying the actual files to the second NAS and not just the snapshots?

If you use the FreeNAS replication option, then you are fine. Replication will copy all snapshots (that's actually all that Replication copies, it doesn't copy the "live" data").

ZFS snapshots are a confusing topic. Initially they consume no disk space, but they starts to consume disk space as the files the snapshot contains get modified or deleted. Those modifications and deletions are what that snapshot size means.

 

[Matt Tyree]

No, you aren't reading it correctly, but that's mostly because that output is utterly cryptic. Take a look at the Storage tab to see how much space is actually in use on the secondary. Hint: it should match the primary.

You're correct, it does: 2.5TB

ZFS snapshots are a confusing topic. Initially they consume no disk space, but they starts to consume disk space as the files the snapshot contains get modified or deleted. Those modifications and deletions are what that snapshot size means.

I can see why the second, automatic snapshot would have no changes, but the first, manual one should, right? I ran that snapshot directly after copying all my data to the drives. It would seem to me that would be a snapshot of everything. In essence, it was all new. That's what confuses me about that 17.5GB. I guess I just have to trust it. Just worries me not having any idea how it works or what to do with the snapshots if I ever need them. I'm such a noob! :-D

The other thing I need to figure out how to work around is the fact that I have six 4TB drives on my main NAS, only 2.5TB of which is currently being used, but that will change. I have a 4TB mirrored volume and a 2TB mirrored volume on the offsite NAS (two 4TB drives and two 2TB drives). Is there any way to combine the 4TB and 2TB volumes so that I can effectively have 6TB to copy the snapshots to?

 

[Robert Trevellyan]

It would seem to me that would be a snapshot of everything.

You have to stop thinking of a snapshot as a copy of the data. ZFS is a copy-on-write filesystem. Usually, when you modify data, the original blocks are marked as available for reuse. When you create a snapshot, you tell ZFS that all currently in use blocks are not candidates for reuse. That's why a snapshot only starts to take up space when further modifications are made, because those changes consume new blocks but the original blocks don't get released for re-use.

 

[depasseg]

but the first, manual one should, right

See the 2.5TB Refer column? That's the amount of data in the dataset that it uses. If you were to clone that snapshot, it would take the 2.5TB from the referred dataset and add the changes captured in the 17.5GB snapshot and voila!

 

[Robert Trevellyan]

Is there any way to combine the 4TB and 2TB volumes so that I can effectively have 6TB to copy the snapshots to?

The only way to do this is to destroy both pools (volumes) and build a new pool with two vdevs, one pair of 2TB and one pair of 4TB. With the FreeNAS GUI, you have to do this in stages using the Manual option.

Correction: I suppose you could also make a RAIDZ1 pool out of all four disks.

 

[Matt Tyree]

See the 2.5TB Refer column? That's the amount of data in the dataset that it uses. If you were to clone that snapshot, it would take the 2.5TB from the referred dataset and add the changes captured in the 17.5GB snapshot and voila!

Yep, I noticed the 2.5TB Refer column.
So, there was 17.5GB of changes between when I copied the data and when I ran the snapshot? I wonder what that was. Weird.
At any rate, I see what you guys mean now. I just have an archaic way of doing things. I still use SyncBack to backup my data to the FreeNAS box. Old habits...All this NAS replication stuff is totally new to me. :-D

And don't even get me started on trying to get a VPN to work. I ended up just running SSH and HTTP (couldn't get HTTPS to work, either) on non-standard ports so I could have remote access.

 

[Matt Tyree]

The only way to do this is to destroy both pools (volumes) and build a new pool with two vdevs, one pair of 2TB and one pair of 4TB. With the FreeNAS GUI, you have to do this in stages using the Manual option.

Correction: I suppose you could also make a RAIDZ1 pool out of all four disks.

Gotcha. I'll probably just let it ride for now and upgrade the storage capacity as I have the time/money for it.

 

[depasseg]

So, there was 17.5GB of changes between when I copied the data and when I ran the snapshot?

NO, there was 17.5GB of changes AFTER you took the snapshot.
When you take a snapshot, you are just telling ZFS "freeze the filesystem as it exists at this very moment, and save all changes separately, somewhere else". Those separate/somewhere else changes are what make up the 17.5GB.

 

[Matt Tyree]

NO, there was 17.5GB of changes AFTER you took the snapshot.

OOOOooohhhhhh <click> :-D
I get it now!
Thank you!!!

 

[Stux]

The only way to do this is to destroy both pools (volumes) and build a new pool with two vdevs, one pair of 2TB and one pair of 4TB. With the FreeNAS GUI, you have to do this in stages using the Manual option.

Correction: I suppose you could also make a RAIDZ1 pool out of all four disks.

You only need to destroy one of the pools. You can then extend the other pool with the now freed mirrored pair.

 

[nojohnny101]

@Matt Tyree Let me just ask to make sure since you are just learning about replication. You have on periodic snapshot setup at the top level and have the "recursive" option checked?

One way you can check to make sure all data is being copied is to clone a snapshot on the PULL target and then share it. Browse and make sure all your data is there (sub-folders and all). To be extra sure, you can note a change you make (drop a random 2gb file or whatever) and then wait for the snapshot to complete. Go to you PULL and do the steps above (clone, share, browse) and make sure it got sent over. Might give you a little piece of mind. NOTE: it is generally not recommended to "rollback" when recovering files.

 

[Matt Tyree]

Let me just ask to make sure since you are just learning about replication. You have on periodic snapshot setup at the top level and have the "recursive" option checked?

I do have a periodic snapshot setup for once a day (my backups only run once a day, so no need for more than that), but I do not have the recursive option checked. Should I change that? I wasn't quite sure how to interpret what recursive did.

I think it's working correctly. I don't have access to the shares via Windows as I don't have a VPN set up. The dataset shows the correct size, though. And the replication task has been running all day to copy over last night's snapshot (I dumped a bunch of 4K video that was backed up).

You only need to destroy one of the pools. You can then extend the other pool with the now freed mirrored pair.

Interesting. I may need to try that. So, I guess since I already have some snapshots replicated to the 4TB pool, I should destroy the 2TB pool and then extend the 4TB pool. Will that keep the snapshots, etc intact?

 

[depasseg]

Should I change that? I wasn't quite sure how to interpret what recursive did.

Most likely, yes, you want that option checked. Otherwise, the sub-dataset (children) won't get snapshotted which means they won't get replicated. If you don't check the recursive box, then you would need to manually create a job for each child dataset.

 

[Matt Tyree]

Most likely, yes, you want that option checked. Otherwise, the sub-dataset (children) won't get snapshotted which means they won't get replicated. If you don't check the recursive box, then you would need to manually create a job for each child dataset.

Ah, I see. In my case there are no child datasets, YET. So, I'm good for now, but I will check that option.

 

[Stux]

I do have a periodic snapshot setup for once a day (my backups only run once a day, so no need for more than that), but I do not have the recursive option checked. Should I change that? I wasn't quite sure how to interpret what recursive did.

I think it's working correctly. I don't have access to the shares via Windows as I don't have a VPN set up. The dataset shows the correct size, though. And the replication task has been running all day to copy over last night's snapshot (I dumped a bunch of 4K video that was backed up).


Interesting. I may need to try that. So, I guess since I already have some snapshots replicated to the 4TB pool, I should destroy the 2TB pool and then extend the 4TB pool. Will that keep the snapshots, etc intact?

Yes, the ones on the 4TB pool

If you needed to get the data off the 2TB and onto the new extended pool there's a way to do that too

1) split the 2TB Pool [cli]. (This reduces redundancy)
2) extend the 4TB with a single 2TB drive (this reduces redundancy)
3) copy data from 2TB pool to 6TB pool
4) add remaining 2TB drive to striped 2TB (this restores redundancy)