Confused about Replication and Periodic Snapshots

[Toastr]

I have 2 Pools that are both of the same size but one is SSD and the other is HDD. I want to use the ssd as a primary drive with all its data copied over to the hdd pool to be redundent. I have a replication and snapshots task that takes snapshots on the ssd pool and replicates every hour to the hdd pool.

I am confused about how snapshots work. When the first snapshot is taken it contains all the data rigt? Then for every snapshot that is it taking all the data again or just data the have changed since the last snapshot? Also what does Snapshot lifetime mean? If it was the latter answer to my previous question does that mean that the data replicated over to my other pool will be deleted after the snapshot expires? In my replication task I have snapshot retention policy set to be the same as the source.

 

[sretalla]

A snapshot isn't the actual data, it's all the pointers to the blocks on the disks that are used at that moment to hold the content of the pool.

When you put new data on the disk, ZFS makes sure that any blocks involved in a snapshot are not modified, likewise when you delete something (blocks mentioned in snapshots aren't actually cleared).

When you take a new snapshot, it can reference a previous snapshot (like, all the blocks in snapshot1 and snapshot 2, plus these extra blocks that changed after snapshot 2).

That way, you can return to any point in time based on the snapshot's block information without actually having to make a copy of any data.

When you replicate a snapshot, it uses both of these points to avoid needing to send blocks that were already sent as part of a previous replication/snapshot and can send just the new ones to the other side.

When a snapshot expires (and the system does the background task of destroying the expired snapshots), the system stops protecting all the blocks, so they can be re-used/overwritten. If, at the time of destruction, a snapshot is referred to by another (or others), the blocks are transferred into the list of the depending snapshot itself, removing the dependency (but keeping the blocks reserved).

Snapshots are themselves part of the replication activity, so you will have snapshots (and all the related blocks) on both sides after each (successful) replication cycle.

Does that help to clear it up for you? Maybe do a bit of reading up on what snapshots are and how they work if it's still not clear:

Overview of ZFS Snapshots - Oracle Solaris Administration: ZFS File Systems

This book is intended for anyone responsible for setting up and administering Oracle ZFS file systems. Topics are described for both SPARC and x86 based systems, where appropriate.

docs.oracle.com

https://www.bsdcan.org/2019/schedule/attachments/500_How%20ZFS%20Snapshots%20Really%20Work.pdf

 

[Toastr]

A snapshot isn't the actual data, it's all the pointers to the blocks on the disks that are used at that moment to hold the content of the pool.

When you put new data on the disk, ZFS makes sure that any blocks involved in a snapshot are not modified, likewise when you delete something (blocks mentioned in snapshots aren't actually cleared).

When you take a new snapshot, it can reference a previous snapshot (like, all the blocks in snapshot1 and snapshot 2, plus these extra blocks that changed after snapshot 2).

That way, you can return to any point in time based on the snapshot's block information without actually having to make a copy of any data.

When you replicate a snapshot, it uses both of these points to avoid needing to send blocks that were already sent as part of a previous replication/snapshot and can send just the new ones to the other side.

When a snapshot expires (and the system does the background task of destroying the expired snapshots), the system stops protecting all the blocks, so they can be re-used/overwritten. If, at the time of destruction, a snapshot is referred to by another (or others), the blocks are transferred into the list of the depending snapshot itself, removing the dependency (but keeping the blocks reserved).

Snapshots are themselves part of the replication activity, so you will have snapshots (and all the related blocks) on both sides after each (successful) replication cycle.

Does that help to clear it up for you? Maybe do a bit of reading up on what snapshots are and how they work if it's still not clear:

Overview of ZFS Snapshots - Oracle Solaris Administration: ZFS File Systems

This book is intended for anyone responsible for setting up and administering Oracle ZFS file systems. Topics are described for both SPARC and x86 based systems, where appropriate.

docs.oracle.com

https://www.bsdcan.org/2019/schedule/attachments/500_How%20ZFS%20Snapshots%20Really%20Work.pdf

Thanks for your explanation really cleared up everything.