Confused re Replication

[Tim1962]

I sort of understand Snapshots. Essentially change files that "sit alongside" the mothership.

Replication takes these and copies to another location....

What I can't get my head around is - if the original dataset etc gets destroyed by, lets say an idiot admin or a local fire, if the replication destination only contains the snapshots and not the original dataset how can I use it to recover the whole ?

Or if I have misunderstood, what can I do to ensure that if my 60Gb Windows VM gets borked I can reinstate it

TIA Tim

 

[sretalla]

A snapshot isn't a copy at all.

The tables that store the locations of all files are pointers to block numbers on the disks in the pool.

When you take a snapshot, the block pointers from that table which are relevant to the dataset(s) in the snapshot scope are copied to a new "snapshot" version of that table.

At the same time, an update is made to the master table to indicate that those blocks are all now part of a snapshot. Any deletion or writing of new files will not touch any block that is part of a snapshot.

When you use the snapshot for replication, it will simply treat the source blocks for any copy from its own block list and ignore anything from the master table.

What appears on the other side of the replication is an exact (block-level) copy of the original master at the point the snapshot was taken, so the original files and all current snapshots still valid at that time are included on the remote side.

The replication of a destroyed dataset should fail (not finding the source anymore), so the deletion itself should not replicate.

You can then just set up a new replication in the opposite direction to put things back to the way you want it. (since replicated datasets are by default marked read-only, you will need to unset that flag at the end).

 

[Tim1962]

Hi Sretalla
You have explained it very eloquently thank you.

However I am still struggling to understand the practicalities in this example

60Gb VM on pool A
2 "local" snapshot regime set up, daily, then weekly for 3m. (That I can understand esp after your explanation)
Replication set up (in this specific case to a separate local disk, same box)
If I look at the destination dataset on pool B it is only 96kb used space.

If Original VM blows up, or whole disk set replaced etc, how can 96kb of the replication snapshot "know" how to recreate 60Gb on a replaced Disk A pool without Pool B holding a copy of the entire original VM somewhere.
I feel like the first replication task needs to "copy" the entire VM dataset across on first run, and then it all makes sense but mine doesn't appear to do this.
Really sorry, I can't find a suitable primer that unlocks this puzzle for me.

Many thanks in advance
Tim

 

[sretalla]

60Gb VM on pool A
2 "local" snapshot regime set up, daily, then weekly for 3m. (That I can understand esp after your explanation)
Replication set up (in this specific case to a separate local disk, same box)
If I look at the destination dataset on pool B it is only 96kb used space.

Are you 100% sure that replication is acting on the entire dataset (and children)? and is replicating successfully? (can you traverse the folder structure and see all the files in the target location?)

 

[Tim1962]

Are you 100% sure that replication is acting on the entire dataset (and children)? and is replicating successfully? (can you traverse the folder structure and see all the files in the target location?)

Not now you have asked !!!! Give me a moment to go into a dark corner and rock back and forth... (checking now)

 

[Tim1962]

Yep definitely rocking back and forth!
Currently replication is chuntering away to itself (a little slowly) but looks like it is heading towards a full dataset replication

Once I've recovered I will check but I think I was trying to replicate just the dataset rather than the enclosed VM zvol

Hopefully this thread will help some other idiot in future

FInal (hopefully!) clarifications
Is there any merit in snapshotting the dataset and zvol "inside" recursively, vs just the zvol itself.
Its currently replicating at around 1Mb / min which for 60Gb is going to take a while. Is that about right inside a single TrueNAS box?

Thanks so much for taking the time

 

[sretalla]

Is that about right inside a single TrueNAS box?

Certainly slower than expected... should be at least the read/write speed of one of the disks in the pool. (minimum 20MB/s)

How old are the REDs? Are you sure they aren't SMR? https://www.truenas.com/community/resources/list-of-known-smr-drives.141/

 

[Tim1962]

TBH no idea what was happening but finally with some trial and error got it (partially!) all going and 60Gb whizzed along and sorted in just a few minutes. Not really sure what I had done. So VM, job done, though I really ought to trial the restore process when I am brave...

The Jails elude me though. (One in particular where I haven't saved the data in a mounted dataset...)

With (extensive!) reading, it looks like there WAS a bug in jail replication in prior versions of FN (I'm on TN12 now) but its not clear whether iocage/jail replication currently is broken, reflects my incompetence, or indeed is just technically impossible. Currently they seem to snapshot OK which is probably belt enough for me but I'd quite like the replication braces.

Thanks again for being at my side on this

 

[sretalla]

I replicate my jail mounts/app data, but not the jails themselves. I'll recreate them from scripts if they are lost and mount the data back.

 

[Tim1962]

More confusion but success... I established a replication task on the root folderand ran it (randomly !!!) in full file system replication then recursive mode and suddenly it ran to completion. Was then able to fully restore so it had worked.

Although I am an idiot, I also feel there is a bug/feature that stopped it working for this particular idiot, though I'm not competent to identify or even describe it fully. The solution at some point may be an option for "do you want to do a full duplication on first run then incremental thereafter, OK I will do that for you" button

Best wishes, Happy Christmas, Tim