"cannot create snapshot - dataset already exists" after creating replication task

[danb35]

I've had my main FreeNAS/TrueNAS system running for a number of years, and had regular snapshot tasks running without issues. I'm now setting up a backup server, and replicating a few datasets to that. Everything seems to be working fine--the replication is taking place, and the main system is reporting that the last snapshot replicated is in fact the last snapshot that was created.

Everything, that is, except that I'm getting an alert email every time the replication task runs, saying:

Code:

New alert:
* Snapshot Task For Dataset "tank/apps" failed: cannot create snapshot 'tank/apps@auto-2021-09-27_17-00': dataset already exists..

There is a snapshot of tank/apps@auto-2021-09-27_17-00, and the GUI reports that that snapshot has been sent to the target system. What would be causing this?

Both systems run TrueNAS CORE 12.0-U5.1. Main system hardware is in my .sig; backup system is a Microserver Gen8, Xeon E3-1220L, 12GB RAM, 1 TB spinner in a single-disk pool (which will be replaced with a mirrored pair of 16 TB disks once they finish burning in).

 

[winnielinnie]

Are there any newer snapshots after @auto-2021-09-27_17-00 when the subsequent replications run?

 

[danb35]

The alert I got 8 minutes ago:

Existing snapshots on the source system:

...and on the target system:

In case the problem had been caused by the replication task trying to run at the same time as the snapshot, I set the task to run at the bottom of the hour. It doesn't seem to be helping.

 

[Basil Hendroff]

Check under Periodic Snapshot Tasks. This alert appears if you try to run overlapping snapshot tasks e.g. a snapshot task for tank/media and a separate task for tank/media/photos.

 

[Spearfoot]

I've had my main FreeNAS/TrueNAS system running for a number of years, and had regular snapshot tasks running without issues. I'm now setting up a backup server, and replicating a few datasets to that. Everything seems to be working fine--the replication is taking place, and the main system is reporting that the last snapshot replicated is in fact the last snapshot that was created.

Everything, that is, except that I'm getting an alert email every time the replication task runs, saying:

Code:

New alert:
* Snapshot Task For Dataset "tank/apps" failed: cannot create snapshot 'tank/apps@auto-2021-09-27_17-00': dataset already exists..

There is a snapshot of tank/apps@auto-2021-09-27_17-00, and the GUI reports that that snapshot has been sent to the target system. What would be causing this?

Both systems run TrueNAS CORE 12.0-U5.1. Main system hardware is in my .sig; backup system is a Microserver Gen8, Xeon E3-1220L, 12GB RAM, 1 TB spinner in a single-disk pool (which will be replaced with a mirrored pair of 16 TB disks once they finish burning in).

Do you have the same replication tasks configured on the backup server as on your primary server? I've been bitten by that before...

 

[danb35]

Do you have the same replication tasks configured on the backup server as on your primary server?

No, there are no replication tasks configured on the backup server.

This alert appears if you try to run overlapping snapshot tasks e.g. a snapshot task for tank/media and a separate task for tank/media/photos.

Hmmm. I wonder if the replication task added a snapshot task. Because I don't think I had overlapping snapshot tasks before, but it looks like I do now:

I've already deleted a second tank/apps tasks with an "error" status. I'll see what happens in an hour.

 

[danb35]

I wonder if the replication task added a snapshot task.

Indeed it did, and I think that's the issue here.

As the screen shot above shows, I have a snapshot task for the whole pool. Also shown in the screen shot is that it's recursive--so it's taking snapshots of tank, tank/apps, tank/emily-tm, and all the other sub-datasets. I'd expected that the replication task would work with an existing snapshot task, but that isn't what happens--when I schedule the replication task, it schedules a new snapshot task. And since all of iX' default schedules are right on the hour, with no randomness, the two tasks want to create a snapshot with the same name, causing the error I'm seeing.

As a bonus, deleting the replication task doesn't delete the newly-added snapshot task, nor does deleting the newly-added snapshot task affect the replication task [see below]--just in case you thought it might behave consistently.

Not sure the best way to address this. I'm thinking it's to recreate the replication tasks, but delete the snapshot tasks they create--they'll then replicate the snapshots created by the recursive snapshot task on tank.

Edit: No, that doesn't work. I thought I'd been able to delete the newly-added snapshots previously, but if I try now, I get this:

So I guess the second option is to delete the snapshot task for the root dataset (or at least make it non-recursive), and set up individual snapshot tasks for each of the datasets under tank. Yuck.

Or a third option is to just ignore the alerts--the snapshots are getting created, and they are being replicated to the target system. So the issue is pretty much cosmetic, but still irritating.