Errors replicating large (5TB) dataset

tentpiglet

Cadet
Joined
Aug 21, 2022
Messages
5
I am attempting to replicate 5TB of data from one truenas system to another. Both running latest versions/all updates.

I receive the following error, shown below.

On the receiving server, I have deleted all snapshots for that dataset, and then attempted the replication again. I get the same error.

On the receiving server, I deleted all the snapshots AND the dataset, then recreated the dataset, and attempted the replication again, I get the same error.

I am baffled. I have no idea why the system thinks there is a 'partially complete state' of a snapshot for a dataset I deleted completely from the system.

can someone give me guidance on how to fix this error so I can get the data to replicate to the other machine?


Error
Active side: destination pool1/VideoEvidence contains partially-complete state from "zfs receive -s".
Logs
[2023/01/11 12:27:11] INFO [replication_task__task_10] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2023/01/11 12:27:11] INFO [replication_task__task_10] [zettarepl.replication.run] For replication task 'task_10': doing push from 'pool1/VideoEvidence' to 'pool1/VideoEvidence' of snapshot='auto-2022-12-28_08-32' incremental_base='auto-2022-12-28_07-32' receive_resume_token=None encryption=False
[2023/01/11 12:27:11] INFO [replication_task__task_10] [zettarepl.paramiko.replication_task__task_10.sftp] [chan 5] Opened sftp connection (server version 3)
[2023/01/11 12:27:11] INFO [replication_task__task_10] [zettarepl.transport.ssh_netcat] Automatically chose connect address '192.168.1.3'
[2023/01/11 12:28:11] WARNING [replication_task__task_10.process] [zettarepl.transport.base_ssh] [ssh:root@192.168.1.3] [replication_process:task_10] Listen side has not terminated within 60 seconds after connect side success
[2023/01/11 12:28:11] INFO [replication_task__task_10] [zettarepl.replication.run] For replication task 'task_10': doing push from 'pool1/VideoEvidence' to 'pool1/VideoEvidence' of snapshot='auto-2022-12-28_09-32' incremental_base='auto-2022-12-28_08-32' receive_resume_token=None encryption=False
[2023/01/11 12:28:11] INFO [replication_task__task_10] [zettarepl.transport.ssh_netcat] Automatically chose connect address '192.168.1.3'
[2023/01/11 12:28:12] ERROR [replication_task__task_10] [zettarepl.replication.run] For task 'task_10' unhandled replication error SshNetcatExecException(None, ExecException(1, 'destination pool1/VideoEvidence contains partially-complete state from "zfs receive -s".\n'))
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 181, in run_replication_tasks
... 10 more lines ...
run_replication_step(step, observer)
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 763, in run_replication_step
ReplicationProcessRunner(process, monitor).run()
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/process_runner.py", line 33, in run
raise self.process_exception
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/process_runner.py", line 37, in _wait_process
self.replication_process.wait()
File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/ssh_netcat.py", line 212, in wait
raise SshNetcatExecException(None, self.listen_exec_error)
zettarepl.transport.ssh_netcat.SshNetcatExecException: Active side: destination pool1/VideoEvidence contains partially-complete state from "zfs receive -s".
 
Top