I tried replication instead of teracopy to copy ~10TB of files to a new server and I get errors... Any suggestions would be greatly appreciated. It looks like it is trying to unmout the source and destination volumes (?!) and it cannot. This is the first time I attempt to use replication so it is possible / likely / I am doing something wrong. TrueNAS 13.0-U4 on both servers (G10 microservers from HP with AMD quad cores), 32GB EEC RAM, 128GB boot SSD on both, zraid 3 for both source and destination volumes, both servers connected to the same switch. Using root with no encryption for ssh (home network, personal but not critical data).
[2023/04/05 09:10:20] INFO [Thread-44] [zettarepl.paramiko.replication_task__task_1] Connected (version 2.0, client OpenSSH_8.8-hpn14v15)
[2023/04/05 09:10:20] INFO [Thread-44] [zettarepl.paramiko.replication_task__task_1] Authentication (publickey) successful!
[2023/04/05 09:10:20] INFO [replication_task__task_1] [zettarepl.retention.calculate] Not destroying 'auto-2023-04-05_00-00' as it is the only snapshot left for naming schema 'auto-%Y-%m-%d_%H-%M'
[2023/04/05 09:10:20] INFO [replication_task__task_1] [zettarepl.retention.calculate] Not destroying 'auto-2023-04-05_00-00' as it is the only snapshot left for naming schema 'auto-%Y-%m-%d_%H-%M'
[2023/04/05 09:10:20] INFO [replication_task__task_1] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2023/04/05 09:10:20] WARNING [replication_task__task_1] [zettarepl.replication.run] No incremental base for replication task 'task_1' on dataset 'zraid3_9x8T/av', destroying destination dataset
[2023/04/05 09:10:21] ERROR [replication_task__task_1] [zettarepl.replication.run] For task 'task_1' unhandled replication error ExecException(1, "cannot unmount '/mnt/zr3-5x8Toshiba/a/h': pool or dataset is busy\ncannot unmount '/mnt/zr3-5x8Toshiba/a': pool or dataset is busy\n")
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 181, in run_replication_tasks
retry_stuck_replication(
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/stuck.py", line 18, in retry_stuck_replication
return func()
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 182, in <lambda>
lambda: run_replication_task_part(replication_task, source_dataset, src_context, dst_context,
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 279, in run_replication_task_part
run_replication_steps(step_templates, observer)
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 542, in run_replication_steps
step_template.dst_context.shell.exec(["zfs", "destroy", "-r", step_template.dst_dataset])
File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/interface.py", line 92, in exec
return self.exec_async(args, encoding, stdout).wait(timeout)
File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/local.py", line 80, in wait
raise ExecException(self.process.returncode, stdout)
zettarepl.transport.interface.ExecException: cannot unmount '/mnt/zr3-5x8Toshiba/a/h': pool or dataset is busy
cannot unmount '/mnt/zr3-5x8Toshiba/a': pool or dataset is busy
[2023/04/05 09:10:20] INFO [Thread-44] [zettarepl.paramiko.replication_task__task_1] Connected (version 2.0, client OpenSSH_8.8-hpn14v15)
[2023/04/05 09:10:20] INFO [Thread-44] [zettarepl.paramiko.replication_task__task_1] Authentication (publickey) successful!
[2023/04/05 09:10:20] INFO [replication_task__task_1] [zettarepl.retention.calculate] Not destroying 'auto-2023-04-05_00-00' as it is the only snapshot left for naming schema 'auto-%Y-%m-%d_%H-%M'
[2023/04/05 09:10:20] INFO [replication_task__task_1] [zettarepl.retention.calculate] Not destroying 'auto-2023-04-05_00-00' as it is the only snapshot left for naming schema 'auto-%Y-%m-%d_%H-%M'
[2023/04/05 09:10:20] INFO [replication_task__task_1] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2023/04/05 09:10:20] WARNING [replication_task__task_1] [zettarepl.replication.run] No incremental base for replication task 'task_1' on dataset 'zraid3_9x8T/av', destroying destination dataset
[2023/04/05 09:10:21] ERROR [replication_task__task_1] [zettarepl.replication.run] For task 'task_1' unhandled replication error ExecException(1, "cannot unmount '/mnt/zr3-5x8Toshiba/a/h': pool or dataset is busy\ncannot unmount '/mnt/zr3-5x8Toshiba/a': pool or dataset is busy\n")
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 181, in run_replication_tasks
retry_stuck_replication(
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/stuck.py", line 18, in retry_stuck_replication
return func()
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 182, in <lambda>
lambda: run_replication_task_part(replication_task, source_dataset, src_context, dst_context,
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 279, in run_replication_task_part
run_replication_steps(step_templates, observer)
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 542, in run_replication_steps
step_template.dst_context.shell.exec(["zfs", "destroy", "-r", step_template.dst_dataset])
File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/interface.py", line 92, in exec
return self.exec_async(args, encoding, stdout).wait(timeout)
File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/local.py", line 80, in wait
raise ExecException(self.process.returncode, stdout)
zettarepl.transport.interface.ExecException: cannot unmount '/mnt/zr3-5x8Toshiba/a/h': pool or dataset is busy
cannot unmount '/mnt/zr3-5x8Toshiba/a': pool or dataset is busy