Pool Replication stops and erros

leothaes

Cadet
Joined
Jun 7, 2022
Messages
4
HP DC7900 convertable tower
Processor: Intel Core 2 Duo E8400 3GHz
Ram: 8GB DDR2
Addon cards: 1 PCI 4 port SATA controller, 1 PCI-E x1 4 port SATA controller
Storage: 3 4TB Seagate NAS HDD, 7 1TB (Mixed between WD Blue and Seagate Barracuda) HDD, 1 Intel 180GB SSD, 1 Seagate 160GB HDD for Boot.

Hi Guys,

Really appreciate some assistance. I'm running TrueNAS-12.0-U8.1. I have a stripped pool (I know stripped is no good, but at the time I just needed to off load the data to the NAS, so I needed the capacity) of 7 1TB 3.5" HDDs for total usable capacity of 6.2TB, 5.4TB of which are currently in use which we will call pool1. I recently purchased 3 4TB Seagate NAS HDDs and created a new pool using Raidz for a total of 7.14GB usable, which we will call pool2, to migrate the previous pool over to. I have also added a 180GB SSD I had lying around as a Log for pool2.

I created a periodic snapshot task for pool 1, which created a few snapshots. I have since disabled the task, since I am not currently making changing to the data on the pool.

I next created a replication task. The source selected is the dataset on pool 1 and the destination selected is pool2 and a added a new dataset name "Archive 100X". I selected recursive and also "Use customer snapshot", and set it to run once, since I planned on doing this on demand.

Running the replication task works for a few minutes and copied 752.47 GiB before stopping and showing an error under tasks->Replication tasks. I've copied the log below at the end of this post.

I've tried running the replication task again, but it errors immediately. Deleting the dataset on pool2 and starting the replication again results in it starting and stopping at the roughly the same amount of data copied.

Warning Log for Replication Task:
warning: cannot send 'Archive/Storage X@auto-2022-06-21_16-14': Input/output error
cannot receive resume stream: checksum mismatch or incomplete stream.
Partially received snapshot is saved.
A resuming stream can be generated on the sending system by running:
zfs send -t 1-1079bfb66c-f0-789c636064000310a500c4ec50360710e72765a526973030b43b43d460c8a7a515a7968064b44260f26c48f2499525a9c5407a41937dfb1e2cfa4bf2d34b33531818a2f7972eb9cf1e93ef8024cf0996cf4bcc4d6560702c4acec82c4bd50f2ec92f4a4c4f558870482c2dc9d735323032d23530d335328c3734d33534019bcfcd80f04f727e6e41516a71717e36031c00008b50252f.

Logs​

[2022/06/22 10:24:18] INFO [replication_task__task_1] [zettarepl.replication.run] Resuming replication for destination dataset 'Archive 100X/Archive 100X'
[2022/06/22 10:24:18] INFO [replication_task__task_1] [zettarepl.replication.run] For replication task 'task_1': doing push from 'Archive/Storage X' to 'Archive 100X/Archive 100X' of snapshot=None incremental_base=None receive_resume_token='1-10545aa686-f0-789c636064000310a500c4ec50360710e72765a526973030b43b43d460c8a7a515a7968064b44260f26c48f2499525a9c540fa46ad6dfb1e2cfa4bf2d34b33531818a2f7972eb9cf1e93ef8024cf0996cf4bcc4d6560702c4acec82c4bd50f2ec92f4a4c4f558870482c2dc9d735323032d23530d335328c3734d33534019bcfcd80f04f727e6e41516a71717e36031c0000a6252560' encryption=False
[2022/06/22 10:24:18] ERROR [replication_task__task_1] [zettarepl.replication.run] For task 'task_1' unhandled replication error ExecException(255, "warning: cannot send 'Archive/Storage X@auto-2022-06-21_16-14': Input/output error\ncannot receive resume stream: checksum mismatch or incomplete stream.\nPartially received snapshot is saved.\nA resuming stream can be generated on the sending system by running:\n zfs send -t 1-1079bfb66c-f0-789c636064000310a500c4ec50360710e72765a526973030b43b43d460c8a7a515a7968064b44260f26c48f2499525a9c5407a41937dfb1e2cfa4bf2d34b33531818a2f7972eb9cf1e93ef8024cf0996cf4bcc4d6560702c4acec82c4bd50f2ec92f4a4c4f558870482c2dc9d735323032d23530d335328c3734d33534019bcfcd80f04f727e6e41516a71717e36031c00008b50252f\n")
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 164, in run_replication_tasks
retry_stuck_replication(
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/stuck.py", line 18, in retry_stuck_replication
return func()
File "/usr/local/lib/python3.9/site-packages/zettarepl/replication/run.py", line 165, in
lambda: run_replication_task_part(replication_task, source_dataset, src_context, dst_context,
... 9 more lines ...
self.replication_process.wait()
File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/local.py", line 164, in wait
self.async_exec.wait()
File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/async_exec_tee.py", line 103, in wait
raise ExecException(exit_event.returncode, self.output)
zettarepl.transport.interface.ExecException: warning: cannot send 'Archive/Storage X@auto-2022-06-21_16-14': Input/output error
cannot receive resume stream: checksum mismatch or incomplete stream.
Partially received snapshot is saved.
A resuming stream can be generated on the sending system by running:
zfs send -t 1-1079bfb66c-f0-789c636064000310a500c4ec50360710e72765a526973030b43b43d460c8a7a515a7968064b44260f26c48f2499525a9c5407a41937dfb1e2cfa4bf2d34b33531818a2f7972eb9cf1e93ef8024cf0996cf4bcc4d6560702c4acec82c4bd50f2ec92f4a4c4f558870482c2dc9d735323032d23530d335328c3734d33534019bcfcd80f04f727e6e41516a71717e36031c00008b50252f
 
Top