Zandr Milewski
Dabbler
- Joined
- Oct 5, 2013
- Messages
- 13
For a several years now I've been running a Thunderbolt 2 RAID0 (4x4TB) connected to my Mac (SoftRAID), backed up to a FreeNAS box with a RAIDZ2 (6x4TB). A different project here opened my eyes to how performant ZFS replication is compared to filesystem backups, so I'm now experimenting with OpenZFS on OS X (O3X). After making a second backup on local storage, I've rebuilt the Thunderbolt array as a 4x4TB stripe in O3X, and am currently replicating the FreeNAS backup back to the TB array.
[EDIT: I'm driving this replication as a "PUSH" from the FreeNAS box.]
This FreeNAS box is old and slow, however (Turion N40L Microserver), so I'm only seeing about 45MB/s. CPU usage is in the 60% range (dual core), which makes me think that I'm CPU bound by a single ssh thread doing encryption. Enter SSH+Netcat, right? That earlier project included an identical box running happily at wire speed with encryption turned off.
Unfortunately, that doesn't appear to work. I've verified that nc is in the path when connecting with remote commands over ssh (zfs wasn't, initially), so I'm not sure how to troubleshoot further. Here's the full log:
[EDIT: I'm driving this replication as a "PUSH" from the FreeNAS box.]
This FreeNAS box is old and slow, however (Turion N40L Microserver), so I'm only seeing about 45MB/s. CPU usage is in the 60% range (dual core), which makes me think that I'm CPU bound by a single ssh thread doing encryption. Enter SSH+Netcat, right? That earlier project included an identical box running happily at wire speed with encryption turned off.
Unfortunately, that doesn't appear to work. I've verified that nc is in the path when connecting with remote commands over ssh (zfs wasn't, initially), so I'm not sure how to troubleshoot further. Here's the full log:
Code:
[2020/03/14 10:37:51] INFO [Thread-7] [zettarepl.paramiko.replication_task__task_3] Connected (version 2.0, client OpenSSH_8.1) [2020/03/14 10:37:51] INFO [Thread-7] [zettarepl.paramiko.replication_task__task_3] Authentication (publickey) successful! [2020/03/14 10:37:51] INFO [replication_task__task_3] [zettarepl.replication.run] For replication task 'task_3': doing push from 'Deckard/Photographs' to 'ThunderBay' of snapshot='auto-2020-03-14_10-37' incremental_base=None receive_resume_token=None [2020/03/14 10:37:51] INFO [replication_task__task_3] [zettarepl.paramiko.replication_task__task_3.sftp] [chan 4] Opened sftp connection (server version 3) [2020/03/14 10:37:51] ERROR [replication_task__task_3] [zettarepl.replication.run] For task 'task_3' unhandled replication error JSONDecodeError('Expecting value: line 1 column 1 (char 0)') Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/zettarepl/replication/run.py", line 139, in run_replication_tasks run_replication_task_part(replication_task, source_dataset, src_context, dst_context, observer) File "/usr/local/lib/python3.7/site-packages/zettarepl/replication/run.py", line 200, in run_replication_task_part run_replication_steps(step_templates, observer) File "/usr/local/lib/python3.7/site-packages/zettarepl/replication/run.py", line 372, in run_replication_steps replicate_snapshots(step_template, incremental_base, snapshots, observer) File "/usr/local/lib/python3.7/site-packages/zettarepl/replication/run.py", line 427, in replicate_snapshots run_replication_step(step_template.instantiate(incremental_base=incremental_base, snapshot=snapshot), observer) File "/usr/local/lib/python3.7/site-packages/zettarepl/replication/run.py", line 481, in run_replication_step ReplicationProcessRunner(process, monitor).run() File "/usr/local/lib/python3.7/site-packages/zettarepl/replication/process_runner.py", line 22, in run self.replication_process.run() File "/usr/local/lib/python3.7/site-packages/zettarepl/transport/ssh_netcat.py", line 128, in run listen = self.listen_exec.head(self._parse_listen_exec, 10) File "/usr/local/lib/python3.7/site-packages/zettarepl/transport/async_exec_tee.py", line 56, in head result = callback(event.data) File "/usr/local/lib/python3.7/site-packages/zettarepl/transport/ssh_netcat.py", line 199, in _parse_listen_exec return json.loads(data) File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Last edited: