SSH connection works but no replication

[nojohnny101]

I have 2 systems running the just released TrueNAS CORE (they were previously on 11.3-U5 and this problem still existed). They are in 2 physical locations and I previously had replication setup. About a month ago the motherboard died (bit by the ASROCK BMC chip problem) so I replaced the motherboard in the target system just fine, no problems there.

Trying to revive the replication task though has become head scratching. It seems the replication engine has undergone quite a bit of upgrades and improvement from when I first set it up. So I thought I would setup a new replication task that would use the existing snapshots that already exist, but it is telling me authentication failed.

I have follow the manual successfully created an SSH key pair. I then created an SSH connection using those key pairs. When creating the SSH connection I can successfully discover the remote host key, which to me means that is successfully connecting to the target TrueNAS CORE system. However when I got to setup a replication task using the SSH connection, it fails when trying to browse the remote file system, telling me authentication failed.

[EACCES] AuthenticationException('Authentication failed.')

What am I missing?

 

[nojohnny101]

Bump, has anyone else encountered this?

 

[nojohnny101]

No one is using the new replication engine with TrueNAS 12? Has anyone been successful in transitioning from the old snapshot system to the new snapshot engine without having to destroy previous replication jobs?

 

[Tabmowtez]

For what it's worth I can't get this working either.
I've generated new keys, new connections etc, and it always comes up with your same error.
I do get the following from the /var/log/middlewared.log on the source system when I try and select a destination pool though:

Code:

[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - starting thread (client mode): 0x625addc0
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - Local version/idstring: SSH-2.0-paramiko_2.7.1
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - Remote version/idstring: SSH-2.0-OpenSSH_8.2-hpn14v15
[2020/12/30 17:04:19] (INFO) paramiko.transport._log():1819 - Connected (version 2.0, client OpenSSH_8.2-hpn14v15)
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - kex algos:['curve25519-sha256', 'curve25519-sha256@libssh.org', 'ecdh-sha2-nistp256', 'ecdh-sha2-nistp384', 'ecdh-sha2-nistp521', 'diffie-hellman-group-exchange-sha256', 'diffie-hellman-group16-sha512', 'diffie-hellman-group18-sha512', 'diffie-hellman-group14-sha256'] server key:['rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa', 'ecdsa-sha2-nistp256', 'ssh-ed25519'] client encrypt:['chacha20-poly1305@openssh.com', 'aes128-ctr', 'aes192-ctr', 'aes256-ctr', 'aes128-gcm@openssh.com', 'aes256-gcm@openssh.com', 'aes128-cbc', 'none'] server encrypt:['chacha20-poly1305@openssh.com', 'aes128-ctr', 'aes192-ctr', 'aes256-ctr', 'aes128-gcm@openssh.com', 'aes256-gcm@openssh.com', 'aes128-cbc', 'none'] client mac:['umac-64-etm@openssh.com', 'umac-128-etm@openssh.com', 'hmac-sha2-256-etm@openssh.com', 'hmac-sha2-512-etm@openssh.com', 'hmac-sha1-etm@openssh.com', 'umac-64@openssh.com', 'umac-128@openssh.com', 'hmac-sha2-256', 'hmac-sha2-512', 'hmac-sha1'] server mac:['umac-64-etm@openssh.com', 'umac-128-etm@openssh.com', 'hmac-sha2-256-etm@openssh.com', 'hmac-sha2-512-etm@openssh.com', 'hmac-sha1-etm@openssh.com', 'umac-64@openssh.com', 'umac-128@openssh.com', 'hmac-sha2-256', 'hmac-sha2-512', 'hmac-sha1'] client compress:['none'] server compress:['none'] client lang:[''] server lang:[''] kex follows?False
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - Kex agreed: curve25519-sha256@libssh.org
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - HostKey agreed: ssh-ed25519
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - Cipher agreed: aes128-ctr
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - MAC agreed: hmac-sha2-256
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - Compression agreed: none
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - kex engine KexCurve25519 specified hash_algo <built-in function openssl_sha256>
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - Switch to new keys ...
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - Trying SSH key b'xxxxxxxxxxxxxxxxxxxxxx'
[2020/12/30 17:04:19] (DEBUG) paramiko.transport._log():1819 - userauth is OK
[2020/12/30 17:04:19] (INFO) paramiko.transport._log():1819 - Authentication (publickey) failed.

It's weird it says userauth is OK then the very next line is auth failed...

 

[nojohnny101]

Thanks for sharing. Maybe I have posted in the wrong sub-forum? I find it hard to believe that no one else is using the replication engine in TrueNAS.

I still haven't been able to get this to work.

 

[Tabmowtez]

As I am migrating from my old TrueNAS box to my new one I just kicked off the zfs send/receive using mbuffer via the command line. You're right though, I assume this is working for someone I would hope...