SOLVED After upgrade to TrueNAS Scale (from TrueNas core) unprivileged users can't receive zfs streams

[Matveev Andrey]

On TrueNAS Core we created different local users with SSH access by keys. We assigned each user to corresponding pool
user1 --> pool1
user2 --> pool2
etc

TrueNAS Core was used as backup server which received ZFS streams from remotes servers.

After upgrade from TrueNAS Core to TrueNAS Scale (upgrading was completed successfully) remote servers are unable to send ZFS streams

When manually tries to SSH to TrueNAS Scale - connection succeed but no zfs command available for users:

root@pve-storage-03345:~# ssh id254@storageB
Last login: Sat Dec 17 19:01:42 2022 from 172.16.253.201

TrueNAS (c) 2009-2022, iXsystems, Inc.
All rights reserved.
TrueNAS code is released under the modified BSD license with some
files copyrighted by (c) iXsystems, Inc.

For more information, documentation, help or support, go here:
http://truenas.com

Welcome to FreeNAS
id254@storageB:~$ zfs list
-bash: zfs: command not found
id254@storageB:~$

StorageB - TrueNAS Scale server

And ZnapZend on sending server complaints similarly

Dec 17 19:00:01 pve-storage-03345 znapzend[88898]: starting work on backupSet tank254/datastore/slave
Dec 17 19:00:01 pve-storage-03345 znapzend[88898]: sending snapshots from tank254/datastore/slave to id254@storageB:tank254/datastore/slave
Dec 17 19:00:01 pve-storage-03345 znapzend[3138602]: snapshot worker for tank254/datastore/master done (87943)
Dec 17 19:00:01 pve-storage-03345 znapzend[3138602]: send/receive worker for tank254/datastore/master spawned (88909)
Dec 17 19:00:01 pve-storage-03345 znapzend[88909]: starting work on backupSet tank254/datastore/master
Dec 17 19:00:01 pve-storage-03345 znapzend[88909]: sending snapshots from tank254/datastore/master to id254@storageB:tank254/datastore/master
Dec 17 19:00:01 pve-storage-03345 znapzend[88912]: bash: line 1: zfs: command not found
Dec 17 19:00:01 pve-storage-03345 znapzend[88915]: bash: line 1: zfs: command not found
Dec 17 19:00:01 pve-storage-03345 znapzend[88925]: bash: line 1: zfs: command not found
Dec 17 19:00:01 pve-storage-03345 znapzend[88898]: ERROR: cannot send snapshots to tank254/datastore/slave on id254@storageB
Dec 17 19:00:01 pve-storage-03345 znapzend[3138602]: send/receive for tank254/datastore/slave failed: Can't call method "message" without a package or object reference at /usr/lib/x86_64-linux-gnu/ZnapZend.pm line 369.
Dec 17 19:00:01 pve-storage-03345 znapzend[3138602]: send/receive worker for tank254/datastore/slave done (88898)
Dec 17 19:00:01 pve-storage-03345 znapzend[88933]: bash: line 1: zfs: command not found
Dec 17 19:00:01 pve-storage-03345 znapzend[88909]: ERROR: cannot send snapshots to tank254/datastore/master on id254@storageB
Dec 17 19:00:01 pve-storage-03345 znapzend[3138602]: send/receive for tank254/datastore/master failed: Can't call method "message" without a package or object reference at /usr/lib/x86_64-linux-gnu/ZnapZend.pm line 369.

Any ideas and help would be very appreciated. Thanks

 

[Matveev Andrey]

One more thing to be specifaied.
With TureNAS Core for all users (except root) user shell was defined as nologin. With TrueNAS Core we had to change it to bash/sh/zsh to be able to SSH from remote servers

 

[Matveev Andrey]

My solution for using ZnapZend for backup external servers to TureNAS scale (SSH + ZFS send/recv)

In user home folder:

mkdir ./bin && cd ./bin
for CMD in mbuffer zfs ; do ln -frs "`which "$CMD"`" ./ ; done

 

[faisalm]

Hello, how did you install mbuffer on TrueNAS Scale?

 

[Matveev Andrey]

Hello, how did you install mbuffer on TrueNAS Scale?

via apt install ...
( chmod +x /usr/bin/apt* )

 

[faisalm]

Ah, thank you. I've been hesitant to enable apt on the system but it could be worth it for a couple small tools like this. Thanks!