Replication tasks fails due to "zfs not found" error

[cat-man-du]

Hello,

So I've searched around quite a bit and haven't seen anyone else with this particular problem, so I'm posting here to see if anyone has some insight.

I have two machines, one TrueNAS Mini XL running TrueNAS 12.0-U5 and another running Proxmox 7 (Linux hypervisor). I'm trying to set up the TrueNAS machine to pull-replicate snapshots as a dedicated (non-root) user from the Proxmox machine on a schedule, but whenever I try to set up the replication task via the TrueNAS Web UI, I'm presented with an error that the zfs binary cannot be found. I've tried adding entries for /sbin:/usr/sbin:/usr/local/sbin into the user's .bashrc and .profile scripts, but to seemingly no avail.

The interesting thing is that the TrueNAS UI seems to work fine if I change the SSH connection to use the root user on the Proxmox machine, but for security reasons I would much rather use a dedicated replication user. So, seeing that root appears to work, I edited the /etc/profile to give the replication user the same $PATH variable as root. Still no dice.

What's frustrating is that, when I login to the Proxmox host as the replication user, the $PATH is correct and I can use zfs commands without any issue. Is anyone aware of something that might be causing such odd behavior?

Below is the exact error I get from the TrueNAS UI:

Code:

Error: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 138, in call_method
    result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self,
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1205, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 973, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/replication.py", line 642, in list_datasets
    return await self.middleware.call("zettarepl.list_datasets", transport, ssh_credentials)
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1248, in call
    return await self._call(
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1205, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zettarepl.py", line 409, in list_datasets
    datasets = await self.middleware.run_in_thread(list_datasets, shell)
  File "/usr/local/lib/python3.9/site-packages/middlewared/utils/run_in_thread.py", line 10, in run_in_thread
    return await self.loop.run_in_executor(self.run_in_thread_executor, functools.partial(method, *args, **kwargs))
  File "/usr/local/lib/python3.9/site-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.9/site-packages/zettarepl/dataset/list.py", line 13, in list_datasets
    return [dataset["name"] for dataset in list_datasets_with_properties(shell, dataset, recursive)]
  File "/usr/local/lib/python3.9/site-packages/zettarepl/dataset/list.py", line 30, in list_datasets_with_properties
    output = shell.exec(args)
  File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/interface.py", line 88, in exec
    return self.exec_async(args, encoding, stdout).wait(timeout)
  File "/usr/local/lib/python3.9/site-packages/zettarepl/transport/base_ssh.py", line 54, in wait
    raise ExecException(exitcode, stdout)
zettarepl.transport.interface.ExecException: sh: 1: zfs: not found

I'd appreciate any further troubleshooting ideas as well.
Cheers!

 

[cat-man-du]

Oh, I was able to solve this, so I'll update for anyone that comes across this in the future:

TrueNAS logins were spawning non-interactive shells, which I was unaware don't source /etc/profile. I hard-coded the necessary paths into /etc/environment (on the Proxmox host), and the replication job was able to use the zfs command as expected.

 

[errorencounted]

Oh, I was able to solve this, so I'll update for anyone that comes across this in the future:

TrueNAS logins were spawning non-interactive shells, which I was unaware don't source /etc/profile. I hard-coded the necessary paths into /etc/environment (on the Proxmox host), and the replication job was able to use the zfs command as expected.

I'm getting a similar error...would you explain a bit more as to what I need to do?

I have a synology NAS that I'd like to use as a secondary backup. I've created and tested the ssh connection but when trying to set up a replication task, I'm getting "zfs: command not found" error.

Thanks!

 

[anodos]

Synology most likely does not support ZFS, ergo replication is not possible.

 

[cat-man-du]

I'm getting a similar error...would you explain a bit more as to what I need to do?

I have a synology NAS that I'd like to use as a secondary backup. I've created and tested the ssh connection but when trying to set up a replication task, I'm getting "zfs: command not found" error.

Thanks!

As anodos mentioned, Synology does not natively support ZFS as far as I know. You could double-check by trying which zfs on the command line of the NAS. If it does not return the path of a ZFS binary, then you almost certainly don't have any requisite ZFS programs installed.

I use a Synology which I inherited from someone else as an offsite backup, but I use a combination of rsync (for non-sensitive data) and restic (for more private data). It's not nearly as convenient or full-featured, but as an offsite or hot spare, it is better than nothing.