SOLVED "[ENOENT] Path {Path} not found" error after replication

ryun

Dabbler
Joined
Mar 17, 2023
Messages
14
I'm having an issue with local replication where the first replication of a dataset goes without a hitch. But the second replicated dataset throws "[ENOENT] Path {Path} not found" errors and "[EFAULT] Failed retreiving USER quotas for {Path}" in the Datasets UI tab.

Things I have tried that has resulted in the same error:
  • One replication task for both datasets together.
  • Two replication tasks, one for each dataset.
  • Rebooting the server
  • Running the replication task again
  • Wiping the pool and trying again (with and without encryption at the pool level)
  • Replicating source root dataset to target root dataset
  • Replicating source child dataset to target child dataset
Here's the output of zfs list on the shell (excluding boot pool):
Code:
NAME                                                         USED  AVAIL     REFER  MOUNTPOINT
BackupTank                                                  1.46T   302G       96K  /mnt/BackupTank
BackupTank/Backups                                          1.46T   302G     1.45T  /mnt/BackupTank/Backups
FamilyDataTank                                              2.42T  1.99T       96K  /mnt/FamilyDataTank
FamilyDataTank/FamilyData                                   2.42T  1.99T     2.42T  /mnt/FamilyDataTank/FamilyData
ReplicaTank                                                 3.88T  3.26T      232K  /mnt/ReplicaTank
ReplicaTank/Replica                                         3.88T  3.26T       96K  /mnt/ReplicaTank/Replica
ReplicaTank/Replica/Backups                                 1.46T  3.26T     1.45T  /mnt/ReplicaTank/Replica/Backups
ReplicaTank/Replica/FamilyData                              2.42T  3.26T     2.42T  /mnt/ReplicaTank/Replica/FamilyData


When I try zfs mount -a in shell I get the following output: cannot mount '/mnt/ReplicaTank/Replica/Backups': failed to create mountpoint: Read-only file system

I tried removing the readonly flag using sudo zfs set readonly=off ReplicaTank/Replica/Backups with no success after a subsequent zfs mount -a

It also doesn't explain why my other replicated dataset does not exhibit the issue as it has the readonly flag set to "on" as well. Here's the output from sudo zfs get readonly in case it is helpful:

Code:
NAME                                                        PROPERTY  VALUE   SOURCE
BackupTank                                                  readonly  off     default
BackupTank@manual-2023-03-12_19-53                          readonly  -       -
BackupTank@daily-2023-03-13_00-00                           readonly  -       -
BackupTank@daily-2023-03-14_00-00                           readonly  -       -
BackupTank@auto-2023-03-15_15-28                            readonly  -       -
BackupTank@auto-2023-03-17_00-00                            readonly  -       -
BackupTank/Backups                                          readonly  off     default
BackupTank/Backups@manual-2023-03-12_19-53                  readonly  -       -
BackupTank/Backups@daily-2023-03-13_00-00                   readonly  -       -
BackupTank/Backups@daily-2023-03-14_00-00                   readonly  -       -
BackupTank/Backups@auto-2023-03-14_14-55                    readonly  -       -
BackupTank/Backups@auto-2023-03-15_00-00                    readonly  -       -
BackupTank/Backups@auto-2023-03-17_00-00                    readonly  -       -
FamilyDataTank                                              readonly  off     default
FamilyDataTank@manual-2023-03-12_19-54                      readonly  -       -
FamilyDataTank@daily-2023-03-13_00-00                       readonly  -       -
FamilyDataTank@daily-2023-03-14_00-00                       readonly  -       -
FamilyDataTank@auto-2023-03-15_15-28                        readonly  -       -
FamilyDataTank@auto-2023-03-16_00-00                        readonly  -       -
FamilyDataTank@auto-2023-03-17_00-00                        readonly  -       -
FamilyDataTank/FamilyData                                   readonly  off     default
FamilyDataTank/FamilyData@manual-2023-03-12_19-54           readonly  -       -
FamilyDataTank/FamilyData@daily-2023-03-13_00-00            readonly  -       -
FamilyDataTank/FamilyData@daily-2023-03-14_00-00            readonly  -       -
FamilyDataTank/FamilyData@auto-2023-03-14_14-55             readonly  -       -
FamilyDataTank/FamilyData@auto-2023-03-14_15-00             readonly  -       -
FamilyDataTank/FamilyData@auto-2023-03-15_00-00             readonly  -       -
FamilyDataTank/FamilyData@auto-2023-03-15_14-15             readonly  -       -
FamilyDataTank/FamilyData@auto-2023-03-16_00-00             readonly  -       -
FamilyDataTank/FamilyData@auto-2023-03-17_00-00             readonly  -       -
ReplicaTank                                                 readonly  off     default
ReplicaTank/Replica                                         readonly  on      local
ReplicaTank/Replica@auto-2023-03-15_15-28                   readonly  -       -
ReplicaTank/Replica@auto-2023-03-17_00-00                   readonly  -       -
ReplicaTank/Replica/Backups                                 readonly  on      inherited from ReplicaTank/Replica
ReplicaTank/Replica/Backups@auto-2023-03-14_14-55           readonly  -       -
ReplicaTank/Replica/Backups@auto-2023-03-15_00-00           readonly  -       -
ReplicaTank/Replica/Backups@auto-2023-03-17_00-00           readonly  -       -
ReplicaTank/Replica/FamilyData                              readonly  on      inherited from ReplicaTank/Replica
ReplicaTank/Replica/FamilyData@auto-2023-03-17_00-00        readonly  -       -

I have a fresh install of TrueNAS-SCALE-22.12.1 on commodity hardware (system specs in signature) and a bare-bones configuration of TrueNAS Scale. No VMs, just 4 pools (including the boot pool). All of the datasets were created as SMB compatible except for the boot pool. I used the basic wizard to make the replication tasks with the recursive option checked of which I have attached a screenshot. I'm also a novice to TrueNAS and Linux in general.
 

Attachments

  • Screenshot 2023-03-17 081005.png
    Screenshot 2023-03-17 081005.png
    49.4 KB · Views: 131
Last edited by a moderator:

ryun

Dabbler
Joined
Mar 17, 2023
Messages
14
Also, I'm not sure if it is related but on the initial install I chose the admin account instead of root as per the recommendation in the installation wizard.
 

ryun

Dabbler
Joined
Mar 17, 2023
Messages
14
Got things working even after reboot. Turns out I needed to run:
  1. sudo zfs set readonly=off ReplicaTank/Replica
  2. sudo zfs mount -a
  3. sudo zfs set readonly=on ReplicaTank/Replica
This is because, as far as I can figure, my parent dataset was read-only and my child dataset that was having issues (Backups) was inheriting that.

Is this a bug or am I doing something ill-advised by trying replicate two datasets to the same target?
 
Top