Hello everyone,
I got some weird behaviours and issues with my replication tasks.
Im running two instances of TrueNAS. First (A) TrueNAS-13.0-U5.3, second (B) TrueNAS-SCALE-22.12.3.3. Im replicating from A to B.
I got two replication tasks running on A, each replicates one of two pools (recursively the complete pool) to B. In each pool there are multiple zvols as disks for VMs.
Replication task 1 is still running without any issues.
Task 2 has multiple issues since 1,5 weeks. All started, when I did a manual rollback from B to A for a single zvol in pool 2. It worked well but afterwards there occured errors. To to the rollback I deactivated the replication task for pool2 for one night and created a dedicated replication task only for one zvol for the VM i wanted to test the rollback for. The next day after the snapshot was replicated I deactivated the task and reactivated the pool2 task again. After that the errors occured.
The task created a new parent dataset "proxmox" in dest B and renamed the old dataset "proxmox" to "proxmoxrecv-4054437-1". Means now I got two datasets with all child-zvols and that requires the double amount of space.
This is issue one.
2nd issue is, that I recieved Truenas alert mails with content "SSD_Pool2/proxmox@auto-2024-03-01_01-00-1H because it was created after the destination snapshot (auto-2024-03-01_00-00) skipping snapshot" and with the hint "Please run `zfs destroy -r HDD_Pool/Backup/SSD_Pool2/proxmox@auto-2024-03-01_00-00` on the target system and
run replication again.." what I did. The same behaviours appeared the next two days and both times I executed the command.
The current behaviour is that the systems still tries to re-replicate the complete pool 2 what takes a huge amount of time (days). He tried this one time and it failed with a similar failure:
"Replication "pool2" failed: Last full ZFS replication failed to transfer all the children of the snapshot SSD_Pool2/proxmox@auto-2024-02-24_00-00. The snapshot HDD_Pool/Backup/SSD_Pool2/proxmox/vm-205-disk-0@auto-2024-02-24_00-00 was not transferred. Please run `zfs destroy -r HDD_Pool/Backup/SSD_Pool2/proxmox@auto-2024-02-24_00-00` on the target system and run replication again.."
Because I tried the similar command three times and it didnt really help I dont want to experiment any more.
Another issue: When I'm on destination B (Truenas Scale) and click under Datasets on any of the two replicated Datasets "proxmox" or "proxmoxrecv-4054437-1" I get one of two messages. Each on both datasets. Sometimes message one, sometimes message two:
For the "old renamed" dataset:
For the "new" dataset:
Regarding the "quotas not found" there is something 'interesting'. In the dataset tree overview it tells "User Quotas: Quotas set for 1 user " and "Group Quotas: Quotas set for 1 group". For both datasets. But when I click on it to see the list, its empty and tells "No User Quotas"/"No Group Quotas".
What is wrong? Does anybody know this behaviour?
Best
I got some weird behaviours and issues with my replication tasks.
Im running two instances of TrueNAS. First (A) TrueNAS-13.0-U5.3, second (B) TrueNAS-SCALE-22.12.3.3. Im replicating from A to B.
I got two replication tasks running on A, each replicates one of two pools (recursively the complete pool) to B. In each pool there are multiple zvols as disks for VMs.
Replication task 1 is still running without any issues.
Task 2 has multiple issues since 1,5 weeks. All started, when I did a manual rollback from B to A for a single zvol in pool 2. It worked well but afterwards there occured errors. To to the rollback I deactivated the replication task for pool2 for one night and created a dedicated replication task only for one zvol for the VM i wanted to test the rollback for. The next day after the snapshot was replicated I deactivated the task and reactivated the pool2 task again. After that the errors occured.
The task created a new parent dataset "proxmox" in dest B and renamed the old dataset "proxmox" to "proxmoxrecv-4054437-1". Means now I got two datasets with all child-zvols and that requires the double amount of space.
This is issue one.
2nd issue is, that I recieved Truenas alert mails with content "SSD_Pool2/proxmox@auto-2024-03-01_01-00-1H because it was created after the destination snapshot (auto-2024-03-01_00-00) skipping snapshot" and with the hint "Please run `zfs destroy -r HDD_Pool/Backup/SSD_Pool2/proxmox@auto-2024-03-01_00-00` on the target system and
run replication again.." what I did. The same behaviours appeared the next two days and both times I executed the command.
The current behaviour is that the systems still tries to re-replicate the complete pool 2 what takes a huge amount of time (days). He tried this one time and it failed with a similar failure:
"Replication "pool2" failed: Last full ZFS replication failed to transfer all the children of the snapshot SSD_Pool2/proxmox@auto-2024-02-24_00-00. The snapshot HDD_Pool/Backup/SSD_Pool2/proxmox/vm-205-disk-0@auto-2024-02-24_00-00 was not transferred. Please run `zfs destroy -r HDD_Pool/Backup/SSD_Pool2/proxmox@auto-2024-02-24_00-00` on the target system and run replication again.."
Because I tried the similar command three times and it didnt really help I dont want to experiment any more.
Another issue: When I'm on destination B (Truenas Scale) and click under Datasets on any of the two replicated Datasets "proxmox" or "proxmoxrecv-4054437-1" I get one of two messages. Each on both datasets. Sometimes message one, sometimes message two:
For the "old renamed" dataset:
For the "new" dataset:
Regarding the "quotas not found" there is something 'interesting'. In the dataset tree overview it tells "User Quotas: Quotas set for 1 user " and "Group Quotas: Quotas set for 1 group". For both datasets. But when I click on it to see the list, its empty and tells "No User Quotas"/"No Group Quotas".
What is wrong? Does anybody know this behaviour?
Best
