SOLVED Replication Failing - Dataset Permission Messed

[Soloam]

Hello, I have 2 Datasets:

- homes
- main
- main/dataset1
- main/dataset2
- main/dataset3

I had set a 2 recursive snapshots, one for "homes" and one for "main". Freenas creates one for home, and one for main's dataset. So far so good!

I'm replicating the content to a local HDD (SATA), so I set 2 replications, one for homes and one for main (recursive, to replicate do the 3 datasets). In the end I get a error generating main, and recive a email:

Hello,
The replication failed for the local ZFS vol0/main/mymedia while attempting to
send snapshot auto-20160607.2020-1h to localhost

If I go to CLI and to /mnt/backups/main/.zfs/snapshot I can see the main snapshots
If I go to CLI and to /mnt/backups/main/dataset1 the folder is empty and no .zfs folder.

Another thing in the GUI all the datasets permission are messed up! dataset 1 has the permissions of dataset3 and dataset2 is with root permission and wheal (it was different) and dataset3 has no user, no group and no permissions.

Can any one help me solve this. Can it be a bug in recursive replication?

Thank you

 

[Soloam]

Ok, this is driving me crazy! I tried to destroy all the existing snapshots and replications tasks and created one for each other (no replications) and the problem persists! I'm not able to replicate my dataset!

I created a new replication to my dataset and when in the gui I see my replicated dataset, the permission window is blank! On the CLI when I go to my dataset the folder is empty and .zfs folder does not exist!

I'm on FreeNAS-9.10-STABLE-201605240427.

Can any one help me? I'm one step away of dropping replication all together, but I really like the file history ability.

Thank you

 

[Soloam]

I tried to clone the snapshot and apparently it worked!

Strange! When I navigate the folder structure I can go as far as the parent dataset! My dataset path is /mnt/vol2/main/mymedia, this is the replication on PULL. I can browse until /mnt/vol2/main/ and when I make a ls the folder is empty! Is this normal?

 

[Soloam]

Ok, I tried to detach the volume and reimporte it, and I can't it says that it can't mount /mnt/vol2/main/mymedia!

Can't any one help me? What am I doing wrong?

 

[depasseg]

You are jumping all over the place, it's really hard to follow what the error is. I'm assuming this is a test environment, and that you have the ability to delete and destroy, right? Which version of freeNAS are you running?

0. Can you provide us with the output of zfs list (in code tags please).
1. Can you start by deleting all snapshot jobs, replication jobs and all existing snapshots?
2. Reboot and verify everything is still removed.
3. Detach and destroy the backup pool.
4. Verify permissions on your datasets are correct, if not, correct them.
5. Create your backup pool.
6. Create your snapshot tasks (one for each main dataset with the recursive option enabled).
7. Wait for the snapshot schedule to kick in and verify snapshots exist in GUI.
8. Verify permissions are still good.
9. Create replication task, but before you enable it, please post a screenshot of the job. We want to make sure that the destination dataset is correct.

 

[Soloam]

Hello depasseg, I'll try to do that! This is not a test environment, it's my actual pool, but the HDD of the backups is new and I can create and destroy! I'll follow all your steps and post back the results.

 

[depasseg]

Sounds good. What version of FreeNAS are you using? I've seen a couple snapshot/replication issues recently and am wondering if there is an underlying issue.

 

[Soloam]

I was able to replicate the exact same problem in a VBox test machine! This way I can test without risking my main pool.

Also I opened a bug report on https://bugs.freenas.org/issues/15837

 

[Soloam]

Ok, I'm using FreeNAS-9.10-STABLE-201606072003, now on a VM, but the same problem occurs on production system!

Code:

NAME                                                    USED  AVAIL  REFER  MOUNTPOINT
freenas-boot                                            604M  30.2G    31K  none
freenas-boot/ROOT                                       597M  30.2G    25K  none
freenas-boot/ROOT/Initial-Install                         1K  30.2G   594M  legacy
freenas-boot/ROOT/default                               597M  30.2G   595M  legacy
freenas-boot/grub                                      6.33M  30.2G  6.33M  legacy
vol0                                                   1.00M  1.91G  25.4K  /mnt/vol0
vol0/.system                                            319K  1.91G  28.4K  legacy
vol0/.system/configs-5ece5c906a8f4df886779fae5cade8a5  25.4K  1.91G  25.4K  legacy
vol0/.system/cores                                     25.4K  1.91G  25.4K  legacy
vol0/.system/rrd-5ece5c906a8f4df886779fae5cade8a5      25.4K  1.91G  25.4K  legacy
vol0/.system/samba4                                     105K  1.91G   105K  legacy
vol0/.system/syslog-5ece5c906a8f4df886779fae5cade8a5    109K  1.91G   109K  legacy
vol0/main                                               102K  1.91G  25.4K  /mnt/vol0/main
vol0/main/mymedia                                      50.8K  1.91G  25.4K  /mnt/vol0/main/mymedia
vol0/main/mymedia/lightroom                            25.4K  1.91G  25.4K  /mnt/vol0/main/mymedia/lightroom
vol0/main/syncthing                                    25.4K  1.91G  25.4K  /mnt/vol0/main/syncthing
vol2                                                    156K   976M    19K  /mnt/vol2
vol2/backups                                             57K   976M    19K  /mnt/vol2/backups
vol2/backups/main                                        38K   976M    19K  /mnt/vol2/backups/main
vol2/backups/main/mymedia                                19K   976M    19K  /mnt/vol2/backups/main/mymedia

Images:

The resume of my system

The snapshot config

The Replication Config

Media Original Permissions

Backup result Permission

 

[Soloam]

Ok, problem solved, according to Wojciech the behavior is expected: https://bugs.freenas.org/issues/15837

Tkank you for your help