Add directory to periodic snapshot/replication

[NasKar]

I have periodic snapshot for my pool v1 that is replicated to another freenas box. I've made some changes to nextcloud storage using @danb35 nextcloud script. My current backup has the db and files directory but not the config and themes directory.
1) If my snapshot is for my entire pool v1 recursively, why did the config and theme directory not get added automatically to the replication?
2) How do I add the config and theme directory to the periodic snapshot/replication without breaking the current replication and having to start from scratch?

 

[sretalla]

my snapshot is for my entire pool v1 recursively

How are you assessing that the directory is not in the snapshot?

 

[NasKar]

How are you assessing that the directory is not in the snapshot?

I browse the backup server to look for the files.

 

[sretalla]

OK, so which dataset is the parent of that directory?

 

[NasKar]

nextcloud is the parent.
Setup is pool v1
so source is /mnt/v1/nextcloud with subdirectories of config, db, files, themes. Nextcloud is the dataset as well as the db and files directory. config and themes are regular directories not datasets. The periodic snapshot is v1 recursive with exceptions for the iocage directory. So I would assume if a new directory is created under /mnt/v1/nextcloud it would get replicated to the backup.

 

[sretalla]

So I would assume if a new directory is created under /mnt/v1/nextcloud it would get replicated to the backup.

I agree, that should be happening.

Are you sure you're mounting the filesystem and not an older snapshot on the backup server?

 

[NasKar]

Are you sure you're mounting the filesystem and not an older snapshot on the backup server?

Not sure how to check that.
In the past I had a problem with directories not showing up until I rebooted the backup server. That resolved after I started from scratch and redid the periodic snapshot and replication. BTW I just reboot the backup server and the files are still missing.

 

[sretalla]

zfs list will show you what's mounted where...

 

[NasKar]

zfs list shows
/tank/super/nextcloud
/tank/super/nextcloud/files
/tank/super/nextcloud/db
Those are all datasets so I presume they would show up. Regular directories wouldn't show up with zfs list

 

[sretalla]

Regular directories wouldn't show up with zfs list

Correct.

so now the question is going to be does your replication work...

 

[NasKar]

all the directories/datasets that were present at the time the snapshot and replication were created are working.

 

[sretalla]

So it worked at least once... are you sure it still works?

 

[NasKar]

Yes it is still replicating files/directories that are being add except in the /mnt/v1/nextcloud directory? Do you think it has anything to do with the fact that the nextcloud/db and nextcloud/files dir are datasets but the nextcloud/config and nextcloud/themes are not? ie. some of the child directories to nextcloud are datasets and some are not.

Code:

ls -l /mnt/v1/nextcloud/
total 36
drwxr-xr-x   2 www  www   5 Jul  4 16:51 config
drwxr-xr-x   6 88   88   14 May  3 11:11 db
drwxrwx---  17 www  www  23 May  2 17:44 files
drwxr-xr-x   3 www  www   4 Jul  4 16:50 themes

 

[sretalla]

Those directories are in a dataset... if that dataset is replicated, then the directories and files within it are too.

That's what's very strange here. Can you try creating another directory at the same level and just put a small file in it and see if that turns up?

 

[NasKar]

Code:

drwxr-xr-x   2 www   www     5 Jul  4 16:51 config
drwxr-xr-x   6 88    88     14 May  3 11:11 db
drwxrwx---  17 www   www    23 May  2 17:44 files
drwxr-xr-x   2 root  wheel   3 Jul 14 08:17 test
drwxr-xr-x   3 www   www     4 Jul  4 16:50 themes

After running the replication task the new dir does not show up on the backup server and a reboot didn't help either

 

[sretalla]

But you're saying that if you put a file in the config directory/dataset, it arrives on the other side?

 

[NasKar]

The config dir is not a dataset just a regular directory with a few files in it. It doesn't replicate to the backup and the new test dir with a test file in it doesn't either. Datasets are the Parent nextcloud dir and the 2 child dir (files and db). Directories in bold are datasets.
--nextcloud
----files
----db
----config
----themes
----test

But you're saying that if you put a file in the config directory/dataset, it arrives on the other side?

nothing in the config dir shows up on the other side.

I also renamed themes to theme, created a themes dataset and copied the files from them theme to themes. Then I ran the replication task and the themes dir and files didn't show up.

 

[sretalla]

And you're not seeing any errors showing up in the FreeNAS GUI relating to failed replication?

I think your replication is broken.

 

[NasKar]

If I delete the replication task and create a new one it will refuse to work as there is already data in the target. I would prefer not to wipe out the target and start the replication as it takes 3 days. Would the best option be to create a new periodic snapshot for the config and theme dir/replication task?

 

[sretalla]

Replication is at the dataset level, so you would have to select the parent dataset (seems it's the one not replicating now anyway).

I'm not sure I would recommend to have replication jobs handling child datasets and parents differently, but you can always try to make it work.