Rsync Creating two backup copies of the same share

[AVSION]

Hi,

I finally put together a backup freenas server using Dell PE T410 with 6 HDD mix brands and size (4 x 1TB + 2 x 2TB) in one pool. i set the pool as mirror as i don't have enough 2TB drive so i can get one bigger pool 3.6TB as my share file is 2.9TB. i know its not the right way, but it will be Turning on /backing up once a week. (is there a better way to do it?)

The problem i have is when the backup file is been copied to the backup server 192.168.1.11 i can see on the backup share folder the are two copies of the same folders and not all the folders/files are been copied, see eg below:

Backup
Tank>Folder1
>Folder2
>SMB>Folder1
>Folder2

where the original share looks like this

Storage>Folder1
>Folder2
>Folder3
>Folder4
looking via the CLI does not show a separate or 2 copies of the same SMB file


[root@freenas ~]# ls /mnt/Tank/SMB/
.DS_Store .TemporaryItems/ Folder1/ Folder2/
.ssh/ Folder3/ Folder4/


For testing I set up(via the legacy GUI) the Rsync on both freenas servers (11.2 BETA3) with the following configurations:

Primary Server 192.168.1.10

Dataset= /mnt/Tank/SMB
Permissions=unix
user=james
group=wheel

Rsync modual
Modual name = sync
path= /mnt/Tank/SMB
Access mode=rw
user=root
group=wheel

Rsync Task
path= /mnt/Tank/SMB
Remote host = 192.168.1.11
Remote modual name = sync
Direction=push
Time / Date= 0, 30, every hour/day/month/day of the week
user=root

services = on

sharing - windows SMB
Path= /mnt/Tank/SMB
name=Storage
Export Read Only=false
Browsable to Network Clients=true
Allow Guest Access=false

Backup Server 192.168.1.11

Dataset = /mnt/backup
Permissions=unix
user=root
group=wheel

Rsync modual
Modual name = sync
path= /mnt/backup
Access mode=rw
user=root
group=wheel

Rsync Task
path=/mnt/backup
Remote host = 192.168.1.10
Remote modual name = sync
Direction=pull
Time / Date= 0, 30, every hour/day/month/dayof the week
user=root

services = on

sharing - windows SMB
Path= /mnt/backup
name=Tank
Export Read Only=false
Browsable to Network Clients=true
Allow Guest Access=false

Any idea why it create two copies of the same share and not coping all the files?

The only thing i can think(but not sure) is the following:
1. Dataset permission - one user is set to james the other is to root but both in the same wheel group
2. users acc not setup properly
3. primary server pool not been upgraded yet, was holding with that.

Please let me know if you need more informarion
Thank you

 

[melloa]

Any idea why it create two copies of the same share and not coping all the files?

Probably your target is wrong.

If you want to rsync (192.168.1.1) /mnt/tank/cifs to (192.168.1.2) /mnt/tank/cifs, you have to omit "cifs" from the target, I.e.:

Assuming you are on 192.168.1.1 server ...

rsync -r --progress /mnt/tank/cifs root@192.168.1.2:/mnt/tank

 

[AVSION]

Thank @melloa for your reply,

i have done all the setting from the legacy GUI, following your instructions do i keep the settings (show in OP) as they are on the GUI and run this command or remove all the settings and run this command? i.e rsync -r --progress /mnt/Tank/SMB root@192.168.1.11:/mnt/backup

when setting the target pool i didnt create cifs dataset, i just used the backup dataset created by the pool as the target and chose backup as the target path in the rsync task and services i.e /mnt/backup. as well when creating the cifs share i chose the same backup dataset. do i need to create a dedicated/separate cifs dataset in the target? i.e /mnt/backup/smb

thank you

Thank you

 

[melloa]

i have done all the setting from the legacy GUI, following your instructions do i keep the settings (show in OP) as they are on the GUI

I couldn't tell you. I stopped using the rsync from freenas log time ago as I don't keep my back-up server on and didn't want the messages on my main server console saying that the rsync had a problem or something like that.

I run my back-ups manually using a script that copies all that is needed.

I'll need to defer your question on how to configure the server via GUI to the other folks here.

 

[Chris Moore]

Here is a good illustrated guide: https://thesolving.com/storage/how-to-sync-two-freenas-storage-using-rsync/

That is the guide I used when I set it up and it worked for me. I just did one way from the primary to the backup.

 

[AVSION]

Thank you @Chris Moore,

This is exactly how i set it up via the GUI but still getting duplicates and missing files, BTW i was using this link as a guide https://www.youtube.com/watch?v=_Bu7BsJHXUQ
i think it might be a bug in beta3, let me know what you think.

I have detached the backup pool, stoped the rsync services on both servers , then created a new backup pool and run the command. @melloa suggested on tmux session

Code:

rsync -r --progress /mnt/Tank/SMB root@192.168.1.11:/mnt/backup

it works and showing one folder ATM, but it is very slow 29MBps compare to the speed i got via the GUI, not sure why, so i'm waiting to see when it get to folder 3. i can monitor the progress via tmux and also created a cifs share for backup dataset and can actually see the all the folders and files. happy to do it via the cli but prefer the backup speed of the GUI.

Will update the progress soon,