Rsynch/Replication options

[Phase]

Hi everyone,

I need to create a copy of a few datasets in another FreeNAS server and keep them synched... if they synch overnight that is sufficient.

The objective is (1) protection from accidental deletion (not using snapshots, see below), (2) protection against loss of pool, (3) doing all of that while doing the synch only on metadata when possible — i.e. when moving a 10 TB directory tree from /foo/mine to /bar/mine I would want only the inode (the zfs equivalent) to be updated as opposed to triggering a 10 TB copy. Same when renaming bob.tar to bobby.tar.

On a side note, I have been afraid to use snapshots because of item (3) above. Not sure what the impact would be on the disk usage. And it seems that replication is only available on snapshots.

All data changes (moves, renames) is done from windows machines.

Thoughts?

Thanks!

 

[sretalla]

I have been afraid to use snapshots because of item (3) above.

Your fears are unfounded. Snapshots are at block level and a file rename only changes the metadata blocks, not the data blocks. (unless /foo and /bar are different datasets, but then it's not a rename, the data will actually move and be replicated... I think with Rsync too).

Not sure what the impact would be on the disk usage. And it seems that replication is only available on snapshots.

Remembering that snapshots are just the pointers to the data blocks at a point in time which protect the listed blocks from deletion or re-use while the snapshot exists, no additional data is used...

If you had files that are being added and/or renamed only, you could take dozens of snapshots daily and consume only kilobytes more space.

Snapshots consume additional/free space when deletion occurs, since they are protecting the blocks on disk for those deleted files being referred to from being re-used.

If you expire your snapshots in a reasonable amount of time, this will tend not to be an issue.

I recommend using replication and snapshots.

If you want to do rsync, make sure you have a look for using metadata only for L2ARC:

L2ARC impact on rsync performance for largely dormant data

I've been doing some more testing and the evidence suggests that using a L2ARC with a metadata only flag has a potential substantial positive impact on rsync performance over AFP. For largely dormant data, the combination of L2ARC dedicated to metadata seems to benefit rsync performance a lot...

www.ixsystems.com

 

[Phase]

Awesome. Thanks!!!