Howto - Backup from Ubuntu running ZFS to TrueNAS Core ?

[flashdrive]

Hello,

since the Veeam Linux client is not useful (no scheduling, not all of the data is being backed up) I am looking for another solution.

PC with Ubuntu should push all the rpool content to TNCore

I have found this but am not proficient enough to set this up:

Home

ZFS autobackup is used to periodicly backup ZFS filesystems to other locations. Easy to use and very reliable. - psy0rz/zfs_autobackup

github.com

I am struggling with the ssh setup:

***

Note that this is called a "pull" backup: The backup server pulls the backup from the server. This is usually the preferred way.


Its also possible to let a server push its backup to the backup-server. However this has security implications. In that case you would setup the SSH keys the other way around and use the --ssh-target parameter on the server.

***

Multiple ideas:

1) get a better Veeam solution
2) get this up and running https://github.com/psy0rz/zfs_autobackup/wiki
3) another tool available for Ubuntu - I have tried to use all the other "backup" related applications in the ubuntu store like the rsync based - to no avail
4) Asigra and other plugins in TNCore... - no idea
5) edit: https://github.com/jimsalterjrs/sanoid
6) other ideas

 

[flashdrive]

I have tried to set up 4) Asigra

Registration page

What a pain ... yes, it is a commercial product - but compared to the Windows Version of Veeam agent (free edition) this is unbearable complicated.

All I am looking for is the functionality of macOS Time Machine for Ubuntu.

 

[Patrick M. Hausen]

Why don't you look into zettarepl? That's what TrueNAS uses for NAS to NAS or local replication. Shouldn't be difficult to get it running on Ubuntu.

GitHub - truenas/zettarepl: Zettabyte File System (ZFS) Replication Tool written in Python

Zettabyte File System (ZFS) Replication Tool written in Python - truenas/zettarepl

github.com

 

[flashdrive]

@Patrick M. Hausen

Thank you for sharing, I was not aware of this. I will give it a try.

 

[john60]

@Patrick M. Hausen

I'm wondering if zettarepl may be a solution to my problem, but I am a novice about zfs, so would appreciate an expert opinion.

I have an ext4 files system on ubuntu containing more than 80 million files and growing quite rapidly. This growth is legit and the files are
rather small. When I try to rsync (using the truenas GUI) between my ubuntu and truenas, things get locked up then fail. So I split the rsync into 5 chucks, each takes
forever and eventually works.

Note: the changes are minuscule, most of the activity is new data.

If I changed the ubuntu disk to use ZFS, and got zettarepl to work between ubuntu and truenas, would you expect zettarepl to be a lot faster than rsync? I'm guessing yes, because my replication between the truenas and a backup nas is rather fast.

@flashdrive

did you get it to work?

 

[Patrick M. Hausen]

ZFS replication works on the block level, not on the file level. It is potentially orders of magnitude faster than rsync. Whether that potential can be realised depends on the circumstances but gazillions of small files seem like a perfect candidate. Just think how rsync needs to traverse all these directories, then check each file's last modified timestamp all the while comparing things on both ends of a network connection.
ZFS just sends all blocks written since the last snapshot sent and that's it.

 

[BanksiaBoy]

ZFS replication works on the block level, not on the file level. It is potentially orders of magnitude faster than rsync. Whether that potential can be realised depends on the circumstances but gazillions of small files seem like a perfect candidate. Just think how rsync needs to traverse all these directories, then check each file's last modified timestamp all the while comparing things on both ends of a network connection.
ZFS just sends all blocks written since the last snapshot sent and that's it.

Is there a built-in group I can add my user to, so that it has enough permissions to do zfs receive, and create datasets from the cli?

 

[Patrick M. Hausen]

ZFS replication without using Root user

Is it possible to use replication with a standard user? The documentation hints that is possible but I failed to achieve the desired result. Go to PULL and click Account → Users → View Users. Click the Modify User button for the user account you will be using for replication (by default...

www.truenas.com