Suggestions for Backup software

[HarambeLives]

Anyone have any suggestions for how to achieve this? I need software to finish the last part of my backup plans

* Main NAS has Snapshots lasting 12 months
* Snapshots related to second NAS which satisfies main local backup requirements
* Using Arq to backup to secondary NAS for long term backup (10 year retention)
* Using Arq to backup to the cloud for off-site backup (5 Day retention)

The last one is

* Offsite and Offline backup to a 4TB SATA Disk which lives in my desk drawer at work. Must have dedupe and versioning

I can't use Arq as I want to differ the software to prevent an issue where the software screws up

It seems like TrueNAS does not handle a USB removable backup well

Ideally the software could run in Windows, as my desktop where the SATA dock is, runs Windows. I have a VM that runs Veeam and Arq that is Windows Server 2016.

It could run in TrueNAS, but I want it to be easily exportable

BlobBackup would work, but there is no way for it to run as a service, and its very new
borgbackup would work, but it does not run in Windows. Better to run on the TrueNAS box and just point it at the drive from Windows Share?

Thoughts? Right now I'm thinking borg is my only real bet.

 

[winnielinnie]

Any reason you can't use Rsync? It's part of TrueNAS's GUI under Tasks > Rsync Tasks

You can have it "push" from your TrueNAS server to your Windows box (under the presumption your USB drive is plugged in and accessible.) The issue I see is that TrueNAS does not allow you to disable the Rsync schedule if you plan to only run it manually.

Another option would be to setup cwrsync (prebundled cygwin/rsync/ssh) on your Windows box and make an executable script that you can run on demand, make a shortcut for it, etc. (This is how I do it.) You just need to create an SSH keypair with the user that is going to authenticate to the TrueNAS server, and add the executables to your Windows "PATH".

 

[HarambeLives]

Does rsync offer version control with dedupe? I want to be able to keep every version, and be able to go back in time month by month

 

[winnielinnie]

Does rsync offer version control with dedupe? I want to be able to keep every version, and be able to go back in time month by month

I've never used such a feature of rsync, but it does support version control with ---backup-dir=foldername (or --backup) combined with something like
--suffix="_"$(date +"%Y-%-m-%d-%H-%M")

There's a lot of options that can be passed. The ones I use are,

• -a (for archive mode)
• -v (for verbose)
• --progress (to display the progress)
• -h (for human-readable sizes)
• -H (to preserve hardlinks)
• -xx (to stay within the same filesystem)
• --delete-delay (to delete files that no longer exist on the source)
• --ignore-errors (seems to be needed for particular files that always error in a user's PROFILE folder)
• --inplace (only if sending to ZFS, to take advantage of copy-on-write and snapshots)

rsync(1) - Linux man page

Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync ...

linux.die.net

 

[winnielinnie]

I want to be able to keep every version, and be able to go back in time month by month

It doesn't have this layer of sophistication. All previous versions of files will be appended with the suffix you define, but they will be moved to live under the same folder defined in --backup-dir

So for instance under the backup folder you might see,

Code:

"Document of Stuff.docx_2021-05-06-16-35"
"Document of Stuff.docx_2021-07-21-18-15"
"Document of Stuff.docx_2021-08-01-19-30"

On second thought, I could be wrong. I'd have to try it out personally on a test folder / dataset.

 

[HarambeLives]

Thank, you are right. I just tried it out

I'm not sure I want versions to be handled like that, I really like the idea of an entire version per run, and not just per file. I'm going to give it a try and see how it goes

 

[winnielinnie]

Just played around with it for a bit and tried out a neat trick that I didn't expect to work, but it actually works!

rsync -avxxhH --progress --delete-delay --backup --backup-dir=.versions/`date +%Y-%m-%d-%H-%M` --exclude="/.versions" username@server.ip:/mnt/poolname/path/to/directory/ /media/path/to/usb/folder/

What this does comes very close to what you described earlier. Notice I'm not using the "suffix" option?

The only issue is I did this test on my Linux computer. Not sure how to translate the "date/time" using Windows and/or cygwin. (Anyone know how to properly use a date/time variable in Windows and/or cygwin?)

--backup, --backup-dir=, and --exclude= must be specified

What ends up happening is if a file is modified in the source, rsync will create a backup folder named .versions/<YYYY-MM-DD-HH-MM>/ and move the previous version in there.

You can change the options and timestamp to your liking.

In my "destination" (my USB drive), I have a folder named .versions, and within this folder are separate timestamped-folders that contain the original versions of modified / previously deleted files with their original names.

Not as cool as ZFS snapshots, but still handy!

 

[Arwen]

@HarambeLives - I use a locally attached hot-swapped disk on my NAS for backups. Technically it's 3 disks, 750GB disk for all except media, and an 8TB & 12TB for all, rotating between them.

Here is where I documented what I do;

How to: Backup to local disks

One neat thing I implemented, were ZFS snapshots. So after each backup, I make a ZFS snapshot on the backup disk. As the disk gets fullish, I delete the oldest. This allows me to have a history and in someways, multiple backups on the same disk.

My scheme uses RSync from the NAS to the locally attached ZFS single pool disk. I guess I could have used ZFS send & receive, but whence I got the method working, I have little incentive to change.

Because I use RSync, the ZFS snapshots on the NAS are ignored. But, that is not a problem for me. In someways it's a benefit.

Last, using ZFS on my backup disks allows me to perform a ZFS scrub integrity test. If I remember correctly, I have found some bit rot, but easy to fix.

 

[kiriak]

maybe a foolish question (I'm a noob)
but why not doing snapshot replication to the USB disk?

 

[winnielinnie]

maybe a foolish question (I'm a noob)
but why not doing snapshot replication to the USB disk?

It doesn't work as expected if your target is a removeable (i.e, not "always present") destination.

The Replication Tasks are designed to be used on a schedule, and preferably tied to a Periodic Snapshot Task, and preferably the destination is readily available without manual intervention.

If he wishes to use it as an "on demand" backup, it will not transfer over the intermediary snapshots when he runs his manual incremental replications. If he specifies which snapshot tasks to tie into and/or specifies the naming schema of which snapshots to replicate, he can bump into an issue if he's not running his manual backups frequently enough. (Because the older expired snapshots are destroyed on the source, and thus there is no "common base" between the source and destination with which to do an incremental replication.) I was bit by this issue myself, and so I reluctantly went back to using a custom script to replicate to a USB drive.

 

[HarambeLives]

I ended up just making a VM and using Borg. I'm now a convert! Borg is AWESOME.