Easy Backup Idea Mainly for Nextcloud

[Sethz]

TrueNAS-SCALE-22.02-RC.1-1

Nextcloud 22.2_1.3.7 (app) Or apps in general as using collabora. So, apps in general.

Both have same pools. Simple mirror.

Let me first say, yes, a novice for both TrueNAS and Nextcloud, but 25 years in IT but from a crap Microsoft base.

What I wanted and thought would be easy, would be a master to master Nextcloud, syncing in real time, the full Nextcloud DB, & data. But as we know nothing is ever easy and straight forward.

After lots and lots of reading, this seems to be doable but kind of complex, unless you understand Klingon!

So, questions I want to ask, to see if I can get some detailed answers. Not just what you used or would use but how you did or would do it.

Options. Nextcloud

As I know I would need to also sync or copy the NC DB and not just the data. Or even the whole docker image of NC (or apps) and data?

Now keep in mind the second NC machine would not need to be live but used as a 1: Backup and 2: as a manual change over failover..

I am anal when it comes to backups and fast recovery times.. All my clients have 3 backup config in place. Onsite, and two cloud. One cloud for data and my own cloud backup systems for both data and system image etc.

1: Master - Master but a solution a moron could follow. Would be the best one..

2: Master – Slave. But can the DB of the master be auto backed up in real time on the master and then updated / restored to the slave?? If the master NC went down, HW or a screw up in config etc. Until a master could be re-setup? And then reverse the process back to the new master?

3: A simple replication / resync from the master TN of the full NC to the second TN box and how would, either, be replicated / resynced back to a new master or if need be, restored to the second NC until a master could be re-setup? And then reverse the process back to the new master?

4: Backup options for the NC DB for 2 & 3.

5: Cloud backup of the NC data from the TN box as well. So, I have my 3 backup options and a fast’ish restore back up and running etc.

6: I know a TN config script would be needed to help with that little backup. (Best one would be?)

Once (if ever) I get this working on the LAN the second TN box would be at another location and would need to workout what needs to be changed to then point to it on the WAN.

 

[Patrick M. Hausen]

Why don't you use the mechanisms built into TrueNAS, e.g. ZFS snapshots and replication?

 

[Sethz]

Thanks I really appreciate your reply. I am thinking that maybe the best way as well. But I gather I am trying to workout the correct way to do this.
Does this capture the NC DB as well. Hence questions 3 & 4 etc.

 

[Sethz]

Thanks for your reply. Much appreciated.. Your logic is sound.

OK. All I want to do then to make it easier (for me:))

Just backup NC- (The DB I gather is the main thing or can I replicate the whole Docker NC app?) & Data. Using Replication to another TrueNAS machine.

What I really need is a hand with the steps needed to replicate source and restore steps.
Trust me I have read a fair bit and watched how to use replication etc. But just need to see what the heck I the steps are.

 

[Sethz]

PS: I have setup 3 TrueNAS machines now so I can mess around without screwing the main setup.

 

[truecharts]

We are currently working on a backup quick-start guides for SCALE Apps.

But in short our current advice would be:
- A local replication from `ix-applications` to another dataset (`backup-applications` for example)
- A remote replication (not cloudsync or rsync!) of the local backup (`backup-applications`) to a remote ZFS based backup provider like rsync.net

It's critical to backup both the ZFS dataset layout as well as the actuall files.

 

[Sethz]

That's great and can't come soon enough:) I did what was suggested above and all seemed ok and was about to try and recovery options to another machine, but rebooted first and then got stuck at boot with "A start job is running for import ZFS pool". Worked out it was the single drive I put in to backup to. Disconnect it and the systems boots. Can reconnect the drive after boot but it as was expected not attached to the pool, so still offline etc. So not sure with that, but have seen a few posts on it.

Was wondering if I did the replication task wrong?

 

[truecharts]

That's great and can't come soon enough:) I did what was suggested above and all seemed ok and was about to try and recovery options to another machine, but rebooted first and then got stuck at boot with "A start job is running for import ZFS pool". Worked out it was the single drive I put in to backup to. Disconnect it and the systems boots. Can reconnect the drive after boot but it as was expected not attached to the pool, so still offline etc. So not sure with that, but have seen a few posts on it.

Was wondering if I did the replication task wrong?

We've currently hold up on it, while the above does provide a backup for the actual data, we find the recovery experience too unpolished to finish a guide for it at the moment.

We cannot give you advice on how to troubleshoot boot or replication issues, we limit ourselves to the Apps and things surrounding that :)

 

[Sethz]

Should I be using Recursive or Full Filesystem Replication or neither for A local replication from `ix-applications` to another dataset `backup-applications' etc?

Either way it does not seem to matter as using either, I always get the boot error "A start job is running for import ZFS pool" on the backup pool.

 

[Sethz]

Should I just use TN Core? Would that make this NC backup and restore an easier solution? And just look at TN Scale down the track.

 

[Heracles]

Hi,

Here, I have 3 TrueNAS servers described in my signature. They all replicate one dataset that I called HAStore. That dataset contains the data folder used by Nextcloud, so this part is replicated. Nextcloud itself is running from a Docker host in my ESXi server. The database is also running from that Docker host.

In that one, I also created another container I called Backup Agent. In it, I mounted the Nextcloud's Docker volumes as read-only. I also installed the MySQL client so that I can do an SQLDump. From that container, I tar, zip and encrypt each Docker volume, as well as the SQL dump. I then save these encrypted file to another folder in the HAStore dataset. The end result is that I have all I need to re-create the containers I need to run Nextcloud while at the same time, none of the TrueNAS server can access the content in clear text thanks to Nextcloud's server-side encryption.

For a restore, I create a new Docker host and the volumes. I decrypt the tarballs and restore each volume. I decrypt the SQL dump and restore it, then deploy the new Nextcloud container.

 

[Sethz]

Looks like a plan. Was looking as stated above 2 maybe 3 boxes. 2 onsite 1 offsite + cloud data backup.

(One of those boxes maybe a vm, with just external attach storage)

Your above solution looks like it is the way to go and what I was leading towards..

But it is the detailed process/procedure/steps in general and with the automated DB backup / restore. Let's face it. MS & Windows have made be plain stupid!

I mean I can do everything I want to do, including a CRM (vtiger) on Synology using Drive, Office, Calendar, Chat etc (in fact have and testing it). But lets face it.. I did not want to run it all on a low end DS218j, with only mirror & a crap 512mb RAM. And the cost of Synology hardware is just plain stupid if I wanted what I need and then replicate it x 2.

lol and I like the names of your servers.. I always use Mesopotamian Gods and Egyptian Gods.

 

[Heracles]

Well, let see how you can do by yourself. Of course, do it all with test data only :

1-Deploy 3 TrueNAS servers that follows the guidelines and recommendations
2-Create a dataset in your main one
3-Create a scheduled snapshot tasks of it
4-Create 2 replication tasks to push it to each of the other server

Once that is done, the high availability storage area is done.

5-Deploy a Docker host in a VM
6-In that Docker host, deploy a Maria / MySQL database (be sure that the data are in a Docker volume to survive upgrades)
7-In that Docker host, deploy the official Nextcloud container image. Use the separate volumes to survive updates (apps, themes, config, root, data)
8-Get all of that up and running (Nextcloud working as you need it)

Create the backup agent

9-Deploy a new container based on Ubuntu (or any other distro you are used to)
10-In it, map the apps, themes, config and root volumes from Nextcloud as read-only
11-Map an extra volume from TrueNAS to store the backups
12-create the tarball for each volume (tar -cvf volume1.tar /path/to/volume_1)
13-Dump the SQL database using mysqldump
13-zip each of your tarball (gzip *.tar)
14-Move them to your TrueNAS backup volume

Once that is done, you managed to create one manual backup.

You now need to restore everything...

15-Stop and destroy the database container
16-Create a new database container with a new data volume
17-Unzip the SQLDump and restore it to the new database
18-Create new Nextcloud volumes to restore to
19-Unzip and untar each backup file in the proper volume
20-Create a new Nextcloud container using the restored volume
21-Re-map the data storage from TrueNAS in that new container
22-Start that container and your Nextcloud should be back online with all its settings and data

Once you got that, you successfully create, backed up and restored your Nextcloud.

You will next have to script the backup itself and name the different tarballs differently not to overwrite the previous one when you take your next backup (unless you get that versioning from another serie of snapshots...)

 

[Sethz]

Thanks for that. Great guide & steps. So now whom do I pay to do it for me:) Or re-learning Klingon it is. chelwI'
Seems like I have a lot of reading to do. :)

 

[truecharts]

To be clear @Sethz Your post was about SCALE Apps.
The advice by @Heracles, however, does NOT cover SCALE Apps.
Even more so: The advice is potentially destructive if applied to SCALE Apps(!).

We are currently working on a backup advisory for the SCALE Apps, a draft of which is available by now on our website.
Please be aware: This is a draft, it's not finished and might not work correctly yet.

 

[Sethz]

To be clear @Sethz Your post was about SCALE Apps.
The advice by @Heracles, however, does NOT cover SCALE Apps.
Even more so: The advice is potentially destructive if applied to SCALE Apps(!).

We are currently working on a backup advisory for the SCALE Apps, a draft of which is available by now on our website.
Please be aware: This is a draft, it's not finished and might not work correctly yet.

Cheers and yes I understand @Heracles was talking about Core and not Scale. I checked out the draft. Looks a lot simpler. Might see if I can get that working on the test systems. I will play around with it and see. And yes I know it is all in the early stages. But looks promising.

 

[HITMAN]

Hello,
can you please point me to that link. Couldn't find it.
Thanks!

 

[truecharts]

Our website at https://truecharts.org always contains all our information.
We aim to not give direct-links as directlinks might change in the future.

To be clear: It's a draft, it might not work, not be restorable or even be potentially destructive.
That's what is meant with "draft" and "WIP".

 

[HITMAN]

Thanks, found it. I Was looking at wrong place :)