Desaster recovery test - can't find suitable solution

[DD4711]

Hello,

I'm currently trying to figure out how to implement disaster recovery.

My needs: access backup data independent from FreeNAS. Scenario: My main FreeNAS is stolen.

What I did until now:
- main storage named "storage" is encrypted on local drives
- create a second encrypted storage named "backup"
- set snapshots on "storage"
- set replication task to copy all snapshots to "backup"

This works good. When I want to make a backup, I attach the USB drive, activate the replication task. Then immediately data is replicated. When finished task is disabled and pool is detached.

Data on this backup disk should be readable in worst case scenario from some other computer (for example Linux Mint / Ubuntu). This is where I failed with all my ideas.

Now I tried the following:
- boot FreeNAS but didn't unlock "storage"
- trying to add my "backup" storage: this does not work, FreeNAS can not find the backup disk in drop down menu while adding process. Not until I unlock "Storage", then I can also add existing "backup" pool. Strange.

-> Why can't I add my "backup" storage without adding before my "storage"? I want to test wheather I can access backup data without "storage". This would be a similar scenario when my FreeNAS is stolen and I need to create a new one.

-> Is there any way to meet my demands? Currently I am out of ideas and need some hints.

Thanks a lot for your help.

 

[AndroGen]

Do not know how to resolve your specific issue, but reading documentation I bumped on very strong suggestion do not use USB drives in FreeNAS.

 

[DD4711]

Where do you find such suggestions? Can't find any.

 

[AndroGen]

you might want to go trough some nice posts from @joeschmuck, there are couple of nice recommendations.
When it comes to the USB, remembering, I has seen multiple once, e.g. this one:
https://forums.freenas.org/index.php?threads/i-am-noob.45825/#post-311251

 

[Heracles]

Hi DD4711,

Snapshots and ZFS replication are pure ZFS features. As such, you will need FreeNAS to read it and there is no way out of this. For your data to be readable without FreeNAS, you will need to copy at file level, with a tool like RSYNC.

Also, be careful with encryption. Most of the time, people end up locked out of their own data more than preventing anyone from reading their content.

Here, my DR is based on a second and offsite FreeNAS and I do ZFS replication. The pool itself is not encrypted but the Nextcloud software I use for my private Cloud does encrypt with the server-side encryption module. The recovery keys are in the database and I fully documented the recovery process.

To avoid the recovery keys being in clear text in the DR NAS, the backup for the database is encrypted with a passphrase. So should I need to restore, I need that passphrase to open and restore the database. Once the database is restored, I can restore Nextcloud and then Nextcloud will fetch and decrypt my data from FreeNAS.

Good luck with your own design,

 

[DD4711]

Hello @ all,

thanks for your support and suggestions.

I think I have now a good solution:

On a notebook I installed FreeNAS as Recovery System. There I attached and unlocked successfully my "backup" USB drive with encrypted pool. I can access my backup data as usual through FreeNAS. It works very good and in case of disaster recovery I can access all my backup data especially all my snapshots.

It was my main attention to check if I can read my backup data on another system -> test passed :)

As I think this is a good method for recovery using USB drives and offsite backup. Easy going!

 

[garm]

@Heracles why on earth do you say ZFS is FreeNAS only? You can import a ZFS pool on most major operating systems, Windows being a notable exception

 

[DD4711]

@Heracles why on earth do you say ZFS is FreeNAS only? You can import a ZFS pool on most major operating systems, Windows being a notable exception

I can't believe, too. But I didn't manage it to get working on Linux Mint.

 

[garm]

Geli encrypted pools are different, you need to mount the encryption first and then import the pool that is being written on the geli devices. Unless you have a really good reason for it, don’t encrypt your data if long retention is the goal. If you must encrypt data, do so on a clear text pool, unless you have a really good reason to encrypt the pool.

You also must ensure the host system uses the same feature flags as FreeNAS.

 

[DD4711]

Geli encrypted pools are different, you need to mount the encryption first and then import the pool that is being written on the geli devices. Unless you have a really good reason for it, don’t encrypt your data if long retention is the goal. If you must encrypt data, do so on a clear text pool, unless you have a really good reason to encrypt the pool.

You also must ensure the host system uses the same feature flags as FreeNAS.

Hello @garm
Thanks for your post. Will need to read about feature flags, that was until now not considered.

Reason for encryption is: if NAS / HDDs are stolen I have no doubt about it. Will think about to store them not encrypted for long retention.

 

[garm]

Well if you have your financial data (or “other” sensitive data) in clear text on your NAS you might consider something like GPG to encrypt individual files or archives. If you have full disk encryption anyway on your laptop the workflow to store them safely on the NAS isn’t that complex.

 

[joeschmuck]

My needs: access backup data independent from FreeNAS. Scenario: My main FreeNAS is stolen.

What I did until now:
- main storage named "storage" is encrypted on local drives
- create a second encrypted storage named "backup"
- set snapshots on "storage"
- set replication task to copy all snapshots to "backup"

Well your needs are not terrible at all, many people want this and nothing is just simple.

There are a lot of threads on our forums about encryption and backups, a search could be overwhelming but it's part of the work you have to do to make a good dependable system/design.

Questions I'd ask you are:
1) Do you need full disk encryption on FreeNAS? You could encrypt using something like TrueCrypt or others out there, this would make it a file level encryption and simplify your FreeNAS encryption headache. But some folks are fine encrypting the entire VDEV.

2) If someone steals your FreeNAS system, where are your backups going to be stored?

3) If you use a file level encryption then you could use a Cloud service to RSync your files and then easily access them from any computer.

4) How much data are you looking to encrypt? I would only encrypt data that needs to be encrypted, video files and music files are not that type of course. DVD/BluRay copies of that data are better in my opinion, but an external USB hard drive could be connected to a Windows/Mac computer and then an RSync could be done for that data as well via your network. It may not be as fast as a direct connection however easy of use should be a key factor in your design as well.

5) If I were a business then I will likely want a fully encrypted FreeNAS VDEV and fully automated off site backup. That is easy to do as well but you must have the offsite FreeNAS machine too. So that is two machines that you must manage.

There are a lot of options but only you can say how you want to do this because everyone has an opinion. And sorry that I'm not really providing you much help, I wish I have the magic answer, heck I wish I had the winning lottery numbers too.

Good Luck!

 

[l@e]

apart of what joeschmuck mentioned you need to define a backup policy, this is where it starts.

so maybe not all of what is stored in primary FN is data created in house or single copy from business operations, i'm sure that all of us put in the box whatever we download from internet to have a fast access when we need often. so for materials like those you dont have to worry even to encrypt.
so my suggestion is:
1- start with data classification
2-based on classifications, create backup strategy for each class and set parameters like retention (importance over the time pasing) , how many copies (how important data is), on-site/offsite copy (if some data is critical for operation a second should be ready for access at shortest time possible), change frequency of data (how much data are you willing to loose from backup to next cycle), type of data you store (documents change frequently during normal operations, databases instead are more dynamic, and archives more stable over time),
3- after the backup policies of above you define the ways accomplish all of them, and this does not mean to have only 1 method for all kind of data (ie: i would suggest 1st production FN has everything and all people work on that, than a offsite box with slower drives gets all of data in less frequent way (1-2 times a day if it is offsite internet can cost), and keep a smaller box for critical part only to replicate every hour or 15 mins (depending on point 2 results), and also for unique data you might want to have that cold backup over USB drive, blueray or even a could service (encrypted or not) .
4- set monitoring processes and logs and don't forget to read them to check if everything is going as supposed.
5- on not busy days make a try and mount last backups to check if the data is there or not, (simulate a failure: what if my primary is down)

note - if you keep data that you dont want to go public, encryption on drives is not guaranteeing safekeeping, so encrypt only when necesary adn what necessary.