Better idea for backup than physically moving discs?

[Hypopheralcus]

Hi everyone!

I've been runnig a FreeNAS 9.2.0-Machine for a few weeks now. It is for home use (media server with 3 users and about 10 client computers in a single LAN). - It's a ESX-Machine with XEON E3-1270v3, 32 GiB ECC Memory (16 GiB dedicated for FreeNAS-VM), Supermicro X10SL7F and another IBM ServeRAID M1015 flashed to IT mode. I'm aware of the problems with ESX-VM but I have done what possible to prevent problems (e.g. forwarded both raid controllers directly to the FreeNAS-VM, etc.)

Everything is fine with my zpool (6x3 TB at RaidZ2). But I'm still missing a proper backup solution (e.g. if anything really goes bad with the VM or with the disks).

As it's a home appliance I don't really want to build a second machine, so i thought of the following:

I have quite a few empty bays and also a bunch of old (but still working) 2 TB-Drives. now i thought of just plugging them into the bays and building a new backup-zpool (maybe just with Z1? or do i need Z2 for backup as well?). once or twice a month (which is ok for my usage) I'd put them into the server and rsync the working pool to the backup. Then I pull them out again and store them off-site until the next backup.

Of course, that is some work to do but I'm not sure how to build a different solution with limited budget and possibilities?

I could of course also use an old computer of mine (no serverboard, no ECC, etc.) and build another FreeNAS there but I'd either have to put the machine into the same house (bad for backups) or into my office with a VPN-bridge but only at an upload rate of 10 Mbit/sec.

So is there any FreeNAS Guru who could just look over my plan and tell me if that would be a good way to do or if there is something easier?

Thanks a lot and kind regards!

Hypo

 

[no_connection]

I plan to use USB dock with bare external drives and simple repairable/recoverable file system.
Then checksum the data on NAS before transfer, keeping the checksum on both NAS and backup disks. Then recheck backup disk once in a while for defects.
ExactFile is fairly practical for generating checksums providing you run Windows. http://www.exactfile.com/

Knowing the integrity of the backup is fairly important, and checking it should be done a few times a year. Normal HDDs can be demagnetized in weak sports with time. Reading the data fixes that (or informs you of loss) which you get for free with an integrity check.

No ECC or Z1 might be fine for doing backup to, problem is it might bite you back once you want to restore that precious data from a real disaster.
With a simple backup you can get most data back, and even know the status of it with checksums. With inadequate protection you might get none back if your unlucky.
Doing both isn't bad ether.

A lot om my data is movies and music, not a worlds loss but still. Not really needed to keep running backups of *dead* data. A small or simple NAS for *live* data that is automatic would be smart. Or cloud depending on size.

 

[joeschmuck]

You could do what you have suggested and a RAIDZ1 is fine. You are on a budget so anything is better than nothing. What I don't like is you adding and removing a pool all the time and if you are not careful you could destroy all your data. It is a risk you need to evaluate for yourself. If I had enough drives to build another FreeNAS machine, I would do so. Your computer doesn't need to be state of the art or have tons of RAM, and you could even run 8.04 if you really wanted to, or even Windows XP, just so long as you could backup all the data is the key.

One other thing you should be aware of is how much data will have changed between backups as this will help with how long it will take to rsync things.

 

[Hypopheralcus]

Hi! Thanks for the inputs!

@no_connection: I'm aware of the data integrity problems (which is a main reason for running FreeNAS for home use on server-grade hardware, as I've already lost whole disks before...). I'd run regular scrubs after a complete rsync before storing the disks off-site again. So this wouldn't be a problem. Also demagnetization shouldn't be too bad when i do backups once or twice a month i guess? I'm running a linux-windows mixed network. There are 2 Windows PCs, 2 Linux PCs and a bunch of Raspberry PIs with Raspbian as well as some Linux VMs (Debian and Mint). I', more and more changing to linux now but I still have the 2 "Main PCs" at Win 7.

@joeschmuck (Are you aware, that "Schmuck" means jewelry in german?)
OK so thanks at first for your explanations. So I understand that a second backup machine would be the preferred way. (Maybe I could even afford a HP Microserver for that). But still I'd have to physically move the machine from office to home and back - at least when there are big data changes? On the other hand, of course I could do the backup physically in the same LAN once and do the incremental (and also then regular, maybe even daily) rsyncs via the VPN-bridge to my office?

Just to make sure I understand your point - I always thought one of the (many) good things in FreeNAS is that you can happily pull out and put back the drives without having to care about sockets and specific orders contrary to standard RAIDs where you could ruin it by just swapping 2 drives in the same pool. So why or how could i ruin my pool by physically moving the disks in and out?

And at last - why would I prefer 8.04 over 9.2? Would it need less ressources? (BTW, I gonna skip Windows XP *g* I'm on my way of leaving Windows as much as I can).

So thanks again and kind regards

Hypo

 

[cyberjock]

If you were going to do backups with rsync, but the destination is ZFS you'd be better off with ZFS replication.

I don't like the idea of swapping disks. That's just asking for trouble.

The only ways I see to do backups are to build a second FreeNAS box or go to some kind of cloud service. If you broke up your "important" data from "unimportant" data with datasets you can choose to backup just the important data and avoid dealing with the entire pool.

Once you get to a sufficiently large quantity of data many options become unfeasible. Rsync has performance limitations once you get to large quantities of data since it's single threaded. Usually I think about 5TB as about the most you can expect to rsync.