Single disk - Copies > 1 but held elsewhere

[Crispin]

Hi folks,

Short question: Can you have copies = 2 but hold the copies on a different vdev?

I have two pools = tank (24TB raw, RAID1-Z) and usbbackup (single 1TB disk)

I have a fireproof safe at home which has a USB pass-through and a 1TB 2.5inch disk inside. I have a daily cron job which rsysncs my important files, around 700GB at the moment, from tank to usbbackup. I have a weekly scrub on the 1TB to check for problems. Thus far it has found none. This has been working well for the past year or two.

If the scrub does start coming back with errors then I'm not sure how I will proceed; perhaps turf the disk (for safety) and get another one. I cannot recover the bad data anyway. perhaps rsync will see the problem and recopy. I don't know.

What I would like to know is if I can have copies=2 (or more) but hold the copies on tank thus saving space on usbbackup.
Pointless you say? No, not really. If scrub returns a read error now I have no chance of recovering it and have to redo everything. If I have copies =2 then it can rebuild it and I'm ok. If my house burns down then the disk is safe and I know the data, as at last scrub, would be fine and recoverable.

It's not bulletproof but it's better than what I have now and that's a good direction to go.

Or, is there a better way to do things?

I could of course get a 2TB disk and be normal but where's the fun in that?

Cheers,
Crispin

 

[Robert Trevellyan]

perhaps rsync will see the problem and recopy

You can tell rsync to do checksums on all files using suitable command-line options, which will be much slower but will recopy damaged files.

is there a better way to do things?

I would vote for reconfiguring your pool with RAIDZ2 instead of RAIDZ1.

What you describe could work, but changing to copies = 2 is only going to affect newly written files, it won't magically duplicate existing files at rest.

EDIT: hold on, you're thinking you can do copies=2 on tank but have one of those copies on usbbackup? No, that's not how it works.

 

[Crispin]

I could do a RAIDZ2 with one disk internal to the NAS and one in the safe. That way post fire / theft I have have the other half of the mirror - that could be an option.

I appreciate that copies=2 is only on new writes (I did RTFM a bit ;) ) but that is easily fixable by recopying the data.

WRT your edit - exactly opposite. I want to hold the copies=2 on tank. If there is corruption on the external disk then it can rebuild easily using the copies on tank. If there is a fire or theft then I have the copy in the safe which is good as per last scrub.

I have always used rsync to keep it up to date and differences (corruption) in the file should be detected by rsync and recopied. I assume then that ZFS would also highlight the fact that there was a problem.

Perhaps I am being overly cautious....

I wonder - I wonder if I can get a USB hub in the safe and two disks... :D

 

[Robert Trevellyan]

I could do a RAIDZ2 with one disk internal to the NAS and one in the safe. That way post fire / theft I have have the other half of the mirror - that could be an option.

Don't even attempt this. Building a vdev out of a mix of SATA and USB drives is a horrible idea, and you won't "have the other half of the mirror" after a fire. That's not how RAIDZ2 works.

You could do copies=2 on tank, but that still relies on tank being available, and a 24TB RAIDZ1 volume is at significant risk of data loss if you have to replace a drive.

differences (corruption) in the file should be detected by rsync and recopied

Not unless you tell it to verify checksums. If you don't, it only checks file modification date and size.

 

[Crispin]

I see the main NAS, with 24TB and tank, as at-risk from fire and theft. I could not care for that. What I'm trying to do is think of a way I can have redundancy and protection on the single 2.5 disk inside the safe. When the main server is stolen / flooded / burnt to a cinder, I want to know that as per last scrub, the data on the 2.5 inch disk is good. I know it's good because it was part of a mirror which was scrubbed and repaired. Now that everything else is in ashes I can still get the disk, remount it and get my files off (it's part of a busted mirror but still there)

I've got rsync checksumming the files so OK there.

 

[Robert Trevellyan]

think of a way I can have redundancy and protection on the single 2.5 disk

You can't have redundancy on a single disk, at least not in the sense of the 'R' in 'RAID'.

I want to know that as per last scrub, the data on the 2.5 inch disk is good

A scrub on a single disk can tell you if the data is good, even though it can't repair errors.

Why not just make a 2nd external backup and put both in the safe? Or hook up a pair of mirrored drives in a drive dock via eSATA and put those in the safe?

 

[Crispin]

Or hook up a pair of mirrored drives in a drive dock via eSATA and put those in the safe?

That's what I'll do but over USB (yes, USB != eSATA)
The reason for the USB is that the safe has a USB pass-through so the disk inside is always active. Nice for realtime backups and don't have to do anything manual like physically move it back into the safe after the backup.

 

[depasseg]

Hi folks,

Short question: Can you have copies = 2 but hold the copies on a different vdev?

No

I have two pools = tank (24TB raw, RAID1-Z) and usbbackup (single 1TB disk)

I have a fireproof safe at home which has a USB pass-through and a 1TB 2.5inch disk inside. I have a daily cron job which rsysncs my important files, around 700GB at the moment, from tank to usbbackup. I have a weekly scrub on the 1TB to check for problems. Thus far it has found none. This has been working well for the past year or two.

If the scrub does start coming back with errors then I'm not sure how I will proceed; perhaps turf the disk (for safety) and get another one. I cannot recover the bad data anyway. perhaps rsync will see the problem and recopy. I don't know.

Why not use ZFS replication instead of rsync to copy from tank to usbbackup?

What I would like to know is if I can have copies=2 (or more) but hold the copies on tank thus saving space on usbbackup.
Pointless you say? No, not really. If scrub returns a read error now I have no chance of recovering it and have to redo everything. If I have copies =2 then it can rebuild it and I'm ok. If my house burns down then the disk is safe and I know the data, as at last scrub, would be fine and recoverable.

Not possible. But I'm confused. You are looking at the case where the backup is corrupt but the source is ok? Wouldn't that be handled by the next replication? Technically, you DO have copies=2 because you have the same data in tank and usbbackup. If you are running ZFS on usbbackup, and are running scrubs on it, then that's the best you can hope for. And there is the possibility of ZFS repairing some corruption, even with only one disk (and without copies=2).

 

[Crispin]

Why not use ZFS replication instead of rsync to copy from tank to usbbackup?

Truth be told - I don't know. I guess I like the manual way of rsync and seeing the files there. Agreed, it's a pointless reason but it makes me all warm and fussy :D I'll look into that option as well.

Not possible. But I'm confused. You are looking at the case where the backup is corrupt but the source is ok? Wouldn't that be handled by the next replication? Technically, you DO have copies=2 because you have the same data in tank and usbbackup. If you are running ZFS on usbbackup, and are running scrubs on it, then that's the best you can hope for.

Yes, it would.
If I did a backup on Sunday and a scrub on Monday and my house burnt down on Friday then I would be at the mercy of the disk between the two events. This would be true of my original question as well so no real point :|

I think I'll buy a 2TB disk and put copies=2 on it :) Let's assume there is a fire - the safe is rated for 2 hours @1850°F and remain below 350°F internally. Ok, that is bloody hot and it's unlikely a house fire will rage at that temp for 2 hours around the safe so it should be ok. However, it might need some help with reading the data.

And there is the possibility of ZFS repairing some corruption, even with only one disk (and without copies=2).

How so?

 

[Robert Trevellyan]

If I did a backup on Sunday and a scrub on Monday and my house burnt down on Friday then I would be at the mercy of the disk between the two events.

I suggest you seriously consider a completely new backup strategy, such as Amazon Glacier or Google Nearline Storage. To my mind, an external USB drive backup doesn't seem like a good match to this level of paranoia. Much better than nothing, but still ...

 

[depasseg]

How so?

I thought I read that somewhere, but perhaps I'm mistaken.

Separate note - 350 degrees seems like a crazy amount of heat for the drive to survive. You really should be looking at some sort of off-site backup.

 

[Robert Trevellyan]

I thought I read that somewhere, but perhaps I'm mistaken.

You might be thinking of the way ZFS stores multiple copies of the metadata.

 

[cyberjock]

Here's how *I* would do this:

Main zpool that stores your data (we'll call it tank).

Second zpool inside your fireproof safe (we'll call it tank2).

Create a dataset on tank (tank/importantstuff) and setup a snapshot and replication schedule to tank2/importantdata. Maybe have it run every 8 hours a day so it's never far behind "real time". Set an expiration date of 30 days (or whatever makes you happy and won't be too big for your USB drive). Ideally I'd do something like 60 days because in the event your house burns down, is involved in a fire, etc it will take you some time to get back to the drive and try to use it. Ideally you don't want all the snapshots to expire before you can use the drive.

Let that server work for you!

I wouldn't use rsync because if you have corruption on the primary zpool, rsync will happily replicate that corruption and you may find you have no good backups at all. Whoops! With the snapshots you can at least rollback the snapshots if something really nasty happens. Since snapshots only take up space equivalent to the blocks that have changed, they possible will take up very little extra space.

HTH.