What file system should I use on my backup?

[RyanJennings]

Hey guys. I have looked at a lot of posts, but don't see much talk of specifics on backups. Any recommendations on what file system to run on my offsite backup? EXT4, NTFS??

After all my reading I see that I should get one more drive. I had planned on 3x4TB in raidz1, but I see that raidz2 is the way to go. Now I am wondering what about read errors from my backup? Is there a file system that doesn't run the risk of loosing the whole drive on a read error?

The files on that setup will be a mix of some that I need to have (pictures, movies), and others I could live without, but would be nice to keep (movies, music).

So for backup I had planned on running rsync to offsite on something less elaborate than freenas. I don't have enough hardware to create another freenas and do replication. The offsite will have 2x1.5TB and 1x2TB. I will have to pick only the good stuff to backup. I am thinking I should avoid JBOD so one failure won't loose the whole works.

Thanks for the help on a slightly off-topic question.

 

[leenux_tux]

Ryan,

I have multiple backups of my data on different systems. I know this probably looks a bit paranoid, however, having read some recent posts (absolute horror stories) where folks have not made backups, incorrectly assuming that their data is perfectly safe because they are using ZFS/RAID and lost absolutely everything; makes me think perhaps I am not too paranoid after all.

I don't do offsite backups (this is my home/home office setup), however, the same principles can still be applied.

• Backups instance 01
  • I have two pools in the same server, one is called TANK, the other called TANKBACKUP. Both are RAIDZ1. I manually do RSYNC's from TANK to TANKBACKUP. TANK uses 4X2TB drives, TANKBACKUP uses 4X1TB. I have the highest compression set for TANKBACKUP, plus I have dedup applied against folders where I know there will be massive duplication of files. I should really look at using ZFS SNAPSHOTS for managing this, however, RSYNC just "works".
• Backups instance 02
  • All I do is plug a USB 3 hard drive into my laptop (Windows based) and run something called "robocopy". Think "rsync" for Windows (but not as many features). Why USB3 ? It's much faster then USB2, makes a huge difference. I only backup specific folders/files.
• Backups instance 03
  • I have multiple other Linux based systems in the house, used as media clients mainly, however, in one I have added a 1TB laptop SATA drive and I run RSYNC to this drive from the FREENAS system for specific folders/files.

You can use Windows or BSD (FreeNAS) or Linux for your backup target, however, the most important thing is to get it done as soon as possible. It's not a nice feeling when you realize your data is toast.

 

[RyanJennings]

Leenux,

Thanks for the reply. You bring up some good points I hadn't thought about. Heavy compression could help me in some cases. Some of my content is probably not very compressible, but some could be compressed. The other point I get is having a manual backup step (as a 3rd or 4th copy) isn't all bad to avoid overwriting your backup with bad data.

My original question still stands though. With all the talk of raidz1 not being good enough anymore because of the likelihood of a read error, should I also be concerned about a read error on my backup? Is there a particular file system that handles that type of read error better than others? I really wouldn't mind if I lost a file here and there. As long as I didn't loose a whole drive.

Thanks again,
Ryan

 

[ser_rhaegar]

You need to move to RaidZ2, RaidZ3 or a RAID6 controller if you want better chances of surviving UREs. RaidZ2 gives 2 drives of parity, RaidZ3 gives 3. Or you could buy expensive enterprise drives with a URE of 1E15 or 1E16 which would work in RaidZ1. However the cost bump of enterprise drives would make it cheaper to just buy more consumer drives and go with RaidZ2/Z3.

 

[RyanJennings]

Ser,

Thanks for the reply. I plan to do RaidZ2 on my file server. My question was what is my best option for a file system on my backups. I don't have the hardware or drives to do RaidZ2 on my backups.

Ryan

 

[leenux_tux]

This all depends on what hardware you have available. Can you post some details ?

 

[RyanJennings]

This all depends on what hardware you have available. Can you post some details ?

The offsite will have 2x1.5TB and 1x2TB. AMD E350 8gb ram. On board sata/nic. Board has one pci x16 slot I could add a nic or sata card, but not real worried about speed on the backup machine so will probably stick with the on board.

Thanks,
Ryan

 

[leenux_tux]

I'm confused as to why you think you don't have enough hardware to run FreeNAS effectivley ? Looking at the information you have provided all you need is a thumbdrive (providing your motherboard allows booting from a thumbdrive of course) and your good-to-go ? Or am I missing something ?

Also, depending on the motherboard you are using (which you haven't specified) you might be able to run 16GB or RAM instead.

From what I am seeing the bottleneck is going to be the network between the two sites, the specification of your remote machine is not going to make any huge difference. The trick with offsite backups is to do the backup locally before you move the system to the remote location, this saves you having a huge data set to move over the WAN in the first instance, plus you state your only interested in storing music and movies. Unless your adding exact copies of DVD's (5GB+) to your master then you won't be having huge amounts of data being transferred.

 

[RyanJennings]

Thanks for sticking with me on this.

I guess I had looked past freenas because I didn't think I would end up with a raid type setup with just the 3 drives and them not all being the same size. I also got the impression that zfs probably isn't the best for single drives. I could have that wrong though. I don't know much about ufs. Those are the two choices with Freenas right? I am not opposed to setting it up if it makes sense.

Motherboard is this http://www.newegg.com/Product/Product.aspx?Item=N82E16813138365
Only has 2 ram slots, I have 2x4GB installed in, one pcie x16, 3x sata ports. With the current price of ram I am not really interested in buying more of that for the backup side since performance isn't a high priority. It can boot from usb drive.

I have 5Mbps between sites if that changes anything. I had planned on running the first sync locally so we are on the same page there. My data is kind of in two groups. I have pictures and home movies (700GB) that for sure need to stick around. Then I have some music and hollywood movies (2TB) that I can live without. I don't do exact copies of movies.

let me know if I have left anything else out,

Ryan

 

[leenux_tux]

Even though you have 2 drives at 1.5TB and another drive at 2TB you can still create a RAIDZ1 out of these (in effect 3x1.5TB). I know there are a number of discussions around the fact that RAID1 has some issues, however, at the moment you have nothing at all. If you can't shell out any more cash for more hardware, whats better ? No backups or a backup? I have used RAIDZ1 for two years now without issues.

As far as I understand you can't create the pool via the GUI, however, you can from the command line.

The other really cool thing about ZFS (one of many) is that, lets say 12 months down the line you get hold of two 2TB drives, you can replace the two 1.5TB drives, one at a time, resilvering/rebuilding the array. FreeNAS does the rebuild automatically. There are a few things you need to do in the GUI to remove and replace drives, however, I know this works as I moved my pool from 4x1TB drives to 4x2TB drives using this exact method. Takes some time obviously because of the number of rebuilds you have to do but it works.

 

[Joshua Parker Ruehlig]

my live data is all on ZFS. my offsite backup (server at relatives house) is on ZFS. I suggest you so something similar.