FreeNAS for safe archiving

[alphacat]

Hello!

I am planning to build a NAS which I would use as an archive for my works (hundreds of thousands of photos and raw video footage etc.) and rsync backup. FreeNAS seems to be great for this - but I am a little confused on which settings I should use for my needs. I wanted to ask for your opinion.

Guidelines:
- I will use one volume to archive my works (I will put files manually through a network share). In my case this means every few days I will drop few files in the archive.
- If possible I will use one volume as a destination for rsync client on my windows machine to back up all my wip files. It would be nice to have snapshots here, but it's not crucial.
- The safety of the archived data is most important.
- The speed of the system is not the most important factor as I don't update the archive very often.

Specs for the machine:
E35M1-I, 8 GB of RAM, 6 HDDs (2TB each), FreeNAS on USB and Lian Li Q08 case.

Things I would like to ask about:

1. Taking the guidelines - should I go with raidz or mirror? Which one would be safer?
1.1 If RaidZ - which would be safer - 2x3drive or 1x6drive? (in first case one for archive, one for rsync, second case - everything on same volume)

2. If I use one of the volumess as a destination folder for rsync incremental backup of one of the folders on my windows machine, can I store freenas snapshots on the volume? Is this a good idea?

3. Any better ideas how to set up freenas to match my guidelines?

Thanks a lot in advance.
Peter

 

[peterh]

A few comments

There is no inherent difference in safety for mirror or raidz, in both cases you can
stand one disk failure ( but not two ) . In both cases you are dependent of
monitoring ( so you can replace a failed drive). The mail reports in freenas would do.

If using raidz i would suggest a third config : raidz with 2x3drives + another 2x3drives
( that will consitute a 2 vdev zfs pool.) Then create 2 filesystems on that pool,
one for archive and one for rsync. This way you share the same pool and may use
more of one without hitting any limits : e.i. 1 TB rsync and 5TB archive. The size
of each filesystem ( as shown by clients ) is easily changed by "zfs set quota= "
( or manipulated in the freenas GUI)

An alternative is to use all 6 drives in a raidz2 pool ( where you can loose
2 drives without loosing your data, this might be handy if you have limited
ability to superwise you freenas.

And don't forget, real safety uses off-site backups ( as simple as naing a copy
on an usb drive and store it at your parents or simular)

 

[StephenFry]

If RaidZ - which would be safer - 2x3drive or 1x6drive?

If you are serious about safekeeping, raidz is OUT. Really, it's a recipe for losing your data.

2x3 drives mirrored or 1x6 drives in raidz2 are the only options, and as you can see from my config, I prefer the raidz2 option.

Also use a UPS, otherwise if the system happens to lose power during the wrong kind of write operation, your data will be at risk.

 

[alphacat]

Thank You so much for all your answers. It all helps but also makes me confused ;) My knowledge of this topic is rather modest.

The point I am confused are:

peterh:
There is no inherent difference in safety for mirror or raidz, in both cases you can stand one disk failure ( but not two ).
1. I assume You were writing about mirroring in case we use only two HDD's. But is it possible to use freenas to mirror a HDD not to one, but to two another HDD's? Or maybe do gradual mirroring like Archive HDD + Mirror, Mirror + Mirror's Mirror?

2. In my case (safety over speed over available disk space) what will be the benefit of raidz2 over mirroring? Is it more available storage space? Are there any safety benefits to this?

I also have a question about the usb key used for booting. Regarding safety of the whole nas system usb key seems like a bottleneck.

3. What happens when the USB which I use to boot freenas is damaged / missing?
3.1 Would having a cloned image of the usb contents change anything? Can freenas replication tasks be done on the usb contents it boots from?

Peter

 

[peterh]

alphacat,
as per iten (1) :
yes you could have mirrors of 3 drives, in that case you may loose 2 drives. The backside is that you will have the capacity of only 1 drive at the cost of 3.

And don't forget that no raid system will replace an offsite backup.

Given that a functional offsite backup is present you may concentrate on building
a cost-effective and usable system that has the capacity to grow in speed and
capacity. The best option is to have two identical systems, this could enable
the backup to be used as primary storage in the even of total loss of primary storage.
( either by using the datalink between mail-site and offline site, or by moving the backup
to the main site in the event of disaster)



2/ raid2 will give 2 things : lower speed and the ability to copy with two drives failing
before data loss. But both raidz and raidz2 will improve storage economy over mirrors.

3/ the USB is a single-point of failure yes. But it easily replicated
( dd if=/dev/<running USB> of=/dev/<new usb> bs=64k ) so you may have
a working one stored at your device. Or you may even download fom freenas.org
and create a fresh install ( of the same version) . IN both cases you need to
keep backups of the freenas config and restore this backup when replacing usb key.

Now i guess that the freenas config is seldom changed so the need is not that acute,
and all zfs related configs are located on the zfs pool itself.

(3.1 ) is what i describes above.

 

[StephenFry]

alphacat, the configuring of FreeNAS is relatively so simple (once you've done it once or twice) that you needn't worry about losing the (USB) drive with the OS on it.

The replication peterh suggests is valid, but if you search the forum a bit, you'll find people have encountered problems with that technique. Unless you have loads of users and passwords and tons of shares to configure, then, yeah, it would make sense. Otherwise, just do a regular "Save config" and load it in a fresh install.

Also, test test test test! Whatever setup you do, you seem to want to purchase about six drives. Once you've got those in the bag, monkey around with them, create different configurations, and most importantly, run some disaster scenarios. You will feel a lot better about your FreeNAS if you are secure in your knowledge of how to replace a drive!