Redundancy vs backups, and should I even look at FreeNAS?

[neopolitan6]

The question I'm going to lead up to: Is FreeNAS a good option for our use case? How should I be thinking about redundancy vs backups?

I'm responsible for a unique blend of storage requirements.
1. small business file server (moderate storage, currently 3-5Gb)
2. family media storage (1-2 TB)
3. backup target for business and family use

Currently, I have a Synology 2 bay NAS. It has no redundancy. I'm confused about how to structure storage and backup, and concerned that I don't have proper backups in place. (Regarding critical business data, it is copied daily to multiple devices, and a cloud service, so I feel better about that.)

The most confusing part is trying to comprehend ZFS. I read that RAIDZ1 is insufficient, because a second failure is likely during a rebuild process. But then if using RAIDZ2, wouldn't a 3rd failure be a significant statistical possibility? Where does one stop? I'm starting to think that redundancy in machines is better than redundancy in drives.

So would it not be better to setup a vdev with 3 or 4 x 4TB in RAIDZ1, for 8 or 12TB storage, and use the Synology with a single drive or 2 in a simple, no redundancy setup, as a backup target? The backups would probably not be of computer backups, but rather data from the file server and media storage.

This way a failure of a drive in the FreeNAS, and a subsequent 2nd failure would only result in loss of data not backed up to the Synology. Which, if I setup the rsync backup, should only be computer backups. A failure in the Synology would not result in any data loss, assuming it was corrected, and a fresh backup of the FreeNAS is done.

Am I on the right or wrong track?
Why would someone spend money for drives to use RAIDZ2, rather than a separate device?

 

[garm]

Hi and welcome to the forum

Let’s start with the basics, redundancy isn’t backup. You cannot choose one over the other and thus the question is moot. They simply serve two very different functions.

Backup is essential, especially if you run a business. Losing family photos can be painful; loosing business data can jeopardize propert, freedom and life..

There are three key aspects of backups; making them, storing them and restoring them. ZFS ensures that making a backup is easy by the snapshot feature. Storing data on ZFS ensures that the data isn’t corrupted over time thanks to the block lever checksums. When restoring backups we are sure that the data we read back is the same as we once saved. ZFS snapshots can be stored on other mediums, but I’m hardcore ZFS fanboy so I don’t use a middleman. Rsync to a different filesystem breaks the integrity chain.

Follow the 1-2-3 rule to have a successful backup strategy with as low risk of data loss as possible.

Redundancy is what makes data integrity possible. You need some kind of redundancy for ZFS to be able to heal corrupted data (all hardware will corrupt data, only some more than others). Redundancy also gives high availability, meaning that we can replace a failed/failing drive without taking the server out of production. Having to restore from backup every time a drive fails is going to get annoying and expensive.

RAIDZ is good for high drive capacity utilization, while mirrors give you higher drive performance. The exact pool layout for a given use case needs closer investigation, but as a general premise “drives are dirt cheap, lost time or data ain’t”.

 

[Chris Moore]

How should I be thinking about redundancy vs backups?

Redundancy exists for the purpose of preventing corruption of the data (at least in ZFS) and to reduce the possibility of down time. I was using an online calculator yesterday (for fun) to figure out what the potential for data loss was with my array configuration and I should loose data about once ever 70 billion hours, if the calculator is to be believed and as long as I maintain the system.
Redundancy is not a backup, never was.
A backup is about being able to restore your data if a catastrophic event happens, like a flood that puts your server under a foot of water. The backup strategy that many people subscribe to is having, the original (1) and a local backup (2) and an offsite backup (3), copies of the data as a minimum.
FreeNAS and ZFS can play a part in safeguarding your data. Many people feel it is the best option for protecting your data from being corrupted on disk due to bit rot.

I'm responsible for a unique blend of storage requirements.

It isn't really that unique.

Currently, I have a Synology 2 bay NAS. It has no redundancy. I'm confused about how to structure storage and backup, and concerned that I don't have proper backups in place. (Regarding critical business data, it is copied daily to multiple devices, and a cloud service, so I feel better about that.)

I would say that a good introduction might be these resources:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

The most confusing part is trying to comprehend ZFS. I read that RAIDZ1 is insufficient, because a second failure is likely during a rebuild process.

That is a theory based on the idea that the failures will come when the drives are old and the stress of the rebuild process might be the cause of a secondary failure. It is based on the statistical rate of failure and statistics are only a tool to try and predict the potential outcome. The recommendation against RAIDz1 is about playing it safe.

But then if using RAIDZ2, wouldn't a 3rd failure be a significant statistical possibility?

No, because it is already unlikely, but possible, to have two failures at once. It is even less likely that there will be 3 failures at the same time. I manage a lot of storage and I have seen two drives fail at the same time, and even seen a situation where three drives were degraded before the first drive was replaced. Things can happen. We plan for the things that are likely to happen, when it comes to redundancy, because we don't want to be silly about it. I am not walking around in a padded suit because I might trip and fall.

I'm starting to think that redundancy in machines is better than redundancy in drives.

That isn't so silly. At home, I run a pair of servers where the second server does nothing but mirror the data on the first server. It is for my convenience because if there is a reason for the first server to be down, I don't want to be stressed (at home) about needing to get it back in operation immediately. It is also a backup, but it is primarily for my happiness and that of my wife and children that watch movies and videos from the server.

So would it not be better to setup a vdev with 3 or 4 x 4TB in RAIDZ1

No, because we don't use RAIDz1 for primary storage with drives larger than 1 TB. If you wanted to have 6 drives in RAIDz2 in primary storage and have five drives in RAIDz1 for backup storage, that would be perfectly fine. The purpose of the redundancy in the primary storage is to reduce the likelihood of down time.

Am I on the right or wrong track?

Yes, because

a failure of a drive in the FreeNAS, and a subsequent 2nd failure would only result in loss of data not backed up

The goal is to prevent ANY loss of data and we want to have a backup for disaster recovery, but we never want to need that backup.

Why would someone spend money for drives to use RAIDZ2, rather than a separate device?

It is down to two things. How valuable is the data? If it were lost, how much would that impact you or the organization? Then, how valuable is your time to recover to a working state. If you are down for a day, two days. How much would that matter? You have to decide.
I run a server at work that takes 12 to 14 days to backup. It has 290 TB of data in the storage pool. If I had to restore from a backup, we would be down for 14 days at the minimum. You bet we have a whole backup server with a live copy of the data so that the primary server going down is just an inconvenience.
Only you can decide what is the right thing for you.

 

[neopolitan6]

Thank you very much! I'm learning…

a good introduction might be these resources:

I'm working through them. Thank you.

redundancy isn’t backup

Thank you for the clarification. In trying to plan the system, I would design it like this:
1) primary data, on FreeNAS (using some level of redundancy)
2) the local backup, on a second box
3) offsite, at home, on a third box

I'm trying to figure out cost/benefits for how to implement them.

How valuable is the data? If it were lost, how much would that impact you or the organization?

Our data is not that valuable; an outage of a few days is acceptable (save our critical business databases, which would get backed up in several different ways; less than 2 GByte). Given that, I'm thinking that RAIDZ1 is sufficient - we can handle a failure with the backups and only be inconvenienced.

ZFS snapshots can be stored on other mediums, but I’m hardcore ZFS fanboy so I don’t use a middleman. Rsync to a different filesystem breaks the integrity chain.

You are suggesting that I consider a second FreeNAS system, rather than use the Synology? That would be duplicating the hardware of the first server, right? Benefit there is ZFS maintaining the data integrity of the backup I would assume.

No, because we don't use RAIDz1 for primary storage with drives larger than 1 TB

The recommendation against RAIDz1 is about playing it safe.

I'm trying to figure out how these relate. I'm considering using 4TB drives. You would recommend against RAIDz1 because of drive size?

Back to my design:
box 1: 5 x 4TB in RAIDZ2 (12 TB storage)
box 2: 4 x 4TB in RAIDZ1 (12 TB storage)
box 3: offsite - is it worth the cost to make this FreeNAS with RAIDZ1 also? or do something less expensive, like openmediavault with a more basic raid setup?

Just adding up the dollars…

 

[Chris Moore]

No. the math is not that linear and you must keep 20% free space because ZFS is a copy or write filesystem.

box 1: 5 x 4TB in RAIDZ2 (12 TB storage)

A pool of five, 4 TB drives in RAIDz2 would only give you about 8.2 TB of usable space. Add another drive, six total, and it brings you to 11.1 TB usable.

box 2: 4 x 4TB in RAIDZ1 (12 TB storage)

A pool of four, 4 TB drives in RAIDz1 would only give you about 8.1 TB of usable space. Add another drive, five total, and it brings you to 11.1 TB usable.

These are estimates generated with the calculator on this site, which is pretty close, but not perfect: https://wintelguy.com/zfs-calc.pl
You will also be able to save a little space through compression.

Just adding up the dollars…

You can build an entire FreeNAS server, with good quality components, for around $600, plus the cost of drives.

I put this list together a little while back, so the components may not be available, but you should be able to find similar:

CASE: Fractal-Design-Define-R5-FD-CA-DEF-R5-BK-Black-Silent-ATX-Midtower-Computer-Case - US $99.99
https://www.ebay.com/itm/263674323506

POWER: Corsair-Certified-CS-M-Series-CS650M-650W-80-Plus-Gold-Active-PFC-Modular-Power - US $64.99
https://www.ebay.com/itm/382130407495

System Board: Super Micro X9SCM-F Motherboard w/ Heatsink/Fan & I/O Shield - US $75.00
https://www.ebay.com/itm/273350608871

CPU: Intel SR00H Xeon E3-1230 3.20GHZ Socket 1155 - - US $55.09
https://www.ebay.com/itm/273313040616

Memory: 8GB Memory RAM for SuperMicro X9 Series - - US $82.00 * 2 = $164
https://www.ebay.com/itm/163130855012

Drive Controller: LSI-SAS-9211-8i-8-port-6Gb-s-PCI-E-Internal-HBA-Both-Brackets-IT-MODE - US $59.99
https://www.ebay.com/itm/152937435505

Drive Cables: Mini SAS to 4-SATA SFF-8087 Multi-Lane Forward Breakout Internal Cable - - US $12.99
https://www.ebay.com/itm/371681252206

Thermal Compound: Noctua NT-H1 Thermal Paste Grease Conductive Compound for CPU/GPU - US $6.95
https://www.ebay.com/itm/302624513215

I may have missed some accessorie, and I didn't include drives, but this should get you all the key components and it is at $539 as of 18 July 2018

Don't waste money on buying new components. It doesn't make the system better.
If you want something ready-made, iXsystems sells a small unit: http://www.freenas.org/freenas-mini/

 

[neopolitan6]

you must keep 20% free space because ZFS is a copy or write filesystem.

I was not aware of this - where can I learn more?

 

[Chris Moore]

I was not aware of this - where can I learn more?

I don't want it to sound cruel, like 'go read the manual', but there is not only a lot of good information in the manual, there are many links to even more information elsewhere. The manual is a great resource: http://doc.freenas.org/11/freenas.html

Then there is the 'useful links' button in my signature for quick access to the things I thought were useful.

Then there is the resources section of the forum: https://forums.freenas.org/index.php?resources/

 

[Chris Moore]

I was not aware of this - where can I learn more?

Another useful bit of reading:
https://www.zdnet.com/article/why-raid-5-stops-working-in-2009/

 

[neopolitan6]

Thank you. I will do more learning. And I really appreciate the parts list!

At this point, my question is: is it worth it to try to learn FreeNAS, or am I better off using a commercial NAS (Synology)? Are the benefits of ZFS and RAIDZ2 worth the work? (An honest question, not trying to be snarky.)

 

[Chris Moore]

I think it's the best way to secure my data.
To me, it's worth it.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[kdragon75]

the other point to be made when talking about backups and redundancy is to step back from the tech and look at the business impact. Can you afford if the data goes offline? For how long is that acceptable? How much can you afford to lose? Once you answer those questions you can look at how to best meet those goals and requirements. Some terms to look up; recovery point objective and recovery time objective. Be sure you fully understand ZFS before you build an array and start populating it with data. It can be hard and even imposable (without juggling data on extra disks) to change things later.
As for the question "Is it worth it?" YES. If you care about the data, yes it's well worth the time to learn new skills.

 

[neopolitan6]

If I wanted to learn about FreeNAS, learn and implement a system, but use something else for backup (like a Synology or openmediavault); is that seriously lacking? I don't want to be foolish, but I do want to utilize my available resources.

Can you afford if the data goes offline? For how long is that acceptable?

Yes, we can. A few days would be acceptable. Most of the critical data is on another server, and the FreeNAS would be backup. This is "production," but any mission critical data would be backed up to 2 other local devices and a cloud backup.