What are the chances of a RaidZ1 array failing during rebuild?(and more)

[mike360x1]

Hi guys,

I'm quite new to FreeNAS but I have done my fair share of research. I'm am looking into setting up a home FreeNAS server with 4 Drives in a RAIDZ1 array with sizes of: 500GB,500GB,1TB,3TB.

Context(Don't have to Read):
I understand that this configuration is quite inefficient seeing as I have two low capacity drives paired with two relatively higher capacity (and performance) drives. I am still saving up to buy a few proper WD RED 3TBs to replace the other three disks in the array in the future (the single 3TB is a WD RED).

However, seeing as this machine will be used in a home environment with three other family members, I figure that it shouldn't be a huge deal because ZFS allows for the replacement of old drives with larger capacity drives in the future.

Questions:
Though, my concerns are,

1. Going with this configuration, how much of the drives will be usable space? Does ZFS treat each drive as a 500GB disk giving me a total of only 1TB of space? (give or take).

2. I read on the forums that for a RAIDZ1 array, a max of 1TB per disk is suggested due to the long rebuild times and the chance of failure on another drive during the rebuild. (cause bigger size = longer rebuild times) Should I be concerned? Also, the non WD RED disks aren't designed for NAS use and they may not have the proper warnings or reporting tools that a NAS drive has which may have the drive failing without warning and thus having a greater chance of disk failure during rebuilding.
(I figure the chances are low for a 4 disk setup with a low - moderate use, but I just want to be ABSOLUTELY sure.)

 

[Robert Trevellyan]

Does ZFS treat each drive as a 500GB disk giving me a total of only 1TB of space?

Correct.

Should I be concerned?

Depends on how much you value the data. Whatever you do, you'll need backups of anything you care about.

the non WD RED disks aren't designed for NAS use and they may not have the proper warnings or reporting tools that a NAS drive has which may have the drive failing without warning and thus having a greater chance of disk failure during rebuilding.

Not correct. The key difference is that NAS drives support TLER or equivalent, so a failing NAS drive won't make the pool completely unresponsive.

 

[rs225]

2. I read on the forums that for a RAIDZ1 array, a max of 1TB per disk is suggested due to the long rebuild times and the chance of failure on another drive during the rebuild. (cause bigger size = longer rebuild times) Should I be concerned?

Use whatever size you please. Make sure your pool runs a scrub on some kind of automatic schedule, and emails you of trouble. Your chance of a drive failing during a resilver is equal to the chance of a drive failing during a scrub. In other words, close to zero. The scare-talk stems from a scenario of no scrubs, which even then, raidz handles many problems far better than RAID.

Always have a backup. It's better than any level of pool redundancy.

 

[AlainD]

Why only 4 disks?

The cost for the server grade MB and ECC ram, makes two extra 3TB drives not a big expense.

I had a drive "failing" on my while resilvering. The HDD is all right, but I suspect a loose "touched" cable. I was very pleased that I used RAIDZ2, the resilver just went on.

Also label you're drives very well before adding data to them. The serial numbers are needed to find the correct drive, but are often not visible while installed. I have the impression that this has improved with SN also on the short end of the HDD.

 

[Mirfster]

2. I read on the forums that for a RAIDZ1 array, a max of 1TB per disk is suggested due to the long rebuild times and the chance of failure on another drive during the rebuild. (cause bigger size = longer rebuild times) Should I be concerned?

Yes, because of this:

I figure that it shouldn't be a huge deal because ZFS allows for the replacement of old drives with larger capacity drives in the future.

So "in the future" you will run into the situation that the drives are >= 1TB. Now you will put yourself in a very vulnerable spot and had plenty of time to copy vital data to the Server... Save yourself the headache and consider at least RaidZ2.

 

[mike360x1]

Yes, because of this:

So "in the future" you will run into the situation that the drives are >= 1TB. Now you will put yourself in a very vulnerable spot and had plenty of time to copy vital data to the Server... Save yourself the headache and consider at least RaidZ2.

Why only 4 disks?

The cost for the server grade MB and ECC ram, makes two extra 3TB drives not a big expense.

I had a drive "failing" on my while resilvering. The HDD is all right, but I suspect a loose "touched" cable. I was very pleased that I used RAIDZ2, the resilver just went on.

Also label you're drives very well before adding data to them. The serial numbers are needed to find the correct drive, but are often not visible while installed. I have the impression that this has improved with SN also on the short end of the HDD.

Everyone is scaring me :'(.

I'm just a casual user with an invested interest in Freenas. I've searched up similar threads and everyone else is saying the same thing. I know that I should always have a backup and ZFS isn't foolproof. I have exactly that (the backup) but they all make it sound like there is going to be a drive failure within the first month of setting up a new pool. :(

Are there stats and failure rates of drives documented? Or is there any place I can go to see other people's stories about failed drives and how often that happened to them?

Because I only have 4 drives in this array (without any ability to change raid type or drive size in the future), a RAIDZ2 array will eat up 50% of my storage as opposed to a 6 disk array with the same redundancy.

Use whatever size you please. Make sure your pool runs a scrub on some kind of automatic schedule, and emails you of trouble. Your chance of a drive failing during a resilver is equal to the chance of a drive failing during a scrub. In other words, close to zero. The scare-talk stems from a scenario of no scrubs, which even then, raidz handles many problems far better than RAID.

Always have a backup. It's better than any level of pool redundancy.

Thanks for that reassurance. I do and will always have a backup for my important data : D

 

[Mirfster]

I have exactly that (the backup) but they all make it sound like there is going to be a drive failure within the first month of setting up a new pool. :(

• If you have and maintain a backup, then that is a good thing
  
• Performing a proper burn-in will help as well
  
• Make sure you schedule SMART Tests (Short and Long) as well as Scrubs
• Set/Configure E-Mail (as well as the "root" user's e-mail address) and you will be notified
  
• Tie it all off with @Bidule0hm 's "Scripts to report SMART, ZPool and UPS status, HDD/CPU T°, HDD identification and backup the config"
• Tada, you can now feel safe. ;)

Are there stats and failure rates of drives documented? Or is there any place I can go to see other people's stories about failed drives and how often that happened to them?

• Circa 2007 and they were talking about 2009:http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/
• I provided more info in this thread: https://forums.freenas.org/index.php?threads/freenas-box-disk-strategy.43475/#post-286534
• Some info here, there is a BackBlaze Report: https://forums.freenas.org/index.php?threads/seagate-being-sued-for-shoddy-hard-drives.41151/
  
• WD Reds are seemingly the most recommended here, I use HGST... YMMV

Because I only have 4 drives in this array (without any ability to change raid type or drive size in the future), a RAIDZ2 array will eat up 50% of my storage as opposed to a 6 disk array with the same redundancy.

Understood. There is always a decision that needs to be made by the Admin/Owner between Space, Speed and Redundancy. Not trying to scare you, just trying to provide you with knowledge to make an informed decision. This is not any limitation on FreeNas/ZFS it is the same across the board. It is just here we are more "up front".

 

[Mirfster]

Afterthought: You never mentioned the specs of the machine you were going to run this on. At the very least read "Hardware recommendations (read this first)" and make sure you are not trying to put this in a system that is going to get you here

 

[mike360x1]

@Mirfster

-I did not know about burn-in thanks!! I will absolutely look into it.

-Scrubs + Smart Tests w/Smart reporting sending emails every day. [Check!]

-Great thanks for that info : D

-I realize that just now XD

-How rude of me, I apologize. They are as follows:

Motherboard: Asrock c2550d4i w/ Intel Avoton C2550
Ram: Crucial DDR3 1600 ECC 8GB*2 [Module:CT102472BD160B.18FED]
PSU: Antec EA-550 Green
Case: Old case taken from a unused Intel P4 mobo. Provides adequate cooling but I will getto mount extra fans inside.

Did I miss anything important? :)

Thanks for your very informative help Mirfster. I think I can rest at ease tonight : )

 

[ethereal]

i think you should definitely think about cooling in the case. the hotter the hard drives get the earlier they are likely to fail - sometimes in quick succession.

i have a full tower with 10 hdds and 1 ssd - 7 fans and i have never had any problems with high temps (under 30 degrees)

 

[Robert Trevellyan]

a RAIDZ2 array will eat up 50% of my storage

after running this through the ZFS-police translator

With a 4-disk RAIDZ2 array, I'm allocating 50% of my raw storage to improving the reliability of my system.

 

[fta]

i have a full tower with 10 hdds and 1 ssd - 7 fans and i have never had any problems with high temps (under 30 degrees)

According to google's data, that's actually too cool and leads to a higher failure rate. Their data suggests the ideal temp is 35-40C.

 

[Mirfster]

According to google's data, that's actually too cool and leads to a higher failure rate. Their data suggests the ideal temp is 35-40C.

I ain't buying it... :P Besides, I can't keep my drives below 30 anyways. ;)

 

[Ericloewe]

ZFS isn't foolproof

Nothing is.

 

[Mirfster]

Nothing is.

To the Cloud!!!... Opps ISP went down... Whelp there goes that...

 

[rs225]

after running this through the ZFS-police translator

With a 4-disk RAIDZ2 array, I'm allocating 50% of my raw storage to improving the reliability of my system.

And after Something Bad Happens it becomes:

"Why did I waste 50% of my capacity when I could have just made a full backup?"

 

[ethereal]

According to google's data, that's actually too cool and leads to a higher failure rate. Their data suggests the ideal temp is 35-40C.

https://www.backblaze.com/blog/hard-drive-temperature-does-it-matter/

What Other People Say
Google and Microsoft have both done studies on disk drive temperature in their data centers. Google found that temperature was not a good predictor of failure, whileMicrosoft and the University of Virginia found that there was a significant correlation.

Disk drive manufacturers tell Backblaze that in general, it’s a good idea to keep disks cooler so they will last longer.

 

[Ericloewe]

Google found that temperature was not a good predictor of failure,

Their conclusion does not follow from their data.

High temperatures absolutely do correlate with failures. It's just the lower range (up to 40 degrees) that doesn't have any significant correlations.

 

[mike360x1]

Thanks guys, I will be running a few extra fans in the system so hopefully there wont be any problems. Thanks for your insight.

 

[ethereal]

Their conclusion does not follow from their data.

High temperatures absolutely do correlate with failures. It's just the lower range (up to 40 degrees) that doesn't have any significant correlations.

i didn't quote fta correctly - he said According to google's data, that's actually too cool and leads to a higher failure rate. Their data suggests the ideal temp is 35-40C.


i said

i think you should definitely think about cooling in the case. the hotter the hard drives get the earlier they are likely to fail - sometimes in quick succession.
i have a full tower with 10 hdds and 1 ssd - 7 fans and i have never had any problems with high temps (under 30 degrees)

Microsoft and the University of Virginia found that there was a significant correlation - higher temp and higher failure
Disk drive manufacturers tell Backblaze that in general, it’s a good idea to keep disks cooler so they will last longer.