RAIDZ1 Vs RAIDZ2 In my config

[Spencer Skinner]

Hi guys,

Like many people i am new coming into the world of FreeNAS, ive done a fair bit of read and been doing allot of my own testing on some old hardware that i had lying around, ive pretty much come to my final choices about how i am going to set everything up.

Except i cant work out what i should do from my RAID solution. Essentially its pretty much a case of cost for me.

The choices ive given myself are these

RAIDZ1 - 3 x 3TB HGST H3IKNAS30003272SE Drives. This would cost me £337 in drive cost

OR

RAID Z2 - 5 x 2TB WD Red WD20EFRX Drives. This would cost £414 in drive cost

This is a wake of about £77 between the two options. I will agree that option 2 is both safer and more upgrade-able in the future. BUT that £77 could be used to get a better external drive to do my external backups on, or it could be used for a better UPS in the case of a power outage.

My question is do i go for the more expensive RAID option and potentially then spend less on UPS kit and External backups, or do i go the cheaper RAID option and get better UPS kit and external backup kit?

My use case if fairly simple, i am a home user. Ill be keeping all my programming projects on here, photos, music, movies, game saves, game data. My brother will also be using this as a backup for all of his design work, he is a Textile Designer, he has photoshop files ranging from 500MB to 8GB to store and move about, he wont be working on the ones stored in the NAS, thats purely backup. We will be streaming allot of the media stored on the server around the house to TV's, laptops etc. Movies and Music especially. All in all both the RAID solutions shouldnt cause any bottle necks to anything im doing.

Sorry for this being quite a trivial question for allot of you, would just like some other well informed opionions.

Thanks
Spencer Skinner

 

[mattbbpl]

RAIDZ2. RAIDZ1 is dead.

Edit: My apologies, that was unacceptably brief. I was distracted by an Oracle deployment as I began the response.

With modern drive sizes, you are too likely to encounter an error from the parity data when you attempt to rebuild a failed drive. During this process, you have no parity against the data being restored, so if that data encounters an error it cannot be rebuilt. That data will be lost.

Having two copies of parity data (accomplished by using RaidZ2) prevents this situation.

 

[Spencer Skinner]

Ahhh right so using RAIDZ2 is a much safer option, so going for 5 x 2TB is my choice. Also which drive do you recommend 2TB WD Red OR 2TB Seagate ST2000 VN000. The Seagate drives reduce the cost of drives to £380. Which reduces the cost nicely? Which drive it better?



Sent from my ONEPLUS A3003 using Tapatalk

 

[Spencer Skinner]

Correction to the original cost of the WD Reds, for option two they are £430. And the Seagate's for option two are £380

Sent from my ONEPLUS A3003 using Tapatalk

 

[Pitfrr]

You could mix both types of drive as well.
Some members would prefer WD against Seagate, this depends on your experience as well. I do have a mix between WD Red and Seagate NAS drives.

Personally I would go for 6 drives in RAIDZ2 (I know it will blow your budget up). I find that 6 drives in RAIDZ2 is a sweet spot between optimising space and the SATA ports usually available (without adding an HBA).
Also if you want to expand later on, you could do it by adding a new pool (of several drives, let's say again 6 drives in RAIDZ2) to the existing one without having to recreate it (if you expand from 5 to 6 drives for example, it can only be done by recreating your pool).

 

[Spencer Skinner]

Yeh i dont have the money to go for 6 drives to be honest and even with 5 i have allot of room to play with, especially seeing as i am only using 2TB drives at the moment, i can easily double by going up to 4TB drives and that doesnt require me re building the VDev just replacing the drives instead. Although i do entirely see your point. As for mixing the drives, i assume your saying to do that so that i dont get drives from all one batch, so when does fail, the likelyhood of another failing is smaller because they are a different batch? Would that not cause some issues when seeking data and reading/writing etc, being that they have different speeds? I am a bit of a scrub when it comes down to all this, my questions may be silly.

 

[mattbbpl]

Ahhh right so using RAIDZ2 is a much safer option, so going for 5 x 2TB is my choice. Also which drive do you recommend 2TB WD Red OR 2TB Seagate ST2000 VN000. The Seagate drives reduce the cost of drives to £380. Which reduces the cost nicely? Which drive it better?



Sent from my ONEPLUS A3003 using Tapatalk

Some people have a thing against Seagate due to past experiences. Others argue that current drives do not represent their past issues.

I'd argue that you should go with the cheaper option as Seagate does indeed seem to be past the issues that plagued their older drives. But that's just me.

 

[joeschmuck]

Thought I'd chime in here...

With respect to the two different hard drives... The HGST drives run at 7200 RPM and thus will create more heat than the WD Red drives. Here in the US the 3TB WD Red is currently cheaper than the HGST, not sure about where you live. Of course I'm partial to the WD Red because my luck has been very good. I'm running on just over 4 years of continuous run time. So pay attention to cooling. But I have nothing against HGST, I'm sure they are fine too.

With respect to RAIDZ1 or RAIDZ2, I'm just repeating what has previously been said because it does matter. There is a huge risk when using RAIDZ1 when a hard drive fails as you could loose everything if a second failure occurs during the rebuild. As for speed of the rebuild, the 7200 RPM drives will resilver faster than the 5200 RPM drives of course but I don't have the facts on the speed differences. A RAIDZ2 gives you better fault tolerance.

So lets talk capacity... You are looking at 4.4 TB of storage capacity using either configuration (5.5 - 20%). Is this really enough to last you for at least 3 years? You want the best value and you should ensure your capacity at least is covered for the warranty period of the drives. So you need to figure out what type of capacity you really need. If you are ripping blueray movies then expect to need a lot of storage, but DVD rips are much less consuming.

While I fully understand that money is a big factor, it's not the only factor. My advice is to buy what you can afford now and if it takes you another few months to buy the remaining hard drives, then that is what you do. But if you want a system now, then you could build it now, play around with it and test it out, then buy additional drives and when you are ready, destroy and rebuild the vdev at a later date and restore your data, once you can afford those extra hard drives. I purchased four drives initially (they cost a lot 4 years ago) and then realized I really needed two more drives and a few months later I recreated my vdev with six hard drives. I'm very pleased with my setup and capacity.

Speaking of mixing batches of hard drives... Some folks do this. There are pros and cons to this, but these days it's all about the warranty for a NAS for home use but if you were a company then warranty doesn't mean anything because you wouldn't be shipping back the hard drive because it would have company information on it and while these hard drive companies might not care about your data, if they took those drives and sold them as failed units, someone could extract data from them.

The UPS argument of buying a better or larger capacity. You need to buy the right one for your area. If you get a lot of power outages then a better quality unit will be needed. Don't skimp on your UPS. This is the same as the main components of your system, if you buy cheap parts then you could be putting all your data at risk and that is a waste of money.

To summarize: Ensure you are buying for the capacity and redundancy you desire.

 

[Spearfoot]

Have considered using a simple mirrored pair of larger (4 or 6TB) drives?

 

[Spencer Skinner]

Thought I'd chime in here...

With respect to the two different hard drives... The HGST drives run at 7200 RPM and thus will create more heat than the WD Red drives. Here in the US the 3TB WD Red is currently cheaper than the HGST, not sure about where you live. Of course I'm partial to the WD Red because my luck has been very good. I'm running on just over 4 years of continuous run time. So pay attention to cooling. But I have nothing against HGST, I'm sure they are fine too.

With respect to RAIDZ1 or RAIDZ2, I'm just repeating what has previously been said because it does matter. There is a huge risk when using RAIDZ1 when a hard drive fails as you could loose everything if a second failure occurs during the rebuild. As for speed of the rebuild, the 7200 RPM drives will resilver faster than the 5200 RPM drives of course but I don't have the facts on the speed differences. A RAIDZ2 gives you better fault tolerance.

So lets talk capacity... You are looking at 4.4 TB of storage capacity using either configuration (5.5 - 20%). Is this really enough to last you for at least 3 years? You want the best value and you should ensure your capacity at least is covered for the warranty period of the drives. So you need to figure out what type of capacity you really need. If you are ripping blueray movies then expect to need a lot of storage, but DVD rips are much less consuming.

While I fully understand that money is a big factor, it's not the only factor. My advice is to buy what you can afford now and if it takes you another few months to buy the remaining hard drives, then that is what you do. But if you want a system now, then you could build it now, play around with it and test it out, then buy additional drives and when you are ready, destroy and rebuild the vdev at a later date and restore your data, once you can afford those extra hard drives. I purchased four drives initially (they cost a lot 4 years ago) and then realized I really needed two more drives and a few months later I recreated my vdev with six hard drives. I'm very pleased with my setup and capacity.

Speaking of mixing batches of hard drives... Some folks do this. There are pros and cons to this, but these days it's all about the warranty for a NAS for home use but if you were a company then warranty doesn't mean anything because you wouldn't be shipping back the hard drive because it would have company information on it and while these hard drive companies might not care about your data, if they took those drives and sold them as failed units, someone could extract data from them.

The UPS argument of buying a better or larger capacity. You need to buy the right one for your area. If you get a lot of power outages then a better quality unit will be needed. Don't skimp on your UPS. This is the same as the main components of your system, if you buy cheap parts then you could be putting all your data at risk and that is a waste of money.

To summarize: Ensure you are buying for the capacity and redundancy you desire.

Wait, why would i only have 4.4TB of space with a RAIDZ2 on 5 x 2TB, isnt that 6TB of useable space because only 2 of the 5 drives are taken up by parity, as for the money situation, ive got the money to spend and i can push if i really want to, but id rather keep the cost controlled for the meantime. As for the drive mixing yeh, i see your point, not really a need for me to do that, i can just get a bunch of WD Reds or Seagates they bother have 36 Month warranties.

As for the time scale of how i want to do it, id prefer to get it all setup and build in one big hit, rather than leave it a few months.

As for the drives, if i have to get the 2TB drives, then ill get the WD Reds or Seagates, if i move any higher than 2TB then ill move straight to HGST.

I think its going to be a definite RAIDZ2 because of the safety of it and its worth the money for the future proofing tbh. The rest of the hardware is all pretty high spec im not cutting cost there at all, its all ECC and buffered memory, Intel Gb NIC etc etc. Cooling wise im not going to cut cost there either, gonna try and get a bunch of really good BeQuiet Fans and stuff like that.

 

[Spencer Skinner]

Have considered using a simple mirrored pair of larger (4 or 6TB) drives?

Yeh that's a fair point but I don't get as much redundancy with that so both can easily fail

Sent from my ONEPLUS A3003 using Tapatalk

 

[Pitfrr]

About the available space: if you're using 5x2TB in RAIDZ2 that will leave you with 3x2TB usable i.e. 6TB. Right.... but you have to consider that you shouldn't fill your pool more than 80% of its capacity. And that would be 4.8TB.

I don't have the link right now but search in the forum there is a RAIDZ calculator that will give you all the detailed space availability depending on your choices.

 

[joeschmuck]

Wait, why would i only have 4.4TB of space with a RAIDZ2 on 5 x 2TB, isnt that 6TB of useable space because only 2 of the 5 drives are taken up by parity

Ah, you haven't done your homework. Your configuration brings it to 5.5TB of usable space and that is what you will see. Now subtract 20% to keep the file system working well equals 4.4 TB of storage. If you reach the 10% free point the file system will change modes to organize the data placement better and be considerably slower for writing operations. So if you plan to keep at least 20% free then you will be fine. My advice to people building a system for the first time is to figure out what capacity you desire for the 3 year warranty period of the drives and then double it if you can. This is because most users may have good intentions but once all that storage becomes available, they tend to store more crap that they really don't need but that is what happens. I got lucky, while I do store some crap, I do delete a lot of the old stuff periodically. Also, most of my storage is use on multiple computer backups, not media files.

I think its going to be a definite RAIDZ2 because of the safety of it and its worth the money for the future proofing tbh.

Good call!

Cooling wise im not going to cut cost there either, gonna try and get a bunch of really good BeQuiet Fans and stuff like that.

You don't have to go crazy with fans, you just need to plan the air flow out. Even a single fan could cool the entire case if structured properly. And of course try to use the largest fans you can to cut down on noise. I have two case fans directly in front of my hard drives pushing air into the case and then the only exhaust fan is in the power supply. My case also allows heat to escape through the top. So just plan it out the best you can.

As for the time scale of how i want to do it, id prefer to get it all setup and build in one big hit, rather than leave it a few months.

If you can afford it then buy the correct number of hard drives up front but figure out your capacity. As you said before, if you need to add capacity at a later date you could replace your hard drives with larger ones, but hopefully that is in 3+ years.


And I also was thinking mirrors as well but for the 5.5TB capacity you would need to buy three 6TB hard drives and mirror all three of them. This would give you 2 drive redundancy using less hard drives. Of course normally I'd recommend this to someone who needs to conserve space.

 

[joeschmuck]

About the available space: if you're using 5x2TB in RAIDZ2 that will leave you with 3x2TB usable i.e. 6TB. Right.... but you have to consider that you shouldn't fill your pool more than 80% of its capacity. And that would be 4.8TB.

I don't have the link right now but search in the forum there is a RAIDZ calculator that will give you all the detailed space availability depending on your choices.

RAID Calculator Links in my signature.

 

[Pitfrr]

Thanks for the link reminder!

 

[Spencer Skinner]

Ah, you haven't done your homework. Your configuration brings it to 5.5TB of usable space and that is what you will see. Now subtract 20% to keep the file system working well equals 4.4 TB of storage. If you reach the 10% free point the file system will change modes to organize the data placement better and be considerably slower for writing operations. So if you plan to keep at least 20% free then you will be fine. My advice to people building a system for the first time is to figure out what capacity you desire for the 3 year warranty period of the drives and then double it if you can. This is because most users may have good intentions but once all that storage becomes available, they tend to store more crap that they really don't need but that is what happens. I got lucky, while I do store some crap, I do delete a lot of the old stuff periodically. Also, most of my storage is use on multiple computer backups, not media files.


Good call!


You don't have to go crazy with fans, you just need to plan the air flow out. Even a single fan could cool the entire case if structured properly. And of course try to use the largest fans you can to cut down on noise. I have two case fans directly in front of my hard drives pushing air into the case and then the only exhaust fan is in the power supply. My case also allows heat to escape through the top. So just plan it out the best you can.


If you can afford it then buy the correct number of hard drives up front but figure out your capacity. As you said before, if you need to add capacity at a later date you could replace your hard drives with larger ones, but hopefully that is in 3+ years.


And I also was thinking mirrors as well but for the 5.5TB capacity you would need to buy three 6TB hard drives and mirror all three of them. This would give you 2 drive redundancy using less hard drives. Of course normally I'd recommend this to someone who needs to conserve space.

Right so in that case it is probably best to add a drive and then go for 6 x 2TB, this fills up my Mobo's capacity and leaves me with masses of room for expansion in future. I wasnt aware of the rule of leaving that much space extra on the side for performance, ill bare that in mind now, so for example if i did 6 x 2TB that should leave me with just under 6TB of usable storage available, yes?

And again the drives would probably be the Seagate ones because ive not seen any glaring faults with them so far in my research and they are slightly cheaper, considering that im buying 6, having that saving would be nice not going to lie.

As for the cool im using a Fractal R4 so ive got enough room to play with airflow if i want to.

Id like to avoid using fewer larger drives because of cost and lack of ease of upgrading in the future, although that said i do like that idea, just the cost really.

 

[joeschmuck]

I'm running six 2TB drives and in a RAIDZ2 configuration you get 7.3TB minus 20% (1.46) equals 5.8TB. Now if you are using your NAS primarily as a media server and some backups, you could use the 10% level and gain yourself about .7TB so you would have 6TB of space and these are rough approximations. If you hit the 10% mark, as I said you will see a slow down in write performance but that doesn't mean your system isn't going to be fine for home use but it will be a warning to either clean up your data or make a larger vdev.

Also know that when you create your pool/vdev, it will by default consume 2GB of swap space from each hard drive in the system. I wouldn't recommend changing this but you should be aware that you will see a slight loss, if you start trying to diagnose where all the hard drive space is.

Lastly, by default lz4 compression is turned on and I'd recommend leaving it that way unless you know what you are doing. So if you are storing files which can be compressed then you will effectively be able to store more. If the files cannot be compressed then you gain nothing extra.

If you haven't already played with FreeNAS then I strongly suggest that you do and before you commit to your data structure. You can create several datasets to keep your data separated and that can be a very good thing for organization. For example I have a dataset for media where my movies and music reside, I have a dataset for personal data, and I have a dataset for my daughters use (mostly college data and some photos).

I hope everything turns out well for you using FreeNAS. It's not a novice program but if you have problems, please do a Google type search (per our forum rules) to see if you can find the answer before just asking for help. Also keep backups of your configuration file and any important data should also be backed up elsewhere, unless FreeNAS is your backup.

Good Luck!

 

[Spencer Skinner]

I'm running six 2TB drives and in a RAIDZ2 configuration you get 7.3TB minus 20% (1.46) equals 5.8TB. Now if you are using your NAS primarily as a media server and some backups, you could use the 10% level and gain yourself about .7TB so you would have 6TB of space and these are rough approximations. If you hit the 10% mark, as I said you will see a slow down in write performance but that doesn't mean your system isn't going to be fine for home use but it will be a warning to either clean up your data or make a larger vdev.

Also know that when you create your pool/vdev, it will by default consume 2GB of swap space from each hard drive in the system. I wouldn't recommend changing this but you should be aware that you will see a slight loss, if you start trying to diagnose where all the hard drive space is.

Lastly, by default lz4 compression is turned on and I'd recommend leaving it that way unless you know what you are doing. So if you are storing files which can be compressed then you will effectively be able to store more. If the files cannot be compressed then you gain nothing extra.

If you haven't already played with FreeNAS then I strongly suggest that you do and before you commit to your data structure. You can create several datasets to keep your data separated and that can be a very good thing for organization. For example I have a dataset for media where my movies and music reside, I have a dataset for personal data, and I have a dataset for my daughters use (mostly college data and some photos).

I hope everything turns out well for you using FreeNAS. It's not a novice program but if you have problems, please do a Google type search (per our forum rules) to see if you can find the answer before just asking for help. Also keep backups of your configuration file and any important data should also be backed up elsewhere, unless FreeNAS is your backup.

Good Luck!

Yeh i have been trying out lots of different dataset, and user configs on my test system over the last few weeks, as for permissions and shares im all set to get that working. Yeh having 6.5TB tops is absolutely fine for me id say tbh.

As for the settings for compression etc, yeh ill be leaving them as they are, might activate data de-duplication on the media dataset because that will inevitably get spammed with stuff.

Yeh im fully aware of the steep learning curve with FreeNAS, hence why I chose it, its very good and it will teach me allot as time goes on. And yes of course i do try and keep to avoiding pestering people in forums for help as and when i can hahaha.

And yeh ill be keeping a config backup file so i dont loose everything ahaha.

Thank you allot for your help, I appreciate it allot.

Have a merry christmas too!

Thanks

 

[joeschmuck]

might activate data de-duplication

You need to read up on deduplication before you enable it. I have no idea how much RAM you expect to have but you better have a ton of it and also understand how much space you could loose in the deduplication tables that would be created. Read up on it.

Merry Christmas to you as well!

 

[Spencer Skinner]

You need to read up on deduplication before you enable it. I have no idea how much RAM you expect to have but you better have a ton of it and also understand how much space you could loose in the deduplication tables that would be created. Read up on it.

Merry Christmas to you as well!

Im gonna use a single DIMM of DDR4 2133Mhz CAS 11, ECC and Buffer Registered memory. Keeping it to a single DIMM so that i can easily up it to 32GB as and when i need to. But yeh ill take your advice there and do some reading because evidently so far ive needed to do some more reading ahaha.

Hope you get all the presents you want!