Raid with multiple pairs of various sizes of HDDs

[photomaton]

I have,
2 x SAMSUNG EcoGreen F2 HD154UI 1.5TB SATA 3.0Gb/s 3.5" Internal Hard Drive
2 x HITACHI 0A38016 1TB 7200 RPM SATA 3.0Gb/s 3.5" Internal Hard Drive
1 x Western Digital Caviar Green WD20EARS 2TB SATA 3.0Gb/s 3.5" Internal Hard Drive
in my HTPC currently running XBMC and ubuntu.

I am planing to move all these drives to a NAS and use a small 500 GB drive in the HTPC.
I have been trying to figure out the best possible raid configuration without loosing significant space (e.g half if I mirror the two pairs)
Can I use raidz without loosing 1TB from 1.5TB pair? does anyone have a suggestion as to the best Raid configuration (or not use raid)

I have googled this a lot, and I have reached an answer cause probably there is no answer!! but any help is appreciated.

thnx

 

[louisk]

Since you have a collection of odd size drives, I would suggest you do the following:
buy 1 2T drive
setup mirrors (in the same pool) for each pair of drives.
This will give you a mirror of 1.5T, a mirror of 1T, and a mirror of 2T, resulting in 4.5T (RAW).

 

[photomaton]

Since you have a collection of odd size drives, I would suggest you do the following:
buy 1 2T drive
setup mirrors (in the same pool) for each pair of drives.
This will give you a mirror of 1.5T, a mirror of 1T, and a mirror of 2T, resulting in 4.5T (RAW).

Thank you for you reply.
Can I use raid0 or raid1 on each pair?

Is there an implementation in zfs which will provide both speed (RAID0) and reliability (RAID1) for paired drives.

What is your opinion on 1+1+1.5+1.5 RAID5 / raidz (raid5 implementation in zfs) which will give me 3 TB of usable space (I loose 2TB) still more than mirroring all the drives. and then use 2 TB for non-essential data

 

[Daisuke]

It depends what you plan to store. Usually, a NAS is used to store large amounts of data like movies, media, etc. Think of it this way: if any of the disks break, do you care if the data is lost? Personally, I recommend you to use RaidZ2 with 6 disks and slowly replace each disk to the largest size you own now (2TB). A ZFS RaidZ2 setup will give you great redundancy, 2 disks can fail and your data is still intact. This will make the total space shrinked to the smallest disk size you have now. So if you add various disk sizes and the smallest one is 1TB, the rest of disks will be also downsized at 1TB. Once you replace your disks and all of them are 2TB, the available space will be uniform and fully maximized.

I use 6 x 2TB disks set as RaidZ2 and the available space is 7TB. Setting the disks as RaidZ1 will give you 9TB of usable space. However, the BIG danger with RaidZ1 is the disks failure. Some people think is OK to have one disk fail, they buy another one and resilver it. However, if another disk fails during the resilvering process (which is fairly common), ALL your data is lost. Definitely not worth the risk IMO.

All in all, RaidZ2 does not replaces a good backup. RaidZ2 is not 100% safe but it does provides a great security for your data. A good place to read more about best practices is the Solaris Wiki.

 

[photomaton]

Thanks for your response.

Once quick question (I am a RAID noob), (hypothetically) if I have 1+2+2+2 TB raid with raidz2 or raidz2, I will have 3 or 2 TB usable space (n-1)*lowest drive or (n-2)* lowest drive respectively. But when i replace the last 1TB with 2 TB, will zfs dynamically increase the space? or do I have to reassign the RAID config again and lose data?

 

[Daisuke]

RaidZ1: (((Number Of Drives) - 1) x (Smallest Disk Size)) - (Space Internally Used)
RaidZ2: (((Number Of Drives) - 2) x (Smallest Disk Size)) - (Space Internally Used)

In your 1+2+2+2TB scenario, you will end with approx. 2TB of usable space (more like 1.8TB IRL). Once you replace the 1TB disk, you will automatically increase the space to 4TB. The resilvering will take care of that for you, you don't need to perform any other fancy steps. That is based on the ZFS documentation, personally I used the proper disks from the start. Any ZFS gurus, please share your knowledge. :)

Based on Solaris recommendations, if you decide to stick with 3 or 5 disks you should use RaidZ1. For 4 or 6 disks use RaidZ2, 7 or 11 disks RaidZ3, etc. On Solaris site, they have a formula that allows you to calculate what is the recommended RaidZ for your setup.

It is important to understand that once you set a number of disks in your array, you CANNOT add more disks later. You can add drives to a volume, but not to a RaidZ1/2 group. If your volume is a 4 drive RaidZ2, you can add another 4 drive RaidZ2 in the future, giving you a RaidZ2+0. But you can't change the current volume to a 6 drive RaidZ2, for example.

Before I built my NAS, I did some research and checked what are my best options related to hardware and software. I decided that for the time being, 7TB will satisfy my needs for few years. By then, new technology will be available so I should be able to purchase larger disks and built a new box on top of the existing one. :)

Edit: If you go the RaidZ2 way, make sure you use at least 8GB of RAM.

 

[louisk]

Thank you for you reply.
Can I use raid0 or raid1 on each pair?

Is there an implementation in zfs which will provide both speed (RAID0) and reliability (RAID1) for paired drives.

What is your opinion on 1+1+1.5+1.5 RAID5 / raidz (raid5 implementation in zfs) which will give me 3 TB of usable space (I loose 2TB) still more than mirroring all the drives. and then use 2 TB for non-essential data

ZFS calls them mirrors. You just pick a pair, select mirror, input the name of the pool. repeat for each pair (make sure you use the same pool name).

 

[w1n78]

It is important to understand that once you set a number of disks in your array, you CANNOT add more disks later. You can add drives to a volume, but not to a RaidZ1/2 group. If your volume is a 4 drive RaidZ2, you can add another 4 drive RaidZ2 in the future, giving you a RaidZ2+0. But you can't change the current volume to a 6 drive RaidZ2, for example.

sorry i'm a noob and would like more clarification on this statement. so right now i have 3x 1TB hard drive in a raidz1. based on the statement above, i cannot add another hard drive to that pool? is that correct? i could however, upgrade the 3 hard drives to a larger capacity let's say 3x 2TB and resilver every time i introduce the new hard drive? correct? would it be quicker or preferred way, to get new hard drives that i will be using, create a new volume, then transfer the data over and create the shares? are my assumptions accurate? right now i have a bunch of different drives i want to use and create a volume using raidz1. then when hard drives start to go back down in prices, i will slowly buy 2TB drives. but after reading this, i really should wait until i have all the hard drives i need before creating a volume. i was always under the impression that i can create a volume with hard drives and continue to add hard drives to it but not replace.

 

[ProtoSD]

A volume/pool can have multiple vdevs, but a vdev which is 1 or more disks cannot change the number of disks it contains after it has been created.

In your case, the 3x 1TB disks are 1 vdev, and you can add another vdev to that pool, but it should contain an equal (not required) number of disks and the same type of redudancy (also not required). So if you decided to add a single disk vdev to your existing 3 disk raidz1 pool, you would lose your entire array if that new single disk vdev failed, regardless of whether your other vdev is raidz1. So when adding new vdevs to a pool, be consistent.

You can also do what you suggested and take the 3 new disks and create a new pool and then copy everything over to it. This is nice because it leaves you with a backup while the transition is occuring.

You can also do the other thing you suggested, replace each disk in your current pool/vdev one at a time with a new larger disk and wait for each disk to finish resilvering (which takes place automatically when you issue the "replace" command).

i was always under the impression that i can create a volume with hard drives and continue to add hard drives to it but not replace.

I'm almost afraid to answer this because I know it will be misconstrued (by someone). Sure, you can go ahead and add random disks into a pool as you get them, but you'll have no redundancy and you'll get unpredictable bad results. It's like putting 4 different of those little emergency spare tires on your car and they all came from different cars. The car might roll, but it'll be a disaster.

-- Proto

 

[ProtoSD]

This post below, as well a several others explain all of these rules:

http://forums.freenas.org/showthrea...on-how-to-best-to-setup-drives&highlight=vdev

There's also a question about it in the Unofficial FAQ in my signature below which also has links to some of those other posts.

 

[w1n78]

A volume/pool can have multiple vdevs, but a vdev which is 1 or more disks cannot change the number of disks it contains after it has been created.

In your case, the 3x 1TB disks are 1 vdev, and you can add another vdev to that pool, but it should contain an equal (not required) number of disks and the same type of redudancy (also not required). So if you decided to add a single disk vdev to your existing 3 disk raidz1 pool, you would lose your entire array if that new single disk vdev failed, regardless of whether your other vdev is raidz1. So when adding new vdevs to a pool, be consistent.

You can also do what you suggested and take the 3 new disks and create a new pool and then copy everything over to it. This is nice because it leaves you with a backup while the transition is occuring.

You can also do the other thing you suggested, replace each disk in your current pool/vdev one at a time with a new larger disk and wait for each disk to finish resilvering (which takes place automatically when you issue the "replace" command).



I'm almost afraid to answer this because I know it will be misconstrued. Sure, you can go ahead and add random disks into a pool as you get them, but you'll have no redundancy and you'll get unpredictable bad results. It's like putting 4 different of those little emergency spare tires on your car and they all came from different cars. The car might roll, but it'll be a disaster.

-- Proto

awesome, it makes more sense now. so i just created my 3x 1TB raidz1 last night and started to add some data for testing. and now this morning while cleaning up, i found a usb enclosure. inside is a 1TB drive. being that i'm just testing, i should wipe out the current volume i have, add the new 1TB drive and create a new 4x 1TB raidz1 (ignoring the benefits of raidz2 for the time being). if i add the new 1TB to the existing 3x 1TB and it fails, it will kill pool. hope i understood that correctly. sorry i'm a bit slow on this. i think i'll just recreate the volume and continue testing.

in the end i really want to run 4x 2TB drives - at least but for the moment i can live with the 4x 1TB until prices come down. i do have a couple of 200-500GB lying around but have to figure out if it's worth plugging in considering energy consumption and i doubt i can fill the volume i currently have at the moment.

thanks for the reply.