FreeNAS 9.2 - Can't create a RaidZ ZFS Volume

[zevoman]

I've been using FreeNAS for years. I've consistently been upgrading the server with the latest NAS builds and last week upgraded to 9.2 from 8.3. I've been running a 3-HDD (1-2TB and 2-1TB drives) RaidZ1 ZFS volume, but wanted to swap in a new 2 TB drive to replace an older 1 TB drive. Rather than re-slivering, I just wanted to start with a clean volume. I backed up all my data and destroyed my existing volume to start with a new, fresh one. However, when I select the ZFS volume manager, I am not able to use all 3 HDDs together. It is only letting me create a mirror with 2 drives and a stripe with the remaining. This is driving me crazy. Any ideas?

 

[gpsguy]

Create the volume with 8.3.2. Then revert to 9.x.

With version 9.x they took away the ability to create RAIDz volumes with different sized disks.



Sent from my phone

 

[cyberjock]

Yeah, you can't do that. 9.x won't let you do things it thinks are "stupid" like mixing drives of different sizes. You'll have to make the pool in 8.3 :(

Feel free to complain at https://bugs.freenas.org/issues/3274

 

[warri]

Yes, we get this question multiple times a week. Also you could use the forum search next time before posting ;)
Maybe worth a sticky, or an entry in the FAQs..?

 

[zevoman]

Well that would have been nice to know before I destroyed my volume. I read through the whole manual (well most of it) and didn't see that anywhere (hardware requirements or in the ZFS volume section). Thanks for letting me know. I've been needing to replace the 2 1TB disks, but thought I could last a little longer. Guess I need to replace the other one now. Thanks again!

 

[cyberjock]

Do keep in mind you can replace disks without this problem. The check is on pool creation. This volume check has really pissed off many people because they buy disks from 2 manufacturers and one is 1000GB and another will be 1000.1GB. The volume manager sees these disks as dissimilar and won't let you mix them.

 

[zevoman]

Seriously? Not only do they have to be the same size, they must also be from the same manufacturer to avoid slight differences? Well, I think I might revert back to 8.3 for the time being then (can you tell all three of my disks are from different manufacturers?)...

 

[cyberjock]

I have dealt with my problem by using 8.3.1 for pool creation, then upgrading to 9.x. I never did a zpool upgrade so if I had to make more changes I could use 8.3.1 again. There's no signficant penalty right now for not upgrading from v28 to v5000, so I'm just hanging out in v28 land. Potentially forever since there's no ETA on when the volume manager will be fixed.

What bothers me the most about the whole thing is that a simple checkbox that could let you ignore disk sizes would solve this problem. Yet, the ticket remains open...

 

[zevoman]

Great idea! Seems like it would be a simple fix, but I guess when you're buried under a pile of tickets, you tend you ignore the easy ones...

 

[ZFS Noob]

Can someone explain you'd want to do this? I've always created RAID sets of identical drives - same make, model, capacity, rotational speed, etc.

I've been doing this for a while so maybe my thinking is archaic, but I don't see a problem with the default settings here. Well, I'd think it's reasonable to use larger replacements for failed drives in a vdev, but other than that I'm having problems understanding why you'd want to combine 1TB drives with 2TB drives in the same parity RAID vdev. A stripe across a mirrored vdev of 1 TB drives, and another of 2TB drives, and another of 3TB drives makes sense even though it's not optimal. I can even understand drives of ~ the same rotational speed from different vendors so you don't get bit hard when the next DeathStar drive fiasco happens.

But why create a RAIDZ2 set of drives with different capacities?

 

[cyberjock]

Some people want to use spare drives to build their RAIDZ2, then after moving their data put their bigger disks(that have data on them) in the pool.

Some people want to use a bunch of spare drives around their house just because they are otherwise collecting dust.

Still others have old RAIDs where they had 1TB drives, then did a disk replacement with a 2TB drive. Now they are wanting to move to ZFS and they're forced to buy either a 1TB drive or all 2TB drives because of this limitation.

The bottom line is that I don't feel it is "the project's" job to keep you from doing weird things that are still safe for your data. If you want to make a 10 disk RAIDZ2 pool where every single disk is a different size, that's your prerogative. It might not be the best way to do things, but there's nothing dangerous about doing it. The "new" volume manager came about because too many idiots wouldn't RTFM and wouldn't read my guide, they'd make a 3 disk RAIDZ1, then add a disk, then add a disk, then add a disk. Then they'd have a 6 disk array that was 4 vdevs. One of those single-disk vdevs would fail and they'd lose ALL of their data. Then they'd come here and cuss us out because of their own ignorance. Sorry, but its not our job to protect people from being stupid. I don't get paid to be here nor do I get paid to make sure you don't do something stupid. Your own data loss is your own fault.

In any case, the new volume manager helped those that are ignorant while simultaneously punishing those of us that know better. Sorry, but that's not a good way to treat customers.

 

[warri]

Can someone explain you'd want to do this? I've always created RAID sets of identical drives - same make, model, capacity, rotational speed, etc.

I did chose different manufacturers on purpose, since drives from the same batch tend to fail at the same time - and if you're unlucky you get a faulty batch and all of your drives go down in a very short amount of time.

But why create a RAIDZ2 set of drives with different capacities?

As cyberjock said: If you already have some drives and want to grow the pool later one. Actually, I currently have a bunch of 3TB disks in my pool and will upgrade the rest as soon as I'm able to upgrade to more appropriate hardware.

 

[cyberjock]

I did chose different manufacturers on purpose, since drives from the same batch tend to fail at the same time - and if you're unlucky you get a faulty batch and all of your drives go down in a very short amount of time.

Except this can also bite you in the booty. When I was in the Navy we did an RMA on a warrantied drive that failed. Same model and everything, but different firmware. The newer firmware was 16 sectors smaller than the old drive, and we couldn't add it to the RAID.

There's tradeoffs between mixing batches and it requires careful math. If you do 2 batches of 4 disks in a RAIDZ2 you double your chances of having 4 bad disks(which means you doubled your chances of bad firmware/batch/etc from killing your pool). It seems everyone does this and I'll never get why they are so blind to this. They are doing what they don't understand, but whatever. But, if you do 2 disks from 4 different manufacturers(notice I didn't say batch) then any company can make crap disks and you'll still have a working pool. It's about actually knowing what you are applying and not just doing something because "I read online to mix batches for reliability." I don't blindly go and do things because someone told me to. If so, there's a bank down the street I'd like you to make a "withdrawal" from with this 9mm. :D

I never deliberately try to mix batches because proper management usually means your backup server has a different vendor for hard drives.

But to each their own. Just because I wouldn't do it doesn't make it wrong.

 

[zevoman]

When I originally created my FreeNAS box (back in 2009), I had 3 - 1 TB drives lying around and rather than investing hundreds of dollars for 3 identical drives to hold my family's photos and movies, I built the volume with the drives (RaidZ1). As the prices of the drives came down, I bought a 2 TB drive on sale and replaced one of my 1 TB drives. Now I couldn't really take advantage of the larger size since the pool was created with 3-1TB drives, but I knew I had a larger drive ready for the future (now). I bought another 2 TB drive about 9 months ago, waiting to install it, but not feeling really motivated since the pool was doing fine. Being off from work the last couple of weeks, I decided to install the 2nd 2 TB drive, but when I noticed 9.2 was out, I upgraded, destroyed my original pool and am now in the situation described above. I wish I had the hundreds of dollars needed to purchase all identical drives for the optimal pool configuration, but my server needs (photos and videos), or my wallet, don't require it.

Thanks again all for the good info. Always nice to read a nice discussion.

 

[ZFS Noob]

When I originally created my FreeNAS box (back in 2009), I had 3 - 1 TB drives lying around and rather than investing hundreds of dollars for 3 identical drives to hold my family's photos and movies, I built the volume with the drives (RaidZ1).

Please note that there are some significant risks using RAIDZ1 nowadays. You'd do better with RAIDZ2...

 

[zevoman]

Meaning with increasing disk sizes, a second disk and/or read failure on rebuild from a first failure would cause you to lose the volume? Or are there other shortcomings from RaidZ1 (RAID 5) that have come to light?

 

[ZFS Noob]

Pretty much. URE's while resilvering kill the pool.

If we assume one URE in 11.3 TB of reads, and you're planning on reading 4TB of data on a resilver, then you've got ~ 36% chance of losing the whole pool while trying to replace the failed drive. Which is the sort of odds that make me choose a different solution. ;)

That assumes you're using the full capacity of the remaining pools, but it's uncomfortably high. I get panicky when a drive fails.

edit: Actually, I just picked this up in an article on this:

• Advanced file systems like ZFS, which combine file system and volume management functionality, can combine their parity data with parent-block checksums to perform ". . . combinational reconstruction of a RAID set." (Thanks, Joerg!)

So maybe ZFS is cooler than I think, and my n00bishness (see username) is making me spout industry knowledge instead of ZFS knowledge. Or maybe the quote is wrong.

 

[cyberjock]

The quote is accurate. That's one feature that makes ZFS far superior to conventional hardware RAIDs.

 

[oracle_sod]

ok, so the only way to do mix sized disks is in the shell or the gui with 8.3.2..... and if we do it in the shell, then the DB doesn't have the correct info, so where can i find a download for 8.3.2 x64 to do this downgrade and then upgrade ? i have been googling with no luck....

 

[gpsguy]

http://cdn.freenas.org/