Best way to configure six drives?

[Z300M]

What would be the best way to configure six drives?

(a) 3 + 3 mirror?

(b) RAIDZ3?

(c) RAIDZ2 plus 1 spare?

Pros and cons of each configuration?

 

[depasseg]

Depends on your performance needs, future scalability and tolerance for failure.

a - I assume you mean striped mirrors.
best performance
easy to expand (just add 2 more drives)
if you lose 2 drive in the same mirro, pool is toast

b - for the super paranoid, who don't want to have to worry about a physical failure

c - for the super paranoid, who don't want to worry about a physical failure, but don't realize that having a hot spare sitting there is just as likely to fail as if it were part of a RAID-Z3 (which is where it would be better utilized.

Depending on how much usable space you want, the best utilization/performance tradeoff, if it were my pool, would be a 6 Drive RAID Z2, with a cold spare sitting on a shelf. If you didn't have close access to the machine (and maybe couldn't quickly replace a failed drive), then RAID-Z3.

 

[Z300M]

Depends on your performance needs, future scalability and tolerance for failure.

a - I assume you mean striped mirrors.
best performance
easy to expand (just add 2 more drives)
if you lose 2 drive in the same mirro, pool is toast.

No, in fact I was thinking of mirrored stripes rather than striped mirrors.

b - for the super paranoid, who don't want to have to worry about a physical failure

c - for the super paranoid, who don't want to worry about a physical failure, but don't realize that having a hot spare sitting there is just as likely to fail as if it were part of a RAID-Z3 (which is where it would be better utilized.

I thought that FreeNAS still didn't handle hot spares. I was thinking of a "well burned-in" cold spare.

Depending on how much usable space you want, the best utilization/performance tradeoff, if it were my pool, would be a 6 Drive RAID Z2, with a cold spare sitting on a shelf. If you didn't have close access to the machine (and maybe couldn't quickly replace a failed drive), then RAID-Z3.

I assume you are thinking of the n^2 + y "rule" (where n is the number of "data disks" and y is the number of "parity disks" -- OK, I know that's not how RaidZ works, but I can't think of a better way to put it); but I've read on these forums that this "rule" no longer applies -- for reasons that I no longer recall.

And doesn't the first of your preferred configurations (6-drive RAIDZ2 + cold spare) have the same disadvantage as my (b) suggestion? Wouldn't that spare 7th drive be better employed as part of a 7-drive RAIDZ3 pool? -- and not just "If you didn't have close access to the machine (and maybe couldn't quickly replace a failed drive)"?

 

[Bidule0hm]

I assume you are thinking of the n^2 + y "rule" (where n is the number of "data disks" and y is the number of "parity disks" -- OK, I know that's not how RaidZ works, but I can't think of a better way to put it); but I've read on these forums that this "rule" no longer applies -- for reasons that I no longer recall.

It's due to compression being the default now. The rule doesn't apply for performance but seems to still apply for overhead reduction. However I'm not 100% sure because I've not talked (yet) to some devs. And even if it's true it's a very small overhead; in short, don't worry and put the number of drives you want ;)

And doesn't the first of your preferred configurations (6-drive RAIDZ2 + cold spare) have the same disadvantage as my (b) suggestion? Wouldn't that spare 7th drive be better employed as part of a 7-drive RAIDZ3 pool? -- and not just "If you didn't have close access to the machine (and maybe couldn't quickly replace a failed drive)"?

No because it's a cold spare, so the drive isn't wearing out doing nothing like with a hot spare.

NB: a RAID-Z3 with 6 drives is very paranoid (and mirrored 3 drives stripes are the opposite...), far more than what I am, why you want this very high redundancy?

 

[depasseg]

I didn't know you could configure mirrored stripes in the GUI. cool!

Hot spare functionality was recently added.

The difference between 6 Drive RAID Z2 and 6 Drive RAID-Z3, is the loss of one more drives worth of space. If space isnt important, then they are at the same disadvantage level.

Yes it would be better employed as a 7th disk in a RAID-Z3, but I was under the impression your system could only handle 6 drives.

 

[Z300M]

I didn't know you could configure mirrored stripes in the GUI. cool!

Maybe I assumed wrongly that it is possible.

Hot spare functionality was recently added.

That's good, but whether I'll use it is another story.

The difference between 6 Drive RAID Z2 and 6 Drive RAID-Z3, is the loss of one more drives worth of space. If space isnt important, then they are at the same disadvantage level.

Yes it would be better employed as a 7th disk in a RAID-Z3, but I was under the impression your system could only handle 6 drives.:)

No: I specified 6 drives because that's the number of 6TB Seagates I picked up today at BestBuy for $61.99 apiece.

 

[depasseg]

I just got 6 of those as well. I'm going to run them in RAID Z1. Because they are my backup's backup. My choices and reasons are going to be different than yours.

So what are your plans for this pool? Do you care more about performance or safety?

 

[Z300M]

It's due to compression being the default now. The rule doesn't apply for performance but seems to still apply for overhead reduction. However I'm not 100% sure because I've not talked (yet) to some devs. And even if it's true it's a very small overhead; in short, don't worry and put the number of drives you want ;)



No because it's a cold spare, so the drive isn't wearing out doing nothing like with a hot spare.

But a hot spare can still be having regular SMART tests. A cold spare might be found to be useless when it's powered up.

NB: a RAID-Z3 with 6 drives is very paranoid (and mirrored 3 drives stripes are the opposite...), far more than what I am, why you want this very high redundancy?

 

[Ericloewe]

I didn't know you could configure mirrored stripes in the GUI. cool!

That's because you can't. It can't be done with ZFS.

 

[Bidule0hm]

A cold spare might be found to be useless when it's powered up.

That's why you need to burn it in before putting it on the shelf as a spare ;)

 

[Z300M]

That's why you need to burn it in before putting it on the shelf as a spare ;)

And I was about to ask: someone -- perhaps you -- posted a script that executes an extensive test of one or more drives, but I can't find it. Was it yours, or do you know where I could find it? I don't think it just runs SMART conveyance and long tests.

 

[Bidule0hm]

It was jgreco :)

I think it's in the burn in hardware thread, do a search and use jgreco in the member field, you should find it easily that way ;)

 

[Z300M]

It was jgreco :)

I think it's in the burn in hardware thread, do a search and use jgreco in the member field, you should find it easily that way ;)

Found it. Thanks.

 

[david kennedy]

That's because you can't. It can't be done with ZFS.

Oracles documentation on ZFS disagrees with this:

Storage Configuration
We recommend that a mirrored (RAID10) configuration be used for Oracle VDI deployments to meet the demanding random I/O patterns produced by a large number of virtual desktops accessing the same physical disk spindles. Mirrored disk pools are able to provide the needed IOPs at low latency, which prevents desktop users from experiencing desktop lag or slowness. Parity disk pools (either single- or double-parity RAID) may provide acceptable performance in low usage or demonstration configurations, but care should be taken to perform a detailed pre-deployment test validation to confirm that such a pool layout is adequate to meet user needs before the system is put into production.

RAID10 - a stripe of mirrors

 

[Z300M]

Oracles documentation on ZFS disagrees with this:

Storage Configuration
We recommend that a mirrored (RAID10) configuration be used for Oracle VDI deployments to meet the demanding random I/O patterns produced by a large number of virtual desktops accessing the same physical disk spindles. Mirrored disk pools are able to provide the needed IOPs at low latency, which prevents desktop users from experiencing desktop lag or slowness. Parity disk pools (either single- or double-parity RAID) may provide acceptable performance in low usage or demonstration configurations, but care should be taken to perform a detailed pre-deployment test validation to confirm that such a pool layout is adequate to meet user needs before the system is put into production.

RAID10 - a stripe of mirrors

I was talking about a mirror of stripes rather than a stripe of mirrors, and it's the former that Ericloewe says can't be done.

 

[Ericloewe]

Oracles documentation on ZFS disagrees with this:

Storage Configuration
We recommend that a mirrored (RAID10) configuration be used for Oracle VDI deployments to meet the demanding random I/O patterns produced by a large number of virtual desktops accessing the same physical disk spindles. Mirrored disk pools are able to provide the needed IOPs at low latency, which prevents desktop users from experiencing desktop lag or slowness. Parity disk pools (either single- or double-parity RAID) may provide acceptable performance in low usage or demonstration configurations, but care should be taken to perform a detailed pre-deployment test validation to confirm that such a pool layout is adequate to meet user needs before the system is put into production.

RAID10 - a stripe of mirrors

Of course you can stripe several mirrors. Each set of mirrors is a vdev.
All vdevs in a pool are always striped (not necessarily balanced, though), so it's impossible to mirror two vdevs.