Mixing New Drives into old vdevs as pool grows?

[Mr_N]

Has anyone thought about taking new drives and incorporating them into the pool in a way that spreads them across multiple vdev's as you grow your pool?

So for example you start with a 6 drive Z2 pool and then a couple of years later you prepare to add a 2nd vdev of 6 drives in Z2 to your pool, so you buy another 6 drives and instead of just making the new vdev with them, use the replace option to one at a time swap out 2 of the drives in the original vdev with 2 new drives and then make the new vdev with 4 new drives and 2 older drives.

Then when adding the 3rd vdev a couple of yrs later do the same thing remove another 2 of the original 6 drives replacing them with new ones and making the 3rd vdev with 4 new drives and 2 old?

1st vdev WWXXYY 2nd vdev WWXXZZ 3rd vdev WWYYZZ 4th vdev XXYYZZ
W drives 6yrs old / X drives 4yrs old / Y drives 2yrs old / Z drives New etc

Would this make your pool any more resilient by making the vdevs more resilient or would it be more effort than its worth?

 

[jgreco]

Yes, it's done all the time in larger environments. It's also good for training operators on how to handle disk failures.

 

[Mirfster]

In my opinion, I am not sure that would really make things more resilient. I understand that the 6 original drives will have a lot more "miles" on them, but to me that would seem like a lot of work for something that may be counter productive. Almost like taking a possibility of failing hard drives from a single vdev and then compounding that to other vdevs.

I guess, I would suggest just using the new drives for the new vdevs. Have extra drives on hand or as spares which could be used if there were a problem a drive in the original vdev (or any for that matter) and backups of your configuration.

Maybe others might think differently... guess seeing a smart report and knowing the expected life span of the drives wouldn't hurt. But if they are near end of life anyways, they should be replaced and not re-propagated anyways.

Edit: Leaving my response up, even though I am fully expecting a smack in the back of the head from jgreco...

 

[jgreco]

Leaving my response up, even though I am fully expecting a smack in the back of the head from jgreco...

Why? It's strictly a matter of how much effort you see each option as, versus perceived benefit. Boils down to opinion plus a recognition that you can protect yourself from statistics in other ways.

If you're on top of things (SMART checks, emails for failures, etc) and have a spare drive handy, then it's in the realm of fairly- to very-unlikely to make a difference, as long as you're running something like RAIDZ2 or better. Statistically, drives are less likely to die in rapid succession once they're past a certain point, so you're reasonably likely to have a much better chance of not being in the bad situation where a drive fails, you replace one, and another fails halfway through the resilver, and then you end up with URE's on one of the other drives. Could it happen? Sure! Has it happened? Absolutely! But it's rare.

In a not-the-same-but-similar vein, many people like to get all the same kind of drives for their arrays ("heterogeneous array"). A lot of the people who promote this are storage admins who are used to buying vendor supplied arrays, where vendors have ulterior motives (including single-sourced discounts, matched performance characteristics, and in the old days, spindle sync) for selling them that. The problem is that if you're unlucky and you get a bad batch of drives, you can start to get failures that pile up and overwhelm the available redundancy. So some of us like heterogeneous pools, with drives from different vendors, to reduce the likelihood of that. On the other hand, it opens us up to some additional risk in the form that we now have two different models and either one of them could have had a bad manufacturing run. So the likelihood that there's SOME (noncatastrophic) disk problem that needs to be dealt with increases, while the likelihood that a manufacturing batch defect nukes our pool is dramatically reduced.

What we're talking about here works out to a similar problem. You may be increasing overall reliability by adding some new disks to old vdevs, reducing the chance of a catastrophic vdev failure. The difference between that and the heterogeneous pool issue is that in this case, you have no choice as to whether to go hetero or homogeneous, so sooner or later you're likely to hit a failure especially of an older disk. So I think it's how you're prepared to cope with that which is a major component here.

Also, while we could argue that this is certainly an option with hard drives, in an SSD array, it is virtually mandatory. A SSD vdev of half a dozen SSD's would be wearing out at approximately the same rate; at a certain point, there's a huge amount of danger that you could lose several disks in a short period of time.

 

[Mr_N]

when i got to the end of writing my post it did seem like alot of work :)

maybe it might make more sense to grow the pool by replacing the 1st vdev with bigger drives once you have 4-5 vdevs in the pool, rather than just adding more vdevs, this way your removing the oldest drives from the pool before they get so old that errors and failures are more likely.

 

[Mirfster]

Well that was not so painful. :p

I see your points, especially on SSDs. When I hit that Powerball jackpot tonight, I will then be able to go that route.

 

[Robert Trevellyan]

you buy another 6 drives and instead of just making the new vdev with them, use the replace option to one at a time swap out 2 of the drives in the original vdev with 2 new drives and then make the new vdev with 4 new drives and 2 older drives

There's a pretty good chance that when you buy your 2nd batch of 6 drives, they will be larger than your original 6, which would make this strategy much less attractive.

 

[Mr_N]

They wont be :) Just a little cheaper, although not as cheap as they once would have been.

 

[Mirfster]

They wont be :) Just a little cheaper, although not as cheap as they once would have been.

Eh? Your riddle confuses my limited brain capacity....

 

[Mr_N]

Prior to the flooding of 2011 when drive supply was interrupted and prices skyrocketed, their downward price trend has never restarted and they continue to be annoyingly expensive as only 2-3 manufacturers have all the market...

 

[jgreco]

Prior to the flooding of 2011 when drive supply was interrupted and prices skyrocketed, their downward price trend has never restarted and they continue to be annoyingly expensive as only 2-3 manufacturers have all the market...

Surely you're not implying that the hard drive cartel caused the flooding in 2011 in order to introduce an artificial price floor on the global hard drive supply.

 

[Mr_N]

well i wasnt but now that you mention it...

http://www.kitguru.net/wp-content/uploads/2015/05/hdd_shipments_q1_2015_asps.png

 

[Ericloewe]

Surely you're not implying that the hard drive cartel caused the flooding in 2011 in order to introduce an artificial price floor on the global hard drive supply. :)

No, that's a typical supervillain plot. Doesn't work because it's too obvious and the people in charge can't help but brag (who wouldn't? It's a pretty big achievement).

No, they just took advantage of the floods to say "Oh yeah, we're totally supply-constricted and stuff, we'll have to jack up prices", even though nobody was actually supply-constricted for more than a few months.