Can I Grow a Pool by Shuffling Mirrors' Drives?

[kentukynitemare]

I'm putting together a new FreeNAS box and trying to make sure I've got this straight. Critical data is backed up on local external drives and offsite, so this is mostly an exercise to learn how to configure ZFS properly.

I have two 4TB WD Reds from an old NAS build sitting around, and intend to purchase three 10TB external drives, two to shuck for the new FreeNAS build and one to plug into my Windows desktop as FreeNAS' backup target/BackBlaze'd drive. I don't have 100% confidence in this whole drive-shucking plan (never done it before, although folks seem to have had pretty good results) so I want to pair each of the two shucked drives in the new build with a WD Red (because blind faith in my low-use reds.) My understanding is this would net me two 4TB mirrored VDEVs (one each 4 and 10 TB drives), striped together to form an 8TB pool that could withstand the loss of both shucked drives without any data loss.

Assuming all goes well with the two 10TB drives in the NAS for some time I'd like to be able to utilize their storage capacity more fully. To do this, my plan would be to (for 4+10TB mirror VDEVs A and B, spare 10TB C):

Remove 4A, replace with 10C and resilver, creating a 2x10TB mirrored pair
Remove 10B, replace with the pulled 4A and resilver, creating a 2x4TB mirrored pair

Which should leave me with a 2-way 10TB mirror (10A and 10C) striped with a 2-way 4TB mirror (4B and 4A) for a total of a 14TB pool without having to destroy/rebuild the pool, as well as a loose 10TB disk (10B) to use as a Windows-attached backup target, third disk in the 10TB VDEV, or pair with a sixth disk for a third mirror.

Is my assessment accurate? I think I've finally got my head wrapped around the basic ZFS concepts, which usually means I'm missing something.

 

[Ericloewe]

Yes, with the obvious risks involved.

That said, you're probably better off testing the new disks for a while and then using them together in a vdev instead of having them mixed with the old ones. Saves you a couple of labor-intensive steps that would be a pain to get wrong.

 

[kentukynitemare]

That said, you're probably better off testing the new disks for a while and then using them together in a vdev instead of having them mixed with the old ones.

You mean something like run two pools and when I'm satisfied the shucked drives are stable destroy their pool, create a new vdev with them and stripe it into the 4TB pool?

Also, while I've got some pointers from this post I'm not sure exactly how to define "these drives appear that they won't die out of the blue." Are there any particular metrics I should keep an eye on during testing, or am I just wringing my hands over nothing and should do my burn-in then keep an eye on the SMART results periodically like I would any other drive to be more aware of impending failures?

 

[garm]

Just running a pool won’t really show you much of the disks reliability. Do a proper burn in and build the pool with matching vdevs

 

[Ericloewe]

am I just wringing my hands over nothing and should do my burn-in then keep an eye on the SMART results periodically like I would any other drive to be more aware of impending failures?

Yup. Unless you rough them up, they're going to be just like any other disk. Some are good, some are not, some are Seagate 3TB or IBM DeathStars - veritable bitbuckets.