Incremental disk replacement or new pool copy

[Fertil-Gerbil]

I have a 12 disk volume that I want to upgrade from its 3tb disks to 16tb disks.

I can replace them one at a time (I recently replaced a disk and it took 6 days), but this will take an estimated 72 days.
The other option is to setup another shelf with the new disks and copy over the data (~21tb).

Is there a recommended course of action?

 

[Arwen]

If you can replace in place, meaning you have extra disk slot(s), it is possible to replace several at once. Without any loss of redundancy. You mention an extra disk shelf, that can help by allowing you to fully install all the replacement disks at once. Then perform the rest of the work remotely until you de-activate the old shelf & it's disks.

Next, your FreeNAS system lists RAID-Z2, that means you could in theory replace 2 at once. Though, without an extra disk slot, you do so without any redundancy.

Further, their are tuables that allow you to both speed up the re-silver, and to allow more than 1 replacement at a time. I don't have them handy, nor how to modify them. But they exist. Of course, speeding up the re-silvers will impact users, if any.

Last, setting up a new shelf is one way to perform the work. But, this only works if the data is not under constant change. If you start a copy today and it takes a week, some or all of the changes during that time could be lost. Or have to be copied after.

 

[sretalla]

Obviously the parallel pool and copy will be much faster.

Also, gives you the chance to consider moving to 2 VDEVs of 6 rather than 1 of 12... potentially better performance considering you would now have such large disks.