Replacing a 10TB disk with a 12TB disk without offlining the 10TB disk?

[chackett]

We are methodically replacing 10TB drives, currently there are 2 drives in the mirror and 2 mirrors in the volume. Is it possible to replace the 10TB drive with the 12TB drive while leaving the 10TB online for redundancy (the 12TB is already in the server but un-used), will the 10TB stay active until resilvering is complete?

FreeBSD 11.1-STABLE

 

[Ericloewe]

Yes, just replace the disk without first offlining the old one. You can also do this to as many disks in parallel as you can connect to your system.

 

[choward]

So if you have a mirror between two disks, it will basically write to the third disk also and not bypass the one you're replacing? Good to know.

 

[Ericloewe]

I don't know whether it will write to the disk being replaced. It's ideally a bit of an academic discussion, since the goal is to have the new disk hang around "for a while" without losing data. This is more typically framed as "I want to maintain my existing redundancy while replacing disks that are still mostly functional".

 

[Patrick M. Hausen]

If you have the means to connect one 12 G disk with all 10 G disks simultaneously you can

- copy the partition table of a 10 G disk to the new 12 G disk
- attach the 12 G disk to the existing mirror

This will copy all data without losing redundancy.

- when finished detach a 10 G disk from the now three-way mirror
- repeat with second 12 G disk (copy partition table, attach, detach 10 G disk)

You will now have a mirror with two 12 G disks using only a 10 G partition. This is not a problem.

- use gpart resize to extend the ZFS partitions to the end of the 12 G disks
- use zpool online -e to force ZFS to recognise and use the new space

Easy peasy and without risk as far as redundancy is concerned.

Important: you must copy the partition table and refer to the new disks by GPTID of the ZFS partition. Never attach raw disk devices like /dev/ada2 or some such to your pool.

HTH. If you can attach old and new disks simultaneously feel free to come back with questions about the details. A zpool status of the existing topology would be helpful in this case.

 

[Ericloewe]

Why would you need to do that manually? Is there a GUI bug I'm not aware of?

 

[Patrick M. Hausen]

Why would you need to do that manually? Is there a GUI bug I'm not aware of?

I have no idea. I do not use the UI for pool manipulation.

 

[choward]

Yeah, this is a pre-emptive replacement--the drive is not failing. We're trying to pull out the older drives before they do become a problem. Nothing tightens the old sphincter like a mirror with a failed disk we're replacing sitting side-by-side with a disk of equal age that we're doing a massive read operation on to re-establish the mirror.

The statement that disk a and disk b (in a healthy mirror) and we're replacing disk b with disk c, if something did happen during that replacement with disk c, disk b would still be a member of the mirror. Is that accurate? I don't like the idea that the process may offline disk b in order to replace with disk c and something happens to either disk a or disk c during the replacement without being able to fallback on disk b. Oh the paranoia... Now I'm probably overthinking things here.

 

[Ericloewe]

Just don't offline and the disk and the replacement will be in-place. The old disk will be detached from the pool once the resilver has concluded.

 

[Davvo]

Use of the WebUI is strongly suggested when possibile.

 

[Jailer]

Use of the WebUI is strongly suggested when possibile.

Agreed. The excellent instructions given by @Patrick M. Hausen should only be utilized by an advanced user or in the event that a replacement cant be conducted using the UI.

 

[jgreco]

I don't know whether it will write to the disk being replaced.

If the pool is active for write, updates may still be written to the disk being replaced. The disk is assumed to be a pool member until it is detached. ZFS does not assume that a replacement operation will be successful.