SOLVED Can I safely move disks to a new controller without resilvering?

[msgyumyum]

Hello all, long time lurker and first time poster.

I have been using a cheap Highpoint 640L SATA expansion card which regularly chokes out a few checksum errors for its drives (never had any errors for disks connected to the motherboard SATA ports). Recently during a scrub all four of the drives on the Highpoint timed out and were kicked offline to put the pool in an online but critical state until I rebooted the server. Fortunately no data was lost since I had arranged the disk pool into vdevs such that the disk pool (just barely) stays online even if the highpoint card fails completely. This event obviously worried me and after doing some reading on these forums, I decided to buy a better HBA (LSI 9207-8i) to replace the Highpoint.

Current situation:

Desired situation:

My question is what the best way to move the disks to the new controller? When I install the new HBA, if I shut down the server and move all the drives at once from the old HBA to the new one, will FreeNAS at startup happily detect the drives and my disk pool will be healthy as if nothing had changed? Or will the UIDs for the drives change and my disk pool will be degraded? Will I be able to import the drives (without having to wait hours and hours for it to resilver)?

Or will I need to move the drives one by one, and wait for the resilver to complete as shown below:

I also have a cold spare drive available (never used), maybe I should rotate it into the array:

I do have a recent backup of my FreeNAS config and my important data on a 16TB external USB drive, but obviously I would prefer to avoid doing a full restore, if possible. My disk pool has 21TB of data used, with 7TB free (71% used).

Here just in case are the full specs for this FreeNAS-11.3-U5 server:
CPU: Intel Xeon E3-1230v5 (4C/8T)
RAM: 16GB DDR4-2133 (ECC)
Motherboard: Supermicro X11SAE-F (Intel C236, 8 SATA ports)
HBA: Highpoint 640L (additional 4 SATA ports)
Boot: 2 x 16GB SanDisk USB3 (never had a failure yet, but I will move these to an M.2 SSD eventually)
Data: 12 x 4TB HGST (HDN724040ALE640)
PSU: Corsair AX860 Platinum
Case: Coolermaster Stacker STC-T01

Thanks in advance for your advice!

 

[Samuel Tai]

ZFS tracks disks by their GPTID, not by their controller location. You can move all the disks wholesale to the new controller, and your pool will be fine.

 

[Etorix]

You could reshuffle the drives any way, ZFS would still identify them and mount the pool. No resilver is involved, but a full scrub after the transfer would not hurt. If there is a free slot in the case, take this opportunity to plug your cold spare as hot spare.

 

[msgyumyum]

ZFS tracks disks by their GPTID, not by their controller location. You can move all the disks wholesale to the new controller, and your pool will be fine.

Thanks for this excellent advice! Indeed, after moving all the disks to the new controller and starting up the server, without any additional effort on my part the pool was healthy and appears to be working perfectly. There was only a small surprise in that only the moved disks now have different device labels (ada#p2 changed to: da#p2) but this was handled transparently by FreeNAS. While the server was offline I mounted a 13th drive and then after starting it up and seeing that everything was fine, I extended the pool by adding the 13th drive as a hot spare to the pool.

 

[msgyumyum]

You could reshuffle the drives any way, ZFS would still identify them and mount the pool. No resilver is involved, but a full scrub after the transfer would not hurt. If there is a free slot in the case, take this opportunity to plug your cold spare as hot spare.

Thanks for this excellent advice! I ran a scrub and added a hot spare to the pool. I agree it makes a lot more sense in the event of a disk failure to have the spare automatically replace the failed disk and let FreeNAS start rebuilding the failed disk immediately rather than waiting for me to read the notification and then physically get to the server, open up the case and plug in the spare drive.

 

[ChrisRJ]

There was only a small surprise in that only the moved disks now have different device labels (ada#p2 changed to: da#p2) but this was handled transparently by FreeNAS.

That is to be expected from a SAS connection, I have the same with the disks that are on the SAS ports of my motherboard.