SOLVED Unable to replace disk in boot-pool mirror - disk to small

[fayelund]

Afte a failure on one of my disks in the boot-pool mirror, I am unable to replace the failed disk.
Im trying to replace the disk with an identical SATADOM as the failed disk(and identical to the remaining disk), but getting an error saying the device is too small.

When I look into the partitioning done it seems like Truenas creates three partitions on the new device, and the old one only havbe two partitions. This seems to make the partition needed too small.

Functional device on the mirror layout:
Disk /dev/sdi: 14.91 GiB, 16013942784 bytes, 31277232 sectors
Disk model: InnoLite SATADOM
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 849F7698-2F7B-11ED-AADC-3CECEF9098E2

Device Start End Sectors Size Type
/dev/sdi1 40 532519 532480 260M EFI System
/dev/sdi2 532520 31268903 30736384 14.7G FreeBSD ZFS

New disk layout:
Disk /dev/sdj: 14.91 GiB, 16013942784 bytes, 31277232 sectors
Disk model: InnoLite SATADOM
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 94F45FFD-588B-4A1F-BEFA-2F5D8D02B1BB

Device Start End Sectors Size Type
/dev/sdj1 40 2087 2048 1M BIOS boot
/dev/sdj2 2088 1050663 1048576 512M EFI System
/dev/sdj3 1050664 31277198 30226535 14.4G Solaris /usr & Apple ZFS

Does anyone know of any magic I can do to rebuild my mirror without doing a reinstall?

 

[fayelund]

It might seem like going cli withj the following command works:
zpool replace boot-pool $old_disk $new_disk

 

[PhilD13]

Could you go in the GUI to

System Settings | Boot | Click the button "Boot Pool Status"... then you have the 3 dots on the right of each element (mirror and disks) and select replace on the failed drive? Pop out the bad one and put in the good one?

Or is that where you are already working from?
This is info that would be handy to have as I'm sure one day I'll have to do the same.

 

[fayelund]

Could you go in the GUI to

System Settings | Boot | Click the button "Boot Pool Status"... then you have the 3 dots on the right of each element (mirror and disks) and select replace on the failed drive? Pop out the bad one and put in the good one?

Or is that where you are already working from?
This is info that would be handy to have as I'm sure one day I'll have to do the same.

This was what I did.
Seems like cli got the disk into the pool, but it's missing boot, som I'm afraid it will not work. So a reinstall might be the only way forward.

 

[sfatula]

Could you go in the GUI to

System Settings | Boot | Click the button "Boot Pool Status"... then you have the 3 dots on the right of each element (mirror and disks) and select replace on the failed drive? Pop out the bad one and put in the good one?

Or is that where you are already working from?
This is info that would be handy to have as I'm sure one day I'll have to do the same.

I've done exactly that and it worked fine (UI) about 4 months ago.

 

[fayelund]

I've done exactly that and it worked fine (UI) about 4 months ago.

That it worked for you does not help me. I did the exact same thing and had an error.

 

[HoneyBadger]

Hey @fayelund

What version of TrueNAS are you running, and did you upgrade from a previous version? I'm assuming you're on a newer release and have upgraded from one in the past (possibly a CORE -> SCALE migration?) and it looks like your existing boot-pool device is using an older partitioning scheme which might be the cause of the trouble here, as it's creating a smaller EFI volume than the new BIOS/EFI hybrid setup.

Can you grab a debug from System -> Advanced -> Save Debug and then submit a ticket using the "Report a Bug" link at the top of the Forums?

In the meantime, you may be able to set up something manually using gpart/fdisk (depending on your current version) but let's figure out the lay of the land right now, and grab that debug file.

 

[fayelund]

Hey @fayelund

What version of TrueNAS are you running, and did you upgrade from a previous version? I'm assuming you're on a newer release and have upgraded from one in the past (possibly a CORE -> SCALE migration?) and it looks like your existing boot-pool device is using an older partitioning scheme which might be the cause of the trouble here, as it's creating a smaller EFI volume than the new BIOS/EFI hybrid setup.

Can you grab a debug from System -> Advanced -> Save Debug and then submit a ticket using the "Report a Bug" link at the top of the Forums?

In the meantime, you may be able to set up something manually using gpart/fdisk (depending on your current version) but let's figure out the lay of the land right now, and grab that debug file.

Ticket created: NAS-124642
I initially tried this on 22.12.3.3, got the system upgraded to 22.12.4.2, but same problem there.

 

[dasunsrule32]

I'm running into the same issue. I've built out a new server and am trying to migrate to a SSD I have and am getting all kinds of issues with this as well. I migrated from CORE > SCALE over a year ago.

I will say, I am working on installing to the new SSD and will restore my config into the new install because I can't wait.

 

[PhilD13]

From what I understand, migrating from an existing core to scale does not add the third & 4th partition that is created in scale on a fresh install.
So the system, post migrate to scale, will not be able to recover a failed boot mirror because it is attempting to create a partition scheme (adds an EFI partition and swap and resizes everything) on the new drive that was not used previously in core creating a mismatch in size.

I also understand that this has been reported and there is no current easy solution other than to wipe the boot drives, reinstall Scale from scratch then import the configuration.

I agree with some others about this that there should be a big warning on the migration, that a drive failure in the boot mirror after migration will cause a failure to recover the boot mirror

You can look at the drives and see what partitions are on them and if they match or are missing.
sudo fdisk -l /dev/sda [change path to disk to suit your system, my boot mirror are these two drives /sda and /sdb ]

 

[sfatula]

From what I understand, migrating from an existing core to scale does not add the third & 4th partition that is created in scale on a fresh install.
So the system, post migrate to scale, will not be able to recover a failed boot mirror because it is attempting to create a partition scheme (adds an EFI partition and swap and resizes everything) on the new drive that was not used previously in core creating a mismatch in size.

I also understand that this has been reported and there is no current easy solution other than to wipe the boot drives, reinstall Scale from scratch then import the configuration.

I would slightly change this, it may be obvious, but, export configuration first in Scale (System Settings -> General -> Manage Configuration -> Download File (be sure to check Export Password Secret Seed), then reinstall and import the just saved configuration. That should indeed get it back to "normal" Scale format.

 

[PhilD13]

Yes, you are correct. That would be much better for instructions. Thanks

 

[fayelund]

From what I understand, migrating from an existing core to scale does not add the third & 4th partition that is created in scale on a fresh install.
So the system, post migrate to scale, will not be able to recover a failed boot mirror because it is attempting to create a partition scheme (adds an EFI partition and swap and resizes everything) on the new drive that was not used previously in core creating a mismatch in size.

I also understand that this has been reported and there is no current easy solution other than to wipe the boot drives, reinstall Scale from scratch then import the configuration.

I agree with some others about this that there should be a big warning on the migration, that a drive failure in the boot mirror after migration will cause a failure to recover the boot mirror

You can look at the drives and see what partitions are on them and if they match or are missing.
sudo fdisk -l /dev/sda [change path to disk to suit your system, my boot mirror are these two drives /sda and /sdb ]

Well, thats my explanation as well.
Luckily im facing a migration to some new HW the next weeks, so I will fix it then. But yes, this should be clearly stated when migrating from core to scale :)