Failed to boot NAS after upgrade to 12.0-U4

[freqlabs]

You may be able to work around this in your system firmware settings by disabling boot firmware for the storage pool disk controllers so only the controller for the boot devices is presented to the boot loader. The loader has a limit of 64 open file handles, which can easily get used up if you have a pool with a large number of disks. The exact wording of the setting in UEFI/BIOS may vary, I'm not familiar with your particular system.

 

[Rabinovitch]

So, that's what happens when you try to boot from the installation flash drive (TrueNAS CORE 12.0-U4.1) with the disk shelves connected to the NAS:

OpROM is disabled for all disks:

 

[Rabinovitch]

When we take all the same hardware but with ASUS Pro WS WRX80E-SAGE SE WIFI motherboard, the screen looks a little different, but with the same negative result:

When the JBODs are connected, the system can't boot from an installation flash drive nor from the system disk when TrueNAS is installed.

 

[morganL]

At this stage, I'd recommend you report a bug given that there are no known hardware changes and it happened between 13.0-U3 and U4. Please provide the data/diagnotsics you have.

We have much larger systems that boot happily. I can't see what you have that is different except perhaps multipath.

 

[Rabinovitch]

At this stage, I'd recommend you report a bug

https://jira.ixsystems.com/browse/NAS-114508

 

[Nicholas M]

Your 'gmultipath status' output indicates that your system is setup to access disks over more than one path: FreeNAS marks/stamps the drives as multi path when it first detects them if it can see them on more than one /dev/daX path. Multipath disks are accessed by their multi path device name - for example 'multipath\disk1' which has one or more \dev\daX devices that actually points to the physical drive. If you reboot a FreeNAS/TrueNAS system and you dont have both paths available - for example a missing second cable, gmultipath will still mount these as a multi path disk, but with only a single path. This will show as degraded, as a single path in a multi path system is bad. I have the same boot issue - but i know im missing a cable. Im not entirely sure why broken multipathing is having an issue at the boot stage.

 

[Rabinovitch]

We fixed the issue with "Miltipath connection is not optimal". TrueNAS still can't boot while the disk enclosures are turned on.
I found an old e-mail from Broadcom support, they suggest to "deactivate the BIOS in the Controller Firmware and test again", but I can't deactivate BIOS even with up-to-date FW. It simply says that

Status = Failure
Description = Un-supported command

for ./storcli64 /c0 set bios state=off
and

Status = Failure
Description = Unsupported erase option

for ./storcli64 /c0 erase bootservices