Bluefin -> Cobia update stuck at 80% ("Preparing initramfs configuration")

[egg82]

As the title says, I triggered the update via GUI and it's been sitting at 80% (Preparing initramfs configuration) for about an hour with no signs of high CPU or memory usage.

I've never (and don't plan to) make any configuration changes via CLI. Only GUI. I sometimes log in to check files and mount external NTFS drives for copying, but not for system changes.

Here's the current system specs:
Version: TrueNAS-SCALE-22.12.4.2 (Bluefin)

Code:

root@nas[~]# zpool list
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
boot-pool   912G  10.5G   902G        -         -     0%     1%  1.00x    ONLINE  -
logs       58.2T  4.78T  53.4T        -         -     0%     8%  1.00x    ONLINE  /mnt
misc        133T   104T  28.5T        -         -     0%    78%  1.00x    ONLINE  /mnt

Code:

root@nas[~]# zpool status
  pool: boot-pool
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:00:06 with 0 errors on Mon Jan 15 03:45:07 2024
config:

        NAME           STATE     READ WRITE CKSUM
        boot-pool      ONLINE       0     0     0
          mirror-0     ONLINE       0     0     0
            nvme0n1p3  ONLINE       0     0     0
            nvme1n1p3  ONLINE       0     0     0

errors: No known data errors

  pool: logs
 state: ONLINE
  scan: scrub repaired 0B in 02:45:51 with 0 errors on Sun Jan  7 02:45:53 2024
config:

        NAME                                      STATE     READ WRITE CKSUM
        logs                                      ONLINE       0     0     0
          raidz1-0                                ONLINE       0     0     0
            7de9fe61-cf5d-45cc-9162-26cbcc9e9bbe  ONLINE       0     0     0
            c6919483-05d5-457c-97c7-69754d6a6cdc  ONLINE       0     0     0
            d08b8911-97db-4460-bd98-29a9d460f038  ONLINE       0     0     0
            3014ad04-0cdd-47fb-a94e-d37908322925  ONLINE       0     0     0
        logs
          mirror-1                                ONLINE       0     0     0
            335e0572-c38c-4108-a4bf-2ed3253e8fe9  ONLINE       0     0     0
            9df6cf5c-3b3f-4e12-a414-6a82a358f319  ONLINE       0     0     0
        cache
          ed1bfc90-0615-45b1-8c05-db023b852346    ONLINE       0     0     0
          92e57d93-8d92-4ebc-85e3-ecbd202873dc    ONLINE       0     0     0

errors: No known data errors

  pool: misc
 state: ONLINE
  scan: resilvered 21.5M in 00:05:07 with 0 errors on Mon Dec 25 20:14:22 2023
config:

        NAME                                      STATE     READ WRITE CKSUM
        misc                                      ONLINE       0     0     0
          raidz2-0                                ONLINE       0     0     0
            88bddaf1-a4b9-4219-8c54-2a03cf3814ec  ONLINE       0     0     0
            e9a4a260-35d6-4a62-8e9e-9a00d8a3a82c  ONLINE       0     0     0
            c979f043-251d-498c-9b5f-fdaa84004ad5  ONLINE       0     0     0
            365d8143-c86a-4035-ac20-02268169a548  ONLINE       0     0     0
            e543b22d-cf00-411a-9df1-24c998bbc2ef  ONLINE       0     0     0
            1c092aaa-a490-4f08-84d3-dd99ad4d309d  ONLINE       0     0     0
            a81e9488-dca0-428a-9ce7-f1128d74e55c  ONLINE       0     0     0
            ffc86464-cd9e-46d4-8235-dbfecf8ea580  ONLINE       0     0     0
        special
          mirror-2                                ONLINE       0     0     0
            5161a2df-7a01-4b31-9b3a-85dab8336ab2  ONLINE       0     0     0
            d9cae232-fee2-41db-bd89-d1bf9127666a  ONLINE       0     0     0
            38acaf15-721d-4d34-8493-eb45ec43087d  ONLINE       0     0     0
        cache
          6a116758-9a42-41f3-bd57-ee268be59d22    ONLINE       0     0     0

errors: No known data errors

SuperMicro server purchased from theserverstore:

• Supermicro SuperStorage 6048R-E1CR36N W/ X10DRi-T4+ (flashed with custom BIOS for NVME boot)
• 2x Xeon E5-2667 v4 3.2GHz 8-Core Processors
• 128GB (4x 32GB) DDR4 Registered Memory
• 12Gbps IT Mode PCIe Storage Controller (Flashed to IT Mode; No RAID Pass-Thru Only)
• Intel X520-DA2 Dual Port 10GbE SFP+ PCIe Adapter (configured in LACP, MTU 9000)
• 2x 1280Watt Power Supply PWS-1K28P-SQ

Drives:

• 2x Western Digital 1TB WD Red SN700 NVMe (boot drives)
• 8x WD Ultrastar DC HC550 18TB (misc pool)
• 4x Seagate Exos X16 16TB ST16000NM002G (logs pool)
• 8x Western Digital 2TB WD Red SA500 SATA (special/cache/log devices)

I'll check it again in the morning, but I'm not hopeful. Any ideas? Should I reboot the machine and try again? Try from the CLI instead?
I'm not sure how to cancel the update operation if it is, in fact, stuck. I'm trying to not brick the server, even though I took a config backup and have IPMI.

 

[egg82]

Update: the system seems to have updated successfully to TrueNAS-SCALE-23.10.1.1 and rebooted at around 12:30 this morning, which leaves it at having taken about 3-4 hours to install the update. I suppose I didn't expect it to be instant, but that feels like a timeout or seven. Not sure what might have happened, but it's definitely no longer an issue and solved.