I messed up my boot pool

taylornate

Dabbler
Joined
Jul 10, 2016
Messages
43
This is a previously working system running TrueNAS core (v13 I think) on bare metal that is no longer booting. I was using a separate storage adapter and RAID array to boot TrueNAS, but I want to free up the PCI slot. So I (naively) placed two new SSD onto the backplane, attached them to boot-pool as a mirror, and detached the array. Of course now it doesn't boot. I've tried performing a rescue install selecting the two new SSD, and also all 3 devices including the array, with the same result (attached). Boot device selected in BIOS is "IDE:#0300 ID1E LUN" which is the only option it gives me under HDD.

Motherboard: Supermicro X8DTN+
Storage: LSI 9211-8i in IT mode with 15x 2TB SAS spinning disks on a backplane, now also with 2x SAS 200GB SSD intended for boot
Boot: LSI 9211-4i in IR mode, 2x consumer grade 120GB SATA drives in RAID1

I think next I'll try removing the new SSDs from the backplane and doing a rescue install just on the original RAID, just to see if I can get it to boot. But, even if that works I'm still going to need help transitioning off it so I thought I'd start my thread and see if anyone has any advice.
 

Attachments

  • truenasboot.png
    truenasboot.png
    74.3 KB · Views: 37

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,925
To boot from your HBA it'll need to be flashed with a Boot ROM installation, in this case (Truenas 13,X) I guess a UEFI ROM.
 

taylornate

Dabbler
Joined
Jul 10, 2016
Messages
43
To boot from your HBA it'll need to be flashed with a Boot ROM installation, in this case (Truenas 13,X) I guess a UEFI ROM.
If I do that will I still be able to use the storage pool? Can I have TrueNAS boot discs on the same backplane/card as the storage pool?
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,925
Can I have TrueNAS boot discs on the same backplane/card as the storage pool?
I believe that you can. Seems like you counted on it...
 

taylornate

Dabbler
Joined
Jul 10, 2016
Messages
43
I believe that you can. Seems like you counted on it...
I guess I did, but I didn't understand that at the time.

Is this what you're talking about? In the card configuration it was already set to "BIOS & OS"

But when I tried to boot it said it's NAS storage and not bootable
To boot from your HBA it'll need to be flashed with a Boot ROM installation, in this case (Truenas 13,X) I guess a UEFI ROM.

With this as a guide, according to sas2ircu (below), I should have the bios on the card and it should be bootable? Sorry, when I set this up years ago I read all about flashing to IT mode but this boot ROM idea is new to me so I'm catching up. What am I missing?

Code:
root@truenas[~]# sas2ircu 0 display
LSI Corporation SAS2 IR Configuration Utility.
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved.

Read configuration has been initiated for controller 0
------------------------------------------------------------------------
Controller information
------------------------------------------------------------------------
  Controller type                         : SAS2008
  BIOS version                            : 7.39.02.00
  Firmware version                        : 20.00.07.00
  Channel description                     : 1 Serial Attached SCSI
  Initiator ID                            : 0
  Maximum physical devices                : 255
  Concurrent commands supported           : 3432
  Slot                                    : 6
  Segment                                 : 0
  Bus                                     : 9
  Device                                  : 0
  Function                                : 0
  RAID Support                            : No


edit:

This command looked interesting:
Code:
BOOTENCL   - Select an Enclosure/Bay as primary boot device

But:
Code:
root@truenas[~]# sas2ircu 0 bootencl
LSI Corporation SAS2 IR Configuration Utility.
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved.


SAS2IRCU: The BOOTENCL command is not supported by the firmware currently loaded on controller 0.


So I guess you're right that I need a different firmware? Is there a firmware that well let me boot but also be appropriately compatible with TrueNAS for storage (IT mode, etc.)? All the discussion I find easily is about IT mode.
 
Last edited:

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,925
I have not flashed an HBA for years, but iirc during the flashing process you have the option to add the boot functionality.

Here's a copy from Stliez's excellent flashing writeup from the Hardware section of our Reources at https://www.truenas.com/community/r...9300-9305-9311-9400-94xx-hba-and-variants.54/ which I respectfully suggest that you might read in its entirety for orientation before you go much further...

----oooo----

"FLASHABLE SOFTWARES ("BOOT ROM/BIOS" AND "FIRMWARE"): Confusingly, the 9211 cards have their flashable files in *two* parts. There are bios and efi "roms" that provides boot-time options, and these are *separate* files independent from the "firmware" (the part that actually controls the disks and responds to the OS). For the boot roms one can flash a BIOS orom, an EFI boot rom, or neither or both (as they can coexist).

For HBA use such as FreeNAS, only the "firmware" part normally matters. You don't need to flash the orom or EFI rom. The boot roms are completely optional and while nice, they cause the boot to be slower and they don't add anything really. It can be flashed onto the card later if you ever change your mind. To repeat: you do not need to flash the bios/efi boot roms to get a fully functional HBA card. You only need to flash the firmware to get the card working fully.

The only time you usually need to flash the "rom" firmware (orom/EFI/bios) is if you want to boot the system itself from the card, and the disks attached to the card must be visible at early boot time to do so. Also for some card settings such as built-in staggered spinup or card enable/disable. But because most users use these cards only for attaching data disks, very few users/systems will ever need to flash the "rom" part of the software. For everyone else, they just slow down the boot process, and the "firmware" part of the firmware is all you'll need.

(If you're interested, the bios/boot roms mainly seem to provide features such as exposing disks to the BIOS/firmware during the boot process (if booting off a disk attached to the card), or showing card ID, SAS topology, list attached devices at boot, and enable/disable. They also allow you to set some options for removable media, card boot order if you have >1 card, number of interrupts used, etc. Most of these options aren't really needed for FreeNAS or for an HBA, especially since FreeNAS itself will tell you what's attached when it boots.)

FIRMWARE/BOOT ROM VERSIONS, VARIANTS AND THEIR USUAL FILENAMES: The firmware file itself comes in two variants, depending if it's to be used as an HBA (the "IT" version) or a RAID card (the "IR" version). For FreeNAS you almost always want the IT version of the firmware, but the 9211 variants mainly seem to come in IR versions so they will need cross flashing, to replace the IR version by the IT version, as well as being flashed again to switch from Dell/whoever's firmware to LSI's firmware builds.

The firmware most people will want to get working is LSI/Avago/Broadcom IT software (currently version P20). It may also be known as the firmware for HBA not RAID cards, but a dell/fujitsu card will usually have dell/fujitsu firmwares as well and you'll probably need that as well. The support website or Google will help you find your own card's firmware. (See "crossflashing process"). The firmware and boot rom files, and sometimes the flashers, are often wrapped up in zip or exe files which need to be unpacked and the relevant files extracted. WinRAR, 7zip or other unpackers that can unpack .exe files can be useful for this.

You will usually need your card manufacturer's IT firmware for this process, even though you only need it temporarily and you'll overwrite it with the LSI firmware almost immediately after flashing it. (I'll explain why below: basically there's a step where LSI have a "lockout" that this helps you to work around). IMPORTANT NOTE: when you go looking for your Dell/IBM/Toshiba/??? IT firmware, you do not need the latest version. Any version, however old (almost!) will do. In fact older versions may be easier to flash. Don't worry at all if it's outdated, since you'll overwrite it a few minutes later with the current LSI firmware. It's only needed for a temporary step in the process.

Filenames for the firmware are usually "2118it.bin" (for LSI IT firmware) and "6GBPSAS.FW" (for Dell firmware), whatever version they are. I haven't checked what filenames IBM, Fujitsu and other OEMs usually give their IT firmware versions. LSI make their IT versions easy to find. Dell do it like this: the firmware for their 9211-8i cards is usually IR version ("i"=internal ports, presumed to be RAIDed), and the firmware for their 9211-8e cards is usually identical but IT version ("e"=external ports, presumed to be HBA). So if you have a Dell card like the H200 or H310, instead of looking for an IT version for the 9211-8i look for the firmware for the external port version (equivalent to the 9211-8e). Dell have an A10 version of the firmware titled "SASHBA_Firmware_6GBPS-SAS-HBA_07.03.06.00_A10_ZPE.exe". Expand the .exe file and grab the "6GBPSAS.FW" file inside it, that's the Dell IT version you need for the 9211-8i or 8e. Once again, if it's some other manufacturer, search their support section, ask on their forums or here, or Google it. But in any case you want their IT (HBA) not IR (RAID) firmware version.

If you want to flash either or both boot roms, their standard filenames are x64sas2.rom (for the EFI boot rom) and mptsas2.rom (for the BIOS boot rom also known as option rom or orom). LSI calls their EFI boot rom a UEFI BSD rom (not to be confused with FreeBSD). You can flash either, neither or any combination. But as I've said, you don't have to, and it won't do much for most people.

----oooo----

This should get you pointed in the right direction. Do let us know how it goes.
 

taylornate

Dabbler
Joined
Jul 10, 2016
Messages
43
This should get you pointed in the right direction. Do let us know how it goes.
This was quite informative, thanks. I worked through the guide and found that the BIOS ROM was already loaded which is what I needed, but I went ahead and loaded the EFI ROM too to be safe.

It turned out what I needed to do was select the boot drives within the motherboard bios.

It's working now--thanks for the help!
 
Top