SOLVED Remotely flash LSI 9300 with FreeDOS Live CD

[blanchet]

Abstract
This guide explains how to create a custom FreeDOS Live CD to remotely upgrade the firmware of LSI 9300 HBA on a Supermicro server.

• The custom LiveCD is created from a Linux computer.
• This guide is based on the instructions from https://www.youtube.com/watch?v=IwPsadNho9M
• This guide can also be useful to create a custom Live CD for other tools that needs FreeDOS.
• For the moment it works only on computers that boot in LEGACY mode, it does not work in UEFI mode.

Introduction
To operate smoothly with TrueNAS, the Broadcom/LSI 9300 HBA needs to run a very specific firmware:

• IT mode
• at least version 16.00.12.00 (this version is only available on the TrueNAS community forum)

LSI 9300-xx Firmware Update

JoshDW19 submitted a new resource: LSI 9300-xx Firmware Update - Fixes a controller reset issue in LSI 9300-xx HBA cards Hey Community, If you are using an LSI 9300 HBA with FreeNAS or the soon-to-be TrueNAS CORE, you may experience some performance issues causing the controller to reset...

www.truenas.com

Despite TrueNAS comes with sas3flash for FreeBSD, this version suffers several drawbacks

• it does not support downgrading the firmware version
• it does not support switching from IR to IT mode (or the opposite direction)

Flashing firmware and BIOS on LSI SAS HBAs

Flashing firmware and BIOS on LSI SAS HBAs

www.broadcom.com

Indeed, only the DOS version of sas3flash works for any scenario, so you have to create a custom FreeDOS Live CD with the flashing utility to update remotely (through IPMI) a Supermicro server running TrueNAS.

This guide has been tested only on Debian Linux 11.
It should also work on Ubuntu and other Linux distributions.

Create the working environment and download the resources
Create a working directory

Code:

mkdir ~/freedos-live-sas3
cd ~/freedos-live-sas3

Download the needed resources:

• FreeDOS 1.3 LiveCD
• sas3flash for DOS
• firmware 16.00.12.00 for Broadcom 9300 HBA

You cannot download sas3flash for DOS from the command line because you have to accept the Broadcom license.
So use your web browser to get the file and copy it later to the working directory

Broadcom Inc. | Connecting Everything

docs.broadcom.com

Code:

cp ~/Downloads/Installer_P16_for_MSDOS_and_Windows.zip ~/freedos-live-sas3
wget https://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.3/official/FD13-LiveCD.zip
wget https://www.truenas.com/community/resources/lsi-9300-xx-firmware-update.145/download -O SAS3_FW_Phase16.0-16.00.12.00_Firmware.zip

Extract the archives

Code:

unzip FD13-LiveCD.zip
unzip SAS3_FW_Phase16.0-16.00.12.00_Firmware.zip
unzip Installer_P16_for_MSDOS_and_Windows.zip

Mount the FreeDOS LiveCD image in a directory to access the content

Code:

mkdir src-cd
sudo mount -t iso9660 -o loop FD13LIVE.iso src-cd/

Prepare the custom FreeDOS LiveCD
Copy the content of the FreeDOS LiveCD to another directory

Code:

mkdir custom-cd
cp -r src-cd/isolinux custom-cd/
chmod -R +w custom-cd/isolinux

Copy the flash utility and the firmwares you need

Code:

cp Installer_P16_for_MSDOS_and_Windows/sas3flash_dos_rel/sas3flsh.exe custom-cd/
cp SAS3_FW_Phase16.0-16.00.12.00_Firmware/9300-8i/SAS9300_8i_IT.bin custom-cd/fw8i-it.bin
echo "SAS3_FW_Phase16.0-16.00.12.00_Firmware/9300-8i/SAS9300_8i_IT -> fw8e-it.bin" >> custom-cd/README.TXT

cp SAS3_FW_Phase16.0-16.00.12.00_Firmware/9300-8e/SAS9300_8e_IT.bin custom-cd/fw8e-it.bin
echo "SAS3_FW_Phase16.0-16.00.12.00_Firmware/9300-8e/SAS9300_8e_IT -> fw8e-it.bin" >> custom-cd/README.TXT

• Copy only the firmware you really need
• In this example I copy the firmware for 9300-8i and 9300-8e
• Shorten the filename because FreeDOS allow only 8 characters + 3 extentions
• I create also a README.TXT file to know the original filename

Optional edit the boot menu title

Code:

nano custom-cd/isolinux/isolinux.cfg

Edit the "live" entry, change the label and the help text

Code:

label live
menu label Use FreeDOS 1.3 Live to flash LSI 9300 firmware
menu default
text help
Go to D: to find the flash utility and firmwares
endtext

It is a good idea to edit the boot menu title and help text to remember the purpose of the LiveCD

Generate the ISO image

Code:

sudo apt install genisoimage
cd custom-cd
genisoimage -o ../fdlive-flash.iso -V "FreeDOS 1.3 LiveCD SAS3FLASH" -b isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table -c isolinux/boot.cat .

• Do not forget the final dot at the end of command line
• You can use VirtualBox to test your new FreeDOS LiveCD

Boot the Supermicro server with the LiveCD

Virtual CDROM does not work with the HTML5 KVM !

• Connect to the IPMI interface of the Supermicro server
• Click on the Remote Console Preview to launch the JAVA KVM
• 
• Open Virtual Media | Virtual Storage
• 
  • Logical Drive Type: ISO File
  • Open: fdlive-flash.iso
  • Click Plug in to mount the ISO image
  • A message appears: Device1:VM Plug-In OK!!
  • Click OK to close the windows
  • 
• Reboot the Supermicro server
• When the computer restarts press F11 to display the boot menu

If the Virtual CD-ROM does not appear in the boot menu, then Enter Setup

• select Enter Setup
• go to BOOT
  • Boot mode : LEGACY (do not forget to restore to UEFI later)
  • Boot Option #1, USB CD/DVD ATEN Virtual CROM YS0J
  • 
• Save and Exit (F4)
• The computer reboots on the LiveCD
• Select Use FreeDOS 1.3 Live to flash LSI 9300 firmware

After booting the prompt display R:\>

Code:

R:\>

Switch to D: and run sas3flsh.exe

Code:

R:\> d:
D:\> sas3flsh.exe -listall
        Adapter Selected is a Avago SAS: SAS3008(C0)

Num   Ctlr            FW Ver        NVDATA        x86-BIOS         PCI Addr
----------------------------------------------------------------------------

0  SAS3008(C0)  16.00.01.00    0e.01.00.08    08.37.00.00     00:18:00:00
1  SAS3008(C0)  16.00.01.00    0e.01.00.07    08.37.00.00     00:af:00:00

        Exiting SAS3Flash.

I have 2 SAS3008 boards #0 and #1 and I have to check the models

Code:

D:\> sas3flsh.exe -c 0 -list
        Adapter Selected is a Avago SAS: SAS3008(C0)

        Controller Number              : 0
        Controller                     : SAS3008(C0)
        Firmware Product ID            : 0x2221 (IT)
        Firmware Version               : 16.00.01.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : SAS9300-8e
        BIOS Version                   : 08.37.00.00
        UEFI BSD Version               : 18.00.00.00
        FCODE Version                  : N/A
        Board Name                     : SAS9300-8e

The board #0 has the firmware 16.00.01.00 and it is a 9300-8e model, so i use fw8e-it.bin

If you do not remember the original filenames use the following command to display D:\README.TXT

Code:

type D:\README.TXT

Flash the firmware with 9300-8e firmware 16.00.12.00

Code:

D:\> sas3flsh.exe -c 0 -f fw8e-it.bin

Check the firmware after the flashing

Code:

D:\> sas3flash -c 0 -list
        Adapter Selected is a Avago SAS: SAS3008(C0)

        Controller Number              : 0
        Controller                     : SAS3008(C0)
        Firmware Product ID            : 0x2221 (IT)
        Firmware Version               : 16.00.12.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : SAS9300-8e
        BIOS Version                   : 08.37.00.00
        UEFI BSD Version               : 18.00.00.00
        FCODE Version                  : N/A
        Board Name                     : SAS9300-8e

Ok board #0 has been correctly reflashed.

Repeat for board #1

Display info for board #1

Code:

D:\> SAS3DOS\sas3flsh.exe -c 1 -list
        Adapter Selected is a Avago SAS: SAS3008(C0)

        Controller Number              : 1
        Controller                     : SAS3008(C0)
        Firmware Product ID            : 0x2221 (IT)
        Firmware Version               : 16.00.01.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : SAS9300-8i
        BIOS Version                   : 08.37.00.00
        UEFI BSD Version               : 18.00.00.00
        FCODE Version                  : N/A
        Board Name                     : SAS9300-8i

Board #1 has the firmware 16.00.01.00 and it is a 9300-8i , so I need fw8i-it.bin

Code:

D:\> sas3flsh.exe -c 1 -f fw8i-it.bin

Display info for board #1

Code:

D:\> sas3flsh.exe -c 1 -list
        Adapter Selected is a Avago SAS: SAS3008(C0)

        Controller Number              : 1
        Controller                     : SAS3008(C0)
        Firmware Product ID            : 0x2221 (IT)
        Firmware Version               : 16.00.12.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : SAS9300-8i
        BIOS Version                   : 08.37.00.00
        UEFI BSD Version               : 18.00.00.00
        FCODE Version                  : N/A
        Board Name                     : SAS9300-8i

Now use the menu MACRO to send CTRL-ALT-DEL to reboot the server

• Unmount the virtual CDROM
  • Virtual Media | Virtual Storage | Plug Out
• When the computer reboots, press DEL to enter the BIOS
• If needed switch back the boot mode to UEFI
• Boot the server

Enjoy your Broadcom 9300 HBA with the appropriate firmware to run TrueNAS !

 

[sretalla]

Great and thorough post, but why wouldn't you just offline the pool (export it without wiping and keep the share config, flash, then re-import the pool) and run it while still in TrueNAS?

 

[blanchet]

Great and thorough post, but why wouldn't you just offline the pool (export it without wiping and keep the share config, flash, then re-import the pool) and run it while still in TrueNAS?

Because:

• on one of my server the boot pool is connected to the HBA, so I cannot offline it
• with the FreeBSD version of sas3flash you can neither switch to IT mode nor downgrade the firmware
• I would like to learn how to create my own custom FreeDOS LiveCD for other flashing utilities (motherboard, network interface, etc)

 

[sretalla]

on one of my server the boot pool is connected to the HBA, so I cannot offline it

Fair point.

with the FreeBSD version of sas3flash you can neither switch to IT mode nor downgrade the firmware

OK, good to know.

I would like to learn how to create my own custom FreeDOS LiveCD for other flashing utilities (motherboard, network interface, etc)

Can't hurt, I suppose.

I didn't mean to disparage your efforts, just wondered why... now I know.