SuperMicro X11-SSH and LSI 3008

[mloiterman]

• motherboard make and model
  • SuperMicro X11SSH-CTF
• CPU make and model
  • Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz (8 cores)
• RAM quantity
  • 32 GiB
• hard drives, quantity, model numbers, and RAID configuration
  • 8 x ST4000LM024
  • RAIDZ2
• hard disk controllers
  • Avago Technologies (LSI) SAS3008
• network cards
  • None - using on board networking
• FreeNAS-11.2-RELEASE (Build Date: Dec 5, 2018 21:28)

Section 2.5.1 of 11.2 Manual suggests "Upgrading the firmware of Broadcom SAS HBAs to the latest version..."

In connection with this recommendation, I investigated my firmware version and have found:

Code:

root@marshall:/ # sas3flash -list
Avago Technologies SAS3 Flash Utility
Version 16.00.00.00 (2017.05.02)
Copyright 2008-2017 Avago Technologies. All rights reserved.

    Adapter Selected is a Avago SAS: SAS3008(C0)

    Controller Number              : 0
    Controller                     : SAS3008(C0)
    PCI Address                    : 00:01:00:00
    SAS Address                    : 5003048-0-1e04-6000
    NVDATA Version (Default)       : 0b.02.31.27
    NVDATA Version (Persistent)    : 0b.02.31.27
    Firmware Product ID            : 0x2721 (IR)
    Firmware Version               : 12.00.02.00
    NVDATA Vendor                  : LSI
    NVDATA Product ID              : LSI3008-IR
    BIOS Version                   : 08.29.01.00
    UEFI BSD Version               : 14.00.00.00
    FCODE Version                  : N/A
    Board Name                     : LSI3008-IR
    Board Assembly                 : N/A
    Board Tracer Number            : N/A

    Finished Processing Commands Successfully.
    Exiting SAS3Flash.

It appears, based on Broadcom website (https://www.broadcom.com/products/storage/host-bus-adapters/sas-9300-8i#downloads), that latest version of firmware is 16.00.01.00 and the latest version of the FreeBSD drive is 20.00.00.00. So my firmware and driver are both out of date. But, my understanding is that driver comes from FreeNAS.

I've already upgraded my server to 11.2 without any issue and everything appears to be working correctly, so my thought is to leave well enough alone, but I'm wondering if I should still upgrade.

If so, there really doesn't appear to be very good or detailed instructions anywhere on Broadcom's site. They seem to indicate you can do it with the sas3flash utility, but it's not clear if I have to boot into the UEFI mode from the motherboard or if it can be done on a running system, say from within FreeNAS. It's really poorly documented and prior to undertaking this, I would like to see if there is any comment from the board on how to proceed.

 

[Chris Moore]

Firmware Version : 12.00.02.00

is what your card is reporting.

Avago Technologies SAS3 Flash Utility Version 16.00.00.00 (2017.05.02)

This is the version of the utility you are using to generate the report

Firmware Product ID : 0x2721 (IR)

You need a firmware upgrade, but you also need to change from the IR version of the firmware to the IT version.

Please review this documentation:

Detailed newcomers' guide to crossflashing LSI 9211 HBA and variants
https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

 

[mloiterman]

is what your card is reporting.

This is the version of the utility you are using to generate the report

You need a firmware upgrade, but you also need to change from the IR version of the firmware to the IT version.

Please review this documentation:

Detailed newcomers' guide to crossflashing LSI 9211 HBA and variants
https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

Guide is very detailed, but concerning that the guide is fairly old (June '17) and the devices discussed are not the same as mine. That Megarec.exe utility is nowhere to be found in the SAS3008 firmware files I've downloaded. Also, my motherboard must boot straight to EFI in order for my NVM ssd to work, so that DOS stuff is not likely to work for me.

Seems very risky with little benefit.

Any comment on benefit of upgrade and / or cross flashing?

 

[Chris Moore]

Seems very risky with little benefit.

You have the risk / reward backward. The risk is to use the system with the wrong firmware installed.
The methods of changing the firmware have not really changed with the newer model cards.
There is (I am sure it is in the document) a UEFI mode flash utility.

 

[Chris Moore]

Also, my motherboard must boot straight to EFI in order for my NVM ssd to work,

PS. I wouldn't know about that. You didn't follow the guide:

Updated Forum Rules 12/5/18
https://forums.freenas.org/index.php?threads/updated-forum-rules-12-5-18.45124/

 

[mloiterman]

PS. I wouldn't know about that. You didn't follow the guide:

Updated Forum Rules 12/5/18
https://forums.freenas.org/index.php?threads/updated-forum-rules-12-5-18.45124/

Whoops...updated first post to include hardware and FreeNAS rev version.

 

[mloiterman]

is what your card is reporting.

This is the version of the utility you are using to generate the report

You need a firmware upgrade, but you also need to change from the IR version of the firmware to the IT version.

Please review this documentation:

Detailed newcomers' guide to crossflashing LSI 9211 HBA and variants
https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

In reading through that guide, I think it's more clear now that it's written for flashing an external card. My LSI chip is on my motherboard. Is the suggested path still the same?

I'm now looking at this:
https://www.broadcom.com/support/kn...44/flashing-firmware-and-bios-on-lsi-sas-hbas

 

[CraigD]

Flashing the chip flashes the chip!

One of my SAS controllers is on the motherboard, the process was the same

Have Fun

 

[mloiterman]

For anyone interested, I was able to change to IT firmware and upgrade. I choose not to follow the process outlined at the LSI cross flashing post because I am using the chip on my motherboard rather than an external card. I don't know if that is relevant or not, but I was reluctant to flash with the regular firmware rather than the one supplied by SuperMicro. I realize it all ultimately comes from broadcom. Anyway, here is process I followed:

0. Within FreeNAS and prior to undertaking firmware update, capture the last nine digits of the SAS adaptor's address by entering at command line: sas3flash -c 0 -list and noting the SAS Address line.
1. Download Phase 15 firmware from Supermicro FTP. Although this version is not the latest from broadcom, it is the latest version provided by SuperMicro:
2. Unzip the BCM3008P152T001.zip file. Note the T, which indicates IT mode.
3. Unzipped directory is called "Firmware." Within there is a directory call IT. Within that directory, I loaded the UEFI contents onto a USB drive.
4. Boot into UEFI mode on the SuperMicro board by pressing F11 at boot.
5. Be sure USB is plugged in and select the USB by typing fs1: It might be fs0:for you, however.
6. Enter SMC3008T.NSH
7. When prompted, enter the last nine digits of the SAS Address.
8. Reboot when it is complete.

Here is what it shows now:

Code:

root@marshall:~ # sas3flash -c 0 -list
Avago Technologies SAS3 Flash Utility
Version 16.00.00.00 (2017.05.02)
Copyright 2008-2017 Avago Technologies. All rights reserved.

    Adapter Selected is a Avago SAS: SAS3008(C0)

    Controller Number              : 0
    Controller                     : SAS3008(C0)
    PCI Address                    : 00:01:00:00
    SAS Address                    : 5003048-0-1e04-6000
    NVDATA Version (Default)       : 0e.00.20.00
    NVDATA Version (Persistent)    : 0e.00.20.00
    Firmware Product ID            : 0x2221 (IT)
    Firmware Version               : 15.00.03.00
    NVDATA Vendor                  : LSI
    NVDATA Product ID              : LSI3008-IT
    BIOS Version                   : 08.35.00.00
    UEFI BSD Version               : 17.00.00.00
    FCODE Version                  : N/A
    Board Name                     : LSI3008-IT
    Board Assembly                 : N/A
    Board Tracer Number            : N/A

    Finished Processing Commands Successfully.
    Exiting SAS3Flash.

 

[Chris Moore]

I don't know if that is relevant or not, but I was reluctant to flash with the regular firmware rather than the one supplied by SuperMicro. I realize it all ultimately comes from broadcom.

If you realize the firmware comes from the same place, why are you reluctant to use the manufacturer firmware? LSI / Broadcom is the maker of the chip and the correct source for the firmware and they are the source of the driver that is integrated into BSD (the operating system).
The most likely reason the Supermicro site doesn't have the latest version is that they just have not bothered to update it. You would be better served by getting the most recent firmware from the Broadcom site. You are whinging about this like you are the only one that ever updated a firmware.

 

[Johnnie Black]

ST4000LM024

Just in case you don't know these are SMR drives, doesn't mean you can't use them, I use them myself, but be aware that for example a resilver with half full disks will take around 3 days.