9.3.1 Update with alert Firmware version 16 does not match driver version 20 for /dev/mps0

Status
Not open for further replies.

bollar

Patron
Joined
Oct 28, 2012
Messages
411
Wow, didn't realize that was possible. Sure would have saved me a lot of time not having to pull out the chassis and pull out each HBA one by one. In cases where you have multiple controllers installed in the same server, would the above sas2flash command update all the controllers at once? Would there be any risks associated with doing it that way with multiple controllers? I know a lot of the M1015 instructions floating around caution to always unplug/remove everything other than the specific controller you want to update. Maybe those instructions were all put together by folks with a healthy done of paranoia as Ericloewe eluded to? :)
I have three, including the onboard X9DRD's 2308 and sas2flash -fwall 9207-8i.bin executed from inside the FreeNAS shell updated all three at one time. All appear to be working properly.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
Ok gents, I got some info from a higher source.

The P20 stuff has been tested in the burnin lab at iX for almost a year. So I'd say "yes, it's been tested on hardware we recommend" and vendor specific stuff may or may not be as 'friendly'. If the P20 stuff freaks you out (and frankly I wouldn't be surprised considering the problems we've had with P16 driver with p20 firmware) then stay on P16 and don't upgrade to 9.3.1 yet. I'm not particularly worried about P20 firmware, with one exception.

P20 has had 3 'versions'. As of this writing the current is 20.00.04.00 and was released on May 21, 2015. There are other versions of 20.00.0x.xx out there. The driver that FreeNAS uses is 20.00.04. In an ideal (and most stable world) you'd want the 20.00.04 driver and firmware. I'm not privy to exactly what was changed, but I've heard through the grapevine that you'd ideally want to be on this latest build and not one of the p20 builds from prior as they have problems. I don't know how big of a deal the problems are for those of us that don't use the RAID function, but I prefer to be safer than sorry.

Here's where things get a little more confusing (yeah, I'll probably do an article on this to clear everything up). Supermicro hasn't provided the p20 firmware with their lsi sas 2008 on-board chips. You can upgrade these using the LSI/Avago stock firmwares. But you have to use specific firmwares. I want to confirm the exact matches before I publish these for the world as I'd rather not provide incorrect info.

In short:

- If you've been on 9.3.0 and p16 firmware and don't have onboard SAS chips, feel free to upgrade to 9.3.1 and flash the p20 firmware (download it from LSI and not elsewhere so you are sure you have the latest build). The risk of problems with p20 drivers and p20 firmware is very low.

- If you've been on 9.3.0 and p16 firmware and have onboard SAS chips that are have p20 firmware and the firmware is from May 2015 or newer, feel free to upgrade the LSI firmware and FreeNAS to 9.3.1. From the information I have the 'big ones' that don't have p20 official firmware are Supermicro boards with the LSI 2008 chipset and AsRock Rack motherboards. I have no ETA on AsRock Rack's release schedule, but apparently running P20 has been unstable for some people. Since AsRock Rack hasn't updated their firmware to P20 yet I have no idea what that means. I'll see what I can found out.

- If you've been a bad boy and have been using p20 firmware all along, you should upgrade to the latest p20 firmware and upgrade to FreeNAS 9.3.1 as soon as possible. P16 drivers and P20 firmware is a recipe for problems. Since it has been discussed in depth many times before I won't bother with any more on the topic.


- I definitely wouldn't recommend people upgrade to FreeNAS 9.3.1 and keep the p16 firmware though. If you don't see a P20 firmware for your hardware, don't upgrade to 9.3.1. Contact your hardware vendor and find out what the solution is. They may be releasing a p20 firmware soon.

- If this stuff freaks you out... just don't upgrade. 9.3.1 is new, and people are still working on getting familiar with the added features, bug fixes, etc. It's not the end of the world to wait a week or two for things to be a little more comfy.

You could have (and still can) upgrade from the shell. The main reason why we haven't generally recommended upgrading from the shell is because there's no telling what will happen to your zpool during that period between when you start the upgrade process and actually reboot. There's *going* to be a period of time where you will be on one driver and a different firmware. So it was about mitigating the problems.

I, personally, don't recommend upgrading from the shell because of all of the possible variables involved. TrueNAS does it automatically. When you upgrade a TrueNAS system, the LSI firmware is included and upgraded accordingly. But TrueNAS hardware is very specific, very heavily tested, etc. When dealing with FreeNAS it's random hardware put together by someone that may or may not have chosen good components, may or may not have customized the OS, and may or may not have other unknown problems.

As always though.. keep a backup. :)
 

Sir.Robin

Guru
Joined
Apr 14, 2012
Messages
554
I flashed mine with the supermicro utility. No problems at all. Just to be safe, I'm now considering upgrading further with LSI's package, but i would rather have Supermicro release their own.

Oh yeah, and my NAS02 will not be upgraded yet :)
 
J

jkh

Guest
- I definitely wouldn't recommend people upgrade to FreeNAS 9.3.1 and keep the p16 firmware though. If you don't see a P20 firmware for your hardware, don't upgrade to 9.3.1. Contact your hardware vendor and find out what the solution is. They may be releasing a p20 firmware soon.

Actually, that's the one scenario that LSI/Avago has always supported. "New driver, old firmware" is one of the tested configurations because you can't always control the pace at which firmware is updated on a customer system, and unless there was a very specific bug in v16 that was hosing you (in which case, it would hose you regardless of the driver version), you shouldn't notice any malfunctions with p20 driver and p16 firmware.

It was the opposite scenario that LSI is firmly against (and we saw issues with), where someone was using the P20 firmware and the P16 driver in combination, since obviously the old driver doesn't have the opportunity to "quirk" a specific version of firmware that didn't even exist back when it was written. Conversely, if the p20 driver knew that, for example, v17 firmware needed to be talked to in a specific way, it could detect that configuration and do the right thing.
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
So given what @jkh said about backward compatibility, I don't see p16 FW w/ p20 driver being a reason for not upgrading. Is that accurate? (IOW, I can upgrade to 9.3.1 now and upgrade my FW to p20 at some point in the future).
 

j_r0dd

Contributor
Joined
Jan 26, 2015
Messages
134
ASRock has said they found the performance of P20 to be bad in their initial tests and might release a P19, but that is probably as good as it is going to get, for now anyways....

@jkh thanks for that info as it may not really matter after all.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
Actually, that's the one scenario that LSI/Avago has always supported. "New driver, old firmware" is one of the tested configurations because you can't always control the pace at which firmware is updated on a customer system, and unless there was a very specific bug in v16 that was hosing you (in which case, it would hose you regardless of the driver version), you shouldn't notice any malfunctions with p20 driver and p16 firmware.

It was the opposite scenario that LSI is firmly against (and we saw issues with), where someone was using the P20 firmware and the P16 driver in combination, since obviously the old driver doesn't have the opportunity to "quirk" a specific version of firmware that didn't even exist back when it was written. Conversely, if the p20 driver knew that, for example, v17 firmware needed to be talked to in a specific way, it could detect that configuration and do the right thing.
That seems to contradict with everything we've been told for many years.. even longer than I've been here and across other forums too.

@jkh (or anyone for that matter)

Anyone able to confirm this with some kind of official documentation from LSI/Avago? I just cringed when I read that because it's so en-trained in those of us with experience here (and elsewhere).

Everyone I know, for as long as I've been using LSI/Avago, has always said that they must match and anything else is unsupported and untested.
 

Sir.Robin

Guru
Joined
Apr 14, 2012
Messages
554
That seems to contradict with everything we've been told for many years.. even longer than I've been here and across other forums too.

@jkh (or anyone for that matter)

Anyone able to confirm this with some kind of official documentation from LSI/Avago? I just cringed when I read that because it's so en-trained in those of us with experience here (and elsewhere).

Everyone I know, for as long as I've been using LSI/Avago, has always said that they must match and anything else is unsupported and untested.

I have no documentation at hand, but that seems to go along with what HP "always" have been touting. Always update Driver first and then upgrade your firmware as soon as possible if critical update exsist.
 
J

jkh

Guest
Well, I checked around a bit more and apparently the only thing that Avago now certifies is "Driver version = Firmware version". There is no "official line" about whether vF < vD or vD > vF is better for any consistent value of better (though everyone agrees that VD = bad).

I also asked @jpaetzel, who has debugged many misbehaving systems in the field, and he also said that "we have demonstrated cases that mps 16 + p20 firmware is bad" and "we have demonstrated cases that mps20 + p16 firmware is bad" so I guess all that anyone can agree on is that not matching them exactly and precisely is bad, has always been bad, and always will be bad. There is no good. We live in a cruel, uncaring world where the air is progressively more polluted, the oceans are full of plastic waste products and dying fish, and our natural resources are being depleted even more quickly than the population is increasing. We are all destined to die choking and alone, packed in a tiny and ever-contracting cage like rats in some cruel science experiment with no point other than to inflict pain and suffering.

Hope this helps!
 

Sir.Robin

Guru
Joined
Apr 14, 2012
Messages
554
Right. Moving on! :)

Supermicro flashing! What are all ther ROM's?:

sas2flash.efi -f 2308IT20.ROM
sas2flash.efi -b mptsas2.rom
sas2flash.efi -b x64sas2.rom
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
Thank you for providing that glimmer of hope in the survival of humanity.
:smile:
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
ftp://ftp.supermicro.com/driver/SAS/LSI/
 

Sir.Robin

Guru
Joined
Apr 14, 2012
Messages
554
ftp://ftp.supermicro.com/driver/SAS/LSI/

I'm wondering "what" not "where" :)

LSI flash: sas2flash -fwall xxxxx.bin -b mptsas2.rom

But then i look at Supermicro's script:

sas2flash.efi -f 2308IT20.ROM
sas2flash.efi -b mptsas2.rom
sas2flash.efi -b x64sas2.rom

why? Whats the difference?
 

Z300M

Guru
Joined
Sep 9, 2011
Messages
882
I'm wondering "what" not "where" :)

LSI flash: sas2flash -fwall xxxxx.bin -b mptsas2.rom

But then i look at Supermicro's script:

sas2flash.efi -f 2308IT20.ROM
sas2flash.efi -b mptsas2.rom
sas2flash.efi -b x64sas2.rom

why? Whats the difference?
I certainly claim no expertise, but it seems to me that 2308IT20.ROM is the firmware, while the other two are BIOS files -- but why two rather than only one, I have no idea. And I am sure I have read that unless you are trying to boot from a drive attached to an LSI controller, you don't need the BIOS, and booting will be quicker as a result.
 

Sir.Robin

Guru
Joined
Apr 14, 2012
Messages
554
Yes the first is the Firmware. And then you have the BIOS for for card. But why two. In the link (http://www.bussink.ch/?p=1511) i posted the guy do flash with LSI utility and gets his files from LSI, but he does not use the mptsas2.rom, he downloads the UEFI_BSD_Pxx package and uses the x64sas2.rom instead of the mptsas2.rom found togheter with the firmware in the firmware package.

So instead of the "ususal" sas2flash -fwall xxxxx.bin -b mptsas2.rom

He uses this: sas2flash -fwall xxxxx.bin -b x64sas2.rom

And supermicro does both? I wanna know why. o_O
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
Why even bother with the rom? It's not like you need to boot from one of those drives.
 

Sir.Robin

Guru
Joined
Apr 14, 2012
Messages
554
Why even bother with the rom? It's not like you need to boot from one of those drives.
Talk to the hand!! :D It's not the point. I have the BIOS disabled. Still wanna know whyyyy!!!
 

Z300M

Guru
Joined
Sep 9, 2011
Messages
882
Sorry, but I don't know the answer.

I am curious why most are using uefi instead of FreeBSD to upgrade, but I guess that doesn't really matter either.


Sent from my iPad using Tapatalk
Having read where people had got the same "not operational" message and had solved the problem by trying the board in a different slot or even in a totally different machine, I felt sure that rebooting or even powering down and up again would be safe, and that was indeed the case -- but it took pclausen's p20.zip files to accomplish the task rather than the package I got directly from LSI.
 

adrianwi

Guru
Joined
Oct 15, 2013
Messages
1,231
After reading the thread with interest, I'm not really sure why I felt the need to upgrade but feeling in a brave/stupid mood I look the plunge :D

  1. Downloaded the P20 firmware from the Supermicro FTP site - ftp://ftp.supermicro.com/driver/SAS/LSI/2308/Firmware/IT/
  2. Unzipped and copied the files from the UEFI folder onto a USB stick
  3. Launched the iKVM Viewer from the IPMI console
  4. Shut down Freenas from the WUI
  5. Hit F11 and booted into the UEFI Shell
  6. Switched to the USB stick with "fa0:" command
  7. Ran the firmware update with "SMC2308T.NSH" command
  8. Entered the last 9 digits of the SAS address (I'd saved this when I originally flashed it to P16)
  9. Rebooted into Freenas and saw the yellow warning about the mismatch
  10. Ran the Freenas 9.3.1 update
After the system rebooted, everything looks fine, so if a noob like me can do it, should be a doddle for anyone else :D
 
Status
Not open for further replies.
Top