M1015 Downgrade to P16 not right

[antsrealm]

Hi,

I had the warning come up since upgrading to the latest freenas so I attempted to downgrade my M1015 using the following lines in the UEFI shell.

Code:

fs0:
sas2flash.efi -o -e 7
sas2flash.efi -o -f 2118it.bin
sas2flash.efi -o -sasadd <myaddress here>

All seemed to succeed except now I don't get the post from the card any more. I have been reluctant to boot into freenas and see what happens in case it kills my pool. I can't see any of the HDD listed in the BIOS either where previously I could.

I was running P19 with no issues and after applying P16 I no longer have the post screen for the card and it looks like it isn't working.

On the sticker on my card it shows as a SAS9220-8i but I'm pretty sure I had the 9211 firmware running previously.

I also tried the whole original process using megarec etc. with no difference to the result.

Can anyone advise where I have gone wrong ?? This is fairly urgent as I have no access to my work files at the moment.

Thanks,

 

[antsrealm]

when I do the -listall options it shows as FW ver 16 but the X-86-BIOS says No Image.

Is there something wrong here?

 

[FlynnVT]

Having erased the card, you now need to flash a BIOS image too:

e.g.: sas2flash.efi -o -b mptsas2.rom​

...or you can leave the BIOS blank if you don't need to boot from the M1015.

A BIOS-less card works the same under FreeNAS, with the upside of being marginally faster to boot overall. I prefer having the BIOS in place anyway, just to be able to see the list of attached discs before booting.

 

[antsrealm]

Yeah ok that makes sense. I did read something like that but wasn't sure if that was the same thing they were talking about. I have booted it up now and it seems ok. I just saw the difference and thought I had stuffed something. Just out of interest if I ever boot freenas up with some drives disconnected for example I accidentally leave half the cables unplugged could I turn it off again plug them back in and have it recover or is it dead after that ?

Thanks,
Tony.

 

[FlynnVT]

My experience makes me believe that you won't loose data, but you might have a bit of work to bring everything back into sync afterwards if changes are made to the filesystem with some discs missing.

e.g.: raidz1 with discs #1, #2 & #3

With all discs disconnected there is no problem.

With 2 discs disconnected, ZFS simply says that the pool is unworkable (zpool status) and doesn't mount. Rebooting with all discs connected brings everything back OK, no further work required. I have direct experience of this.

What happens if only 1 disc was disconnected? Would FreeNAS/ZFS automatically mount the "degraded" pool? I expect so. If you then modified the pool, through atime access or actual writes, the "offline" disc would be out-of-date when added later. You might have to offline/online it or scrub to bring everything back into sync. I played around with this in a VM when I first started using FreeNAS - to satisfy myself that I wasn't entering a world of pain. I remember that I was happy with the result and recovery scenario, but not what the precise sequence was!

 

[antsrealm]

Yeah that's great. I can handle a scrub. I was just curious if for eg with the 2 discs removed as mentioned above would it try to mount it then have it fail due to not enough parity data. Good to know it wont mount and you can simply reconnect the discs and reboot.

Thanks.

 

[Ericloewe]

Having erased the card, you now need to flash a BIOS image too:

e.g.: sas2flash.efi -o -b mptsas2.rom​

...or you can leave the BIOS blank if you don't need to boot from the M1015.

A BIOS-less card works the same under FreeNAS, with the upside of being marginally faster to boot overall. I prefer having the BIOS in place anyway, just to be able to see the list of attached discs before booting.

There is no reason whatsoever to add the boot ROM. It's absolutely useless for FreeNAS and has known issues that can be avoided by not flashing the boot ROM, without any loss of functionality.

Specifically, drives may not even show up during POST - it's a known issue that was fixed in P17. When several cards are installed, there may also be conflicts between their ROMs.

 

[antsrealm]

Yep got it. Thanks Eric. Just wanted to check before I had a potential disaster on my hands. Everything worked out fine.

 

[Radu]

OK fine enough, I have only one question why did you downgraded to p16 from p19? And why is it so important that the fw on the controller to be the same with the driver version? For example i have M1015 with p19 fw and everything works fine, of course with the yellow alert from freenas.

 

[Ericloewe]

OK fine enough, I have only one question why did you downgraded to p16 from p19? And why is it so important that the fw on the controller to be the same with the driver version? For example i have M1015 with p19 fw and everything works fine, of course with the yellow alert from freenas.

Because it's known to cause problems.

You can follow our advice or not, but don't complain afterwards if you end up losing data.

 

[Radu]

First of all, thanks for the answer. I can downgrade the fw no problem about that, but I want to know why in more detail about the problems that can occur.
In the future versions of Freenas will you plan to upgrade the driver version from p16 to p19 ? Please tell me where can i find some more specific information about the problems that can occur due to driver and fw version differences.

 

[Z300M]

Having erased the card, you now need to flash a BIOS image too:

e.g.: sas2flash.efi -o -b mptsas2.rom​

...or you can leave the BIOS blank if you don't need to boot from the M1015.

A BIOS-less card works the same under FreeNAS, with the upside of being marginally faster to boot overall. I prefer having the BIOS in place anyway, just to be able to see the list of attached discs before booting.

I'm using both the X10SL7-F's on-board SAS (flashed to IT mode P16) and an M1015 (also flashed to IT mode P16), but the only drives I see at boot time are those connected to the M1015. Is this expected behavior, or did I not flash the BIOS of the on-board SAS? (It's been too long ago: I don't remember -- and I probably just followed someone else's instructions.)

 

[Radu]

This is the same case for me also(i have both ROM's but together only the M1015 prevails :) ) and i have a similar setup like yours. I guess that the problem is with the 2 HBA adapters which, at boot time, they are negotiating something, i do not know what, and in the end only the disks from the M1015 are shown. I had also a problem with the pcie slot; if i did not put it manually in BISO on pcie v2.0 from time to time, at boot, the M1015 did not post and did not worked. After the reboot it worked but putting in BIOS the slot to pcie v2.0 fixed this problem. I am runing p19 on both the controllers and until now i had no problems, i am thinking to downgrade to p16 but first of all I want to find out why should i do this.

 

[Ericloewe]

I'm using both the X10SL7-F's on-board SAS (flashed to IT mode P16) and an M1015 (also flashed to IT mode P16), but the only drives I see at boot time are those connected to the M1015. Is this expected behavior, or did I not flash the BIOS of the on-board SAS? (It's been too long ago: I don't remember -- and I probably just followed someone else's instructions.)

[/QUOTE]

I believe that the expected behavior is for all drives to show up in either controller's BIOS extension. However, a couple of things have been known to wreak havoc on the boot ROMs:

• Mismatched firmware versions seem to lead to conflicts that sometimes even prevent the system from booting successfully (removing the option ROMs from the BIOS' list should avoid this problem long enough for it to be corrected)
• The SAS 2008 and SAS 2308 seem to not be entirely interchangeable in this regard. I wouldn't exclude the possibility of the SAS 2308 knowing how to enumerate an SAS 2008, but the SAS 2008 not being aware of SAS 2308 controllers (if true, this may have been fixed in newer firmwares, assuming there's no problem on the PCI-e side).
• P16 has a bug that often (always?) leads to drives not showing up during the boot ROM prompt - but they do show up after entering the option ROM prompt.
• The UEFI implementation may be implementing some kind of sandboxing for BIOS-emulated option ROMs, which may hinder proper enumeration of other cards.
• The presence of two (loaded) option ROMs may lead to conflicts caused by the attempt to hook the same interrupt in two different ROMs.

In short, if you really want the option ROM (even though its only function in IT mode is somewhat broken in P16), try to flash only the SAS 2308 on the motherboard with the boot ROM, leaving the M1015's SAS 2008 empty.

First of all, thanks for the answer. I can downgrade the fw no problem about that, but I want to know why in more detail about the problems that can occur.
In the future versions of Freenas will you plan to upgrade the driver version from p16 to p19 ? Please tell me where can i find some more specific information about the problems that can occur due to driver and fw version differences.

Why? That's a very good question for which nobody has an answer.

LSI's official stance is that nothing changed between P16 and P20 in IT mode and that newer firmware versions should be backward-compatible with older drivers.
However, we have seen a gradient of nastiness from P17 to P20. People have weird issues with their controllers which are magically fixed when they flash P16 on their controllers. These range from annoying interface errors on P17 to outright sudden and complete pool loss (presumably due to several disks being disconnected at once for no reason) on P20.

The details have been discussed in a number of different threads around here over the last few months.

As for future plans, let's say it's complicated.

The FreeBSD P16 driver has several community-made fixes which have not been merged upstream by LSI. The FreeBSD stance is that this version of P16 is better than LSI's P17-P20, which do not include the fixes and, on paper, add absolutely nothing for IT mode.
Furthermore, LSI is aware that FreeBSD uses a modified P16 driver and their engineers seem comfortable with that.

 

[cyberjock]

The long answer is this.

LSI releases a driver and firmware as a matched set. That is the only combination that LSI tests. So anything else is untested and comes with zero warranty and no guarantee you won't suddenly have major problems up to and including data loss.

If you called LSI to troubleshoot a problem they will ask for your driver and firmware version. If they don't match LSI won't even talk to you further until they do match.

Internally, the driver talks to the controller with a "language" that is tested and verified to work properly between the driver and firmware. As firmware is updated over time, the "language" may change and there may be problems that were never tested, expected, or obviously desired. The driver talks to the controller to issue commands and read or write data. So long as the code the driver needs to use on the firmware and the firmware for those codes don't change, everything is fine. So some combinations have not been "shown" to be a problem (such as v16 driver and v19 firmware) but that's not to say that someday you'll hit some edge case or a FreeNAS update will bump into some problem because the software issues some command that has never been used.

In short, it's pretty dangerous to mismatch the firmware and driver. It has cost quite a few people their data, and they often have no idea what is coming until months later. Nobody (not LSI, not iXsystems, no programmer anywhere) has verified that your mismatched firmware and driver will work properly for all cases, and assuming that since it has worked for the last X weeks/months is NOT good validation that you should keep the mismatch you have. All you have to do is hit that edge case once and you might lose it all. It is not "cool" to go to sleep at night and wake up the next morning to find your data unrecoverable.

So match them up properly at your own risk/reward. I immediately ignore threads and users that have mismatched drivers and firmware because it's not supported in the slightest. If a "WARNING" in the WebGUI isn't enough of a clue to that "you gotta fix this" I don't know why I should spend time helping you with your problem, whether related to the LSI controller or not. You ignore warnings. Is that *ever* a good idea? Is it worth anyone's time in this forum as an unpaid volunteer to help someone that has deliberately put themselves in a situation we've intended to avoid by adding a warning?

Hope this clears things up and people will *finally* give this up and just do what the WebGUi says. We thought very hard and long about this and we put that warning in starting with 9.3 BECAUSE of the abundance of people that were actually losing their data and didn't know better.

 

[Radu]

OK, fair enough, thank you for the answers will downgrade to p16. Just another small question ... i'm wandering if TrueNAS uses p16 for lsi hba adapters ...

 

[cyberjock]

OK, fair enough, thank you for the answers will downgrade to p16. Just another small question ... i'm wandering if TrueNAS uses p16 for lsi hba adapters ...

It does. But the hardware is strictly controlled (and updated) by iXsystems.

 

[Radu]

OK, downgraded from p19 to p16. One question though, i could not find the p16.00 for the LSI SAS 2308 that i have on my SM MB. On SM ftp there is only p16.00.01 IT mode; on LSI website i was unable to find model 2308 ...
For the M1015 all OK. See attached picture!

Do you have any idea what fw exactly can i use for 2308 SAS controller?

No more alerts :D

 

[Ericloewe]

The one on Supermicro's FTP server is almost certainly the right one.

 

[Jailer]

The one on Supermicro's FTP server is almost certainly the right one.

It is. That's where I got the firmware I'm using on the board in my sig.