LSI 9211 on FreNAS 9.10

[andyiam]

From all of the searching I've done I know this has probably been asked before, but I couldn't find anything with the search tool.

I just rebuilt my NAS with upgraded hardware and in doing so bought an LSI 9211 HBA. It's in IT mode on P17 firmware.

This is my first time using an HBA so please forgive my ignorance. I had assumed that FreeNAS would recognize it automatically, but this doesn't seem to be the case. When I go to list disks, nothing is shown. I'm guessing I need to install or enable the driver for it, is that correct? Does anyone have any documentation or a guide I could read through? I really appreciate any help with this.

 

[Jailer]

Try flashing the card to P20 IT mode and try again. It should be showing your disks with a mismatched firmware/driver but rule that out before you proceed further.

 

[Mirfster]

This is my first time using an HBA so please forgive my ignorance.

How are the drives connected?
What are your system specs?
If you are connecting the drives directly to the HBA are they connected via a Forward or Reverse Breakout Cable?
If the card has the BIOS installed, does it show the drives being detected when the system is powering on?

 

[andyiam]

How are the drives connected?
What are your system specs?
If you are connecting the drives directly to the HBA are they connected via a Forward or Reverse Breakout Cable?
If the card has the BIOS installed, does it show the drives being detected when the system is powering on?

I have a U-NAS 8 bay case. The drives are connected via 2 breakout cables that go from the 2 SAS ports on the 9211 to the 8 ports on the hot swap bays. I have:

• ASROCK C236 WSI Mini ITX Motherboard
• Intel i3 6100 3M
• 16GB DR4 ECC,

That is something I've been perplexed with. Sometimes when I boot the LSI bios will take over, other time sit will go straight to the FreeNAS bootloader. Can't figure out the reason. When I do get into the LSI BIOS the drives show up.

 

[Stux]

The latter point is good news. Suggest updating to p20 firmware and we'll take it from there ;)

 

[andyiam]

This is odd. When I try to use the sas2flash utility it says no LSI SAS adapters were found.

Any idea what could be causing that? I downloaded the utilities and firmware files and booted to the native uefi shell the motherboard provides.

 

[Mirfster]

Doesn't make sense. Either the card is not seated properly or it died?

 

[andyiam]

I bought the card off ebay. Wondering if I got hosed. Would this explain the seeming 50/50 split of booting to FreeNAS bootloader vs the LSI BIOS?

If I have to trash this 9211 should I look for another or get a different HBA?

 

[Mirfster]

Normally if the LSI has the BIOS installed, then you would see that before FreeNAS starts. It would simply show you what drives were detected. Some choose not to have the BIOS to speed up the boot process, but I prefer to leave it so I have another way to check.

Not saying you have to trash it, but if it is only showing half the time, then that card may be going bad. Nothing wrong with using a LSI 9211-8i, it is actually one of the highest (if not most) recommended HBA. An IBM M1015, Perc H200 or Perc H310 would work as well. They would need to be cross-flashed to actually be an LSI 9211-8i. ;)

 

[Stux]

This is odd. When I try to use the sas2flash utility it says no LSI SAS adapters were found.

Any idea what could be causing that? I downloaded the utilities and firmware files and booted to the native uefi shell the motherboard provides.

Is it an LSI retail card? Or is it an IBM M1015/M1115 etc?

What does it say in the BIOS option rom boot screen?

This error happens when you try to use sas2flash on an IBM M1015/M1115 which hasn't had its mind wiped with megarec yet. To fix it on the ServeRAID boards... which are sometimes sold as LSI 9211-8i cards, when technically, they aren't... yet... you run some nifty scripts... and then they are.

 

[andyiam]

I pulled the card and took a few pictures. I do have it connected via a ribbon cable that came with the case. I've never had an issue with one before, but could be a failure point. I wouldn't expect something so basic to be faulty though. I'm going to put the card back and will report with what the bios says.

 

[Mirfster]

I'm going to put the card back and will report with what the bios says.

Make sure it is fully seated.

The drives are connected via 2 breakout cables that go from the 2 SAS ports on the 9211 to the 8 ports on the hot swap bays.

Just to be clear, have you ever seen any of the drives appear (either in the BIOS or in FreeNAS) since this is a fresh rebuild using the LSI card?
I understand that you are using a breakout cable, but if you have never seen the drives show up I would wonder if you are using a Reverse Breakout Cable or a Forward Breakout Cable?

Also, if you have a different machine you can plug the card in there and see if is detected. This may help to see if perhaps you have a bad PCIe slot...

 

[Mirfster]

Just for clarity, since you said that the drives are connected to each port I am assuming that you do not have a SAS Expander. Instead it is a one-to-one type connection. In that event you would want a "Forward" Breakout Cable. If you do have a SAS Expander, then it would be a "Reverse" Breakout.

If this is your case, I am assuming you need a "Forward" Breakout. Even though it states:

Hot-Swappable SATA/SAS Backplane, Expansion room for 1x PCI/PCI-E

I am under the impression that it is still a one-to-one connection and not a SAS Expander. If I am incorrect in this, please feel free to let me know.

 

[andyiam]

It's definitely seated well. I made sure of that when I put the card back in. I have seen the drives appear. I took pictures of the boot process. It takes around 10 minutes for the BIOS process to complete, around 10 minutes. Is that normal?

First, it shows the copyright information about a minute later it says initializing. After another 2-3 minutes it says searching for devices on HBA0. Finally about 5 minutes after that it shows the drives for a few seconds and then tried to boot to one of them. But of course it can't and I just get the message that says this is a ZFS data disk. System halted.

And yes, it is one to one. Here are some pictures of the backplane. Each drive bay has it's own port.

 

[Ericloewe]

I pulled the card and took a few pictures. I do have it connected via a ribbon cable that came with the case. I've never had an issue with one before, but could be a failure point. I wouldn't expect something so basic to be faulty though. I'm going to put the card back and will report with what the bios says.

Nothing screams fake, however:

I do have it connected via a ribbon cable that came with the case.

These are very often flaky.

 

[Mirfster]

It takes around 10 minutes for the BIOS process to complete, around 10 minutes. Is that normal?

No

Finally about 5 minutes after that it shows the drives for a few seconds and then tried to boot to one of them. But of course it can't and I just get the message that says this is a ZFS data disk. System halted.

Need to check your BIOS and set the proper Boot Device order. Assuming you have FreeNAS installed on a USB, you would set that as the first device.

 

[Spearfoot]

Agree with @Mirfster that 10 minutes is way too long for the HBA to initialize, and you need to configure your system's BIOS settings so that it boots from the USB flash drive.

Your photographs also show two things:

1. You're running the older P19 version firmware; you need to update this to P20.
2. You're running 4 of the Seagate ST3000DM001 hard drives; these are notorious for failing and you need to test them thoroughly before committing any important data to them. In fact, I would go so far as to agree with the good @DrKK and suggest you replace them immediately.

 

[andyiam]

Thanks for the tip about the hard drives. They are definitely flaky. I got them for my original NAS back in 2012. They've been giving me trouble ever since and I've already replaced 2 as you can see. My goal is to get this back up and running and then replace them as soon as financially possible with more reliable SATA drives or ideally SAS drives.

I reseated the HBA again directly into the SATA port without the ribbon cable, same thing. I'm going to put it into my desktop after dinner and see if it still takes as much time and if I can update the firmware. I'm worried this is pointing towards a dead HBA. But it's odd to me that it seems to function, just extremely slowly.

For my reference, how long should it take to initiallize? I was thinking it'd be a matter of seconds.

 

[Ericloewe]

I was thinking it'd be a matter of seconds.

Yeah.

 

[andyiam]

Not sure how or why but now the disks are showing up in FreeNAS... This is after I tried it in my desktop. It got through the initialization process much faster. I'm very uneasy about this HBA with the issues I've had.

I'm going to import the volume and see what happens. Hopefully enough of the disks are alive to save it.

On the console I'm seeing this error:

Code:

mps0: warning io_cmds_active is out of sync - resyncing to 0

What does that mean?
Also, Another one that is coming up.

Code:

msp0: ATA ID command timeout cm 0xffffffe00009a76c0
mpssas_get_sata_identify: request for page completed with error 0mps0: sleeping 3 seconds after SATA ID error to wait for spinup