Driver for new LSI card

[Alfred Melvin]

Am I to use tunable section to preload drivers for new LSI card? and if so can I use a USB flash as storage for drivers?

 

[jgreco]

The 9201-16i is based on the LSI SAS2116 (Fusion MPT 2.0).

The question that needs to be asked is what driver is appropriate for the card. We have both SAS2208 and SAS2008 controllers here, and I don't know where yours falls.

The SAS2008 (commonly loved here as IBM M1015's crossflashed to LSI 9220-8i mode) uses the FreeBSD mps* driver, and if set up in IT mode, devices are likely to just magically appear in the FreeBSD CAM device list ("camcontrol devlist"). This is by far the best and happiest way to go.

The SAS2208 uses the FreeBSD mfi* driver, which is a steaming crock, not because of the driver (thanks Scott!), but because LSI's design appears to force all access to devices to go through their logical drive abstraction layer, which also appears to mean that not only do you have to configure all your drives on the controller (via BIOS or CLI), but also that you won't be getting SMART or other data, at least not directly. On the positive side, FreeNAS does have "mfiutil" and it's a fsck of a lot nicer than LSI's BIOS version of it.

Both these drivers are included in FreeNAS and work out-of-the-box. My suspicion is that you're saddled with the mfi driver and that you haven't created any logical drives yet, and as a result FreeNAS will behave as though there are no drives attached to the system because it doesn't see them under the standard paradigm. If you can get to a FreeNAS command prompt and do "mfiutil show drives" and you see a bunch of "UNCONFIGURED GOOD" drives, that's the problem.

 

[cyberjock]

Ouch. If you can't get SMART data that's almost a guaranteed recipe for failure later. How sucky!

 

[jgreco]

That might be a bit of an overreaction. The LSI controllers are generally well-respected in the industry, something that wouldn't be true if they were missing a feature that made it a "guaranteed recipe for failure later." The management tools range from annoying-but-adequate to fairly nice, though they tend to hide the low level details. However, the controllers are designed such that tools on the server (in the manner of ESXi) or by a MegaRAID Storage Manager instance (most other platforms) are expected to be running, and it appears as though neither case is true for FreeNAS, so you'll miss out on being able to be aware of SMART errors that might predict a failure, but the controller is also monitoring drive performance and can predict failures - possibly in slightly different ways, though. I'm pretty sure FreeNAS will still notice if one if its drives drops out, and the tools to cope with that appear to be quite sufficient.

It does mean you're paddling against the current as far as the FreeNAS design goes, but it ought to be workable.

 

[cyberjock]

But wouldn't all of that mean:

1. No SMART emails when you have a problem?
2. None of the SMART monitoring in FreeNAS GUI works?
3. Running smarctl from the command line won't provide the SMART data?

How do you best mitigate these losses? For me, all 3 of those are amazing tools and I'm disappointed as it is that I can't use those with my Highpoint controller. That's why I'm not using it despite having 3 of them...

 

[Alfred Melvin]

My question is where and how I should install the FreeBSD-P15 driver for this card. This is the first time I've used anything other than motherboard SATA ports
and was not sure how to proceed with adding this card. My system is running fine with the WD drives and I was exited about adding the SSD's and Seagate's listed in my signature.

 

[jgreco]

But wouldn't all of that mean:

1. No SMART emails when you have a problem?
2. None of the SMART monitoring in FreeNAS GUI works?
3. Running smarctl from the command line won't provide the SMART data?

How do you best mitigate these losses? For me, all 3 of those are amazing tools and I'm disappointed as it is that I can't use those with my Highpoint controller. That's why I'm not using it despite having 3 of them...

The obvious comeback is "what makes SMART so special?" Since you're actively bashing Seagate's implementation of it in another thread if I recall correctly. ;-)

Seriously, SMART is pretty DUMB. However, it is a cross-vendor feature, and that does mean that it is likely to be easier for something like FreeNAS to support it than for FreeNAS to support whatever arcana is required for $RANDOM_SUCKY_HBA_BEING_DISCUSSED (in LSI's case, a command line utility and other fun).

- - - Updated - - -

My question is where and how I should install the FreeBSD-P15 driver for this card. This is the first time I've used anything other than motherboard SATA ports
and was not sure how to proceed with adding this card. My system is running fine with the WD drives and I was exited about adding the SSD's and Seagate's listed in my signature.

Yeah, I don't actually know. Is it a driver you acquired from LSI? I may have similar hardware sitting around and may even be willing to experiment a bit, but you'll need to help by providing more info.

 

[Alfred Melvin]

I wasn't bashing Seagate as much as supporting the fact that I've had no problems with WD. The Seagate 2TB drives were on sale for $94 at local Best Buy. 12TB for under $600!!
I'm still reading and hope to figure out how to do this.

 

[cyberjock]

The obvious comeback is "what makes SMART so special?" Since you're actively bashing Seagate's implementation of it in another thread if I recall correctly. ;-)

Seriously, SMART is pretty DUMB. However, it is a cross-vendor feature, and that does mean that it is likely to be easier for something like FreeNAS to support it than for FreeNAS to support whatever arcana is required for $RANDOM_SUCKY_HBA_BEING_DISCUSSED (in LSI's case, a command line utility and other fun).

I agree that SMART is pretty dumb. But it's really the only way to have any clue what the hard drive is doing "health-wise" aside from just observing it for the click-of-death or other bad sounds and using technical terms like "it sounds bad".

 

[Alfred Melvin]

UMMM!! I bought a HBA card from LSI. Wouldn't it come as an IT firmware card?
How do I view the firmware and/or drivers from FreeNAS console?

 

[jgreco]

Ok, productive or at least reasonable question. The answer is that LSI's cards are driven by several different drivers and that similar chipsets can support different features. IBM's M1015 is an extremely popular card with the FreeNAS community. IBM ships it with one kind of firmware and it is popularly reloaded with a different kind of firmware because the chipset apparently can support either. But you have one of the higher end cards I believe, which makes me guess that it is likely to resemble our 2208 here.

As you're booting, look for the LSI message. This is what our LSI MegaRAID 2208 (Supermicro onboard) controller looks like.



Notice it says "SAS-MFI BIOS" and talks about "WebBIOS". If you hit control-H to go into the WebBIOS, it'll give you tools to configure your controller and the attached drives. They try to make it look Windows-y and while it's possible to control from the keyboard, it helps to have a mouse. The controller selection screen there will tell you the make and firmware of your controller.



The "-0015" means it is P15 firmware. Going in, you then get the logical view of your storage.



The LSI intention is for individual drives to be aggregated into virtual drives, such as that virtual drive 0 which is a RAID1 of two SSD's, which is a common config here for our ESXi hosts. But that's not really what you want for FreeNAS. If you just plug drives into the LSI, they'll show up as "Unconfigured Good" drives:



so at this point you know the controller and the drives are talking. So the question becomes, what then. And my guess is this is where you are stuck. Apparently it is possible to configure the LSI controllers to pass through unconfigured-good drives to the underlying OS but they don't do it by default, and I don't see an immediately obvious way to set that. And it is more difficult because the disks don't show up in "camcontrol devlist". A "dmesg | grep mfi" from the FreeNAS CLI yields



which shows the RAID1 virtual device happily showing up as mfid0, but no mfid1. You can do a "mfiutil show drives" to see attached drives and "mfiutil show volumes" to show the available volumes. So what you probably want to do is do "mfiutil show drives" and then take note of the number (first column) of each "UNCONFIGURED GOOD" drive, then run "mfiutil create jbod NUMBER" for each of those numbers. Be warned that creating JBOD is almost certainly guaranteed to be overwriting what is on the disk. I don't think the MegaRAID Firmware Interface system has a way to cope with acting as a dumb SATA/SAS controller, so you will not be able to migrate disks back and forth from an LSI MFI controller to other random types of controller.

Okay, so try that, and let us know, eh.

For completeness, though, here's what the IBM M1015, crossflashed to LSI SAS2008 ("pretend to be a SAS9211-8i") mode, and operating under the LSI MPT firmware and FreeBSD mps0 driver looks like.

BIOS probe:



You can hit control-C during that to get to the card configuration utility (yay for consistency).



Shows you the adapter type and firmware revision. Note firmware version 15 and IT mode. You can look under SAS Topology at your attached disks, but really there's not a whole lot to be seen here in IT mode.



Within FreeBSD, that'll probe as a controller serviced by the mps driver, and the drives will appear as normal "daX" devices and appear in "camcontrol devlist", so pretty straightforward pleasure there. A "dmesg | grep mps" would show something like:



Oh, and just to muddy the waters further, there's actually a FreeBSD mpt driver, but it is used for older SAS controllers like the IBM BR10i. Bleh

 

[Alfred Melvin]

Finally got all the components made/bought and working. Custom power supply hook up (17 HDD's ) was time consuming to solder and arrange.
Saw the post about LSI cards you did. My new questions is if the Freenas system sees all 17 drives do I still need to configure the card with the "WebBios" or
can I proceed with the Freenas setup of my drives?

 

[jgreco]

Check to see if you're seeing raw disk devices under "camcontrol devlist". If so, you should be good to go. If you're seeing "LSI Virtual Device" or something like that, then bad.