Resource icon

What's all the noise about HBA's, and why can't I use a RAID controller?

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
I noticed that Supermicro is shipping the SAS 9500 / SAS3808 in some new SuperStorage units, but I can't seem to find it in the FreeBSD mpr driver. Can anyone shed some light on this? Is it not yet supported, not documented...?
A long time later, I have a Supermicro SuperStorage with an SAS3808 controller metaphorically in front of me. The mpr driver in 13 sees the card just fine, but these things no longer work with sas3flash, only storcli (yuck).

I can't for the life of me get storcli to flash the latest P27 firmware, from the factory P23. I've tried:
  • TrueNAS Core's shipped, older version of storcli [Unspecified error]
  • The latest version of storcli straight from Broadcom [Firmware image hash mismatch]
  • P26 firmware, just in case the P27 file is corrupted on Broadcom's website [Firmware image hash mismatch]
  • The EFI version of storcli [Firmware image hash mismatch]
For now, I'm going to test things out with P23 and contact Supermicro in the meantime to see what they say about this. They don't have any firmware images published in their product page for the AOC-S3808, but I suspect some vendor limitations to the firmware.
 
Joined
Jul 3, 2015
Messages
926
This is very interesting. Looking forward to hearing the outcome. Thanks for sharing.
 
Joined
Dec 29, 2014
Messages
1,135
I don't know if it applies to you, but I had one of my systems (Cisco UCS) that wouldn't work correctly when trying to flash for OS level tools like sas3flash. It would work if I used the UEFI tools.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
Tried the UEFI version, same effect with a slightly different error message. I'm fairly confident at this point that it's not a tool or driver thing.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
A long time later, I have a Supermicro SuperStorage with an SAS3808 controller metaphorically in front of me. The mpr driver in 13 sees the card just fine, but these things no longer work with sas3flash, only storcli (yuck).

I can't for the life of me get storcli to flash the latest P27 firmware, from the factory P23. I've tried:
  • TrueNAS Core's shipped, older version of storcli [Unspecified error]
  • The latest version of storcli straight from Broadcom [Firmware image hash mismatch]
  • P26 firmware, just in case the P27 file is corrupted on Broadcom's website [Firmware image hash mismatch]
  • The EFI version of storcli [Firmware image hash mismatch]
For now, I'm going to test things out with P23 and contact Supermicro in the meantime to see what they say about this. They don't have any firmware images published in their product page for the AOC-S3808, but I suspect some vendor limitations to the firmware.
Digging this up. Life got in the way (by that I mean that the server is really weird about half the DIMM channels and drops them at firmware updates and possibly other situations) and I only emailed Supermicro support yesterday. Got a reply overnight pointing me to an arcane corner of their servers where P25 is available (Broadcom released P28 earlier this week) and the general vibe that, indeed, the cards don't just take Broadcom firmware. Crossflashing may be possible, but that's not a can of worms I want to test right now.

In an astonishing coincidence, a user over at the STH forums provided a link:
 

JeffNAS

Dabbler
Joined
Oct 18, 2023
Messages
19
Does anyone know of a reliable HBA for SAS 12gbps for the lastest version of TrueNas SCALE?
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
LSI SAS 9300-8i, or probably any other LSI 3008 based controller. As long as it has the correct IT firmware flashed. See the HBA resource for a link.
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
FWIW, I too had a lot of fun with ATAPI, SATA port multipliers, and SATA HBAs, etc on MacOS. Put five dvd Drives into an enclosure with an aim to rip my CD and DVD collection.

The first attempt was via a USB/SATA port multiplier. Worked ok sometimes but invariably would eventually lead to the whole array going non-responsive. Then all rips would terminate. Very annoying.

Then tried using SATA HBA cards in an external thunderbolt enclosure and discovered that the ones I bought didn’t feature ATAPI support. The lack of documentation didn’t help.

Finally gave up on all that and bought one datoptic eS3U31 stick per DVD drive, ran a short 12” eSATA
-SATA cable from each dongle to a DVD drive and then consolidated all the dongles in a USB 3 hub. This setup prevents the spread of nonsense and allows individual drives’ USB connection to be reinitialized easily, if necessary.

Ripped my collection and now the array is mostly dormant.
 

dlp1950

Dabbler
Joined
Mar 4, 2018
Messages
17
I am fairly new to some of this and have just purchase a used Dell PowerEdge T330, with an UCSA-901 H330 Raid controller. Of course it won't find or read the drives, and I had read quite a bit about it (articles by "jgreco" were quite helpful) so I just ordered a LSI SAS9300-8i 12Gbps, before I realized that it said unRAID. So I assume it will be HBA, but I am confused will TrueNAS still set up the RAID Pools? Any help is greatly appreciated
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
I’d verify that it’s been flashed to IT mode, which I believe is synonymous with UnRAID or TrueNAS/FreeNAS, etc.

Just look up the flashing instructions in the resources section and probe the HBA from the command line re: what firmware it is using and whether said firmware is the latest IT version available. Regardless of vendor, it is always a good idea to verify same, just as it’s good practice to pre-qualify HDDs before adding them to a working pool.
 

latexyankee

Cadet
Joined
Feb 2, 2024
Messages
5
Calling all cisco users...if possible @Elliot Dierksen

Apologies if this is against the rules or I'm overstepping here, I have an issue and this is the only place I can get a correct answer. I'm brand new to these forums as this is my 3rd post. The other 2 on the same topic but an older thread I feel may be forgotten, I don't know for sure.

I've done my research. Its been about 10 days-2 weeks now. I've looked at what should be compatible with my hardware. I'm still left with a question, cisco specific. I will use this thread for reference:


I have ordered UCSC-SAS12GHBA for my C240 M4 server. For all wondering this SHOULD be a cisco stamped LSI 9300 8i. The reason I went this route is because of the thread above AND the fact that while an LSI card is all but guaranteed to work, these cisco servers seem to be very picky about hardware that is not "cisco" branded and can ramp the fans to 100% without the ability to alter the fan curve/profile. From everything I've read the past few weeks if you can get a cisco part to work for you, SPECIFICALLY a part listed in their spec sheet (which this is)...then that is your best bet.

Now the the cisco way to upgrade FW is through the HUU. I'm confident the card will work based on @Elliot Dierksen experience and post history. My worries are with the IT mode requirement. This will be my first attempt at TN, I'm replacing the stock raid card which DOES have jbod mode but @Elliot Dierksen recommends to use the HBA as the raid card can still cause future issues. Does anyone know if these cards are flashed to IT mode by the cisco HUU itself or is that a separate process. Of course cisco recommends you do not upgrade this cards FW stand alone as it is specifically configured to work on UCS hardware but cisco says that about literally EVERYTHING. There's the right way and the Cisco way.

So I'm wondering how I can determine the card is in IT mode, if not - is that still needed with this particular card (I assume yes) and how I could accomplish this? By using the public standard methods or an alternative? Cisco makes everything a few steps more difficult than you assume lol.

Again I understand a genuine or other 3rd party LSI card would suffice but it seems silly NOT use cisco parts when these servers are by far the most finicky, Dell seems hardware agnostic.

Mods. I do not have sufficient privileges to post in the hardware space, I understand this and I'm glad we don't have the same repetitive posts clogging up the boards. If you decline this post can you place it in the appropriate sub forum? I have searched this topic for 2 days and I'm certain the only answer I'm going to get is on these boards.

Thank you for the wealth of information I've digested over the past week. I'd be completely lost without it. Any insight or experiences are GREATLY appreciated.
 
Joined
Dec 29, 2014
Messages
1,135
I am repeating a little from my reply to your other post. I would stay with the Cisco HBA modules since they fit in dedicated slots and use the custom cabling that comes with the servers. You flash them from the regular HUU. The HBA modules are IT mode only, so no worries about cross flashing. You can use other parts in the Cisco servers, but the CIMC may run the fan up higher if it is a card it doesn't recognize. I tried to stay with the Cisco cards where possible. The exception in my FreeNAS was using Chelsio NIC's for the storage network. I started with a T520 for 10G, and then upgraded to a T580 when I increased my storage network to 40G. Both worked quite well. There are several articles on tuning for higher speed networks that are really important to read, particularly the primer by jgreco. If you are doing NFS which will enable synch writes, you need a high speed SLOG to get good write performance. Lots of RAM is the biggest thing I have found for good read performance. You can check out the specs on my servers in my signature.
 

latexyankee

Cadet
Joined
Feb 2, 2024
Messages
5
I am repeating a little from my reply to your other post. I would stay with the Cisco HBA modules since they fit in dedicated slots and use the custom cabling that comes with the servers. You flash them from the regular HUU. The HBA modules are IT mode only, so no worries about cross flashing. You can use other parts in the Cisco servers, but the CIMC may run the fan up higher if it is a card it doesn't recognize. I tried to stay with the Cisco cards where possible. The exception in my FreeNAS was using Chelsio NIC's for the storage network. I started with a T520 for 10G, and then upgraded to a T580 when I increased my storage network to 40G. Both worked quite well. There are several articles on tuning for higher speed networks that are really important to read, particularly the primer by jgreco. If you are doing NFS which will enable synch writes, you need a high speed SLOG to get good write performance. Lots of RAM is the biggest thing I have found for good read performance. You can check out the specs on my servers in my signature.
PERFECT, this is great news. the cisco cards are cheaper as well since they're not in demand.
I did grab an mlom 10GB, it was $10 we will see if it plays nice, its the cisco part from the spec sheet as well.

Well awesome thanks for replying, if I have any issues I'll update accordingly.
 
Joined
Dec 29, 2014
Messages
1,135
PERFECT, this is great news. the cisco cards are cheaper as well since they're not in demand.
I did grab an mlom 10GB, it was $10 we will see if it plays nice, its the cisco part from the spec sheet as well.

Well awesome thanks for replying, if I have any issues I'll update accordingly.
You are welcome. I was never able to get any of the converged NIC's to work with Core. Perhaps they will work with Scale. I have used Chelsio and Intel non-converged NIC's with good results.
 
Top