DS4246 with Oracle 375-3609-03 21 of 24 drives detected

[boggie1688]

I have a Netapp DS426 JBOD connected via an Oracle 75-3609-03 8 Port 6 Gb/s SAS HBA

I recently installed 4x SSDs and only 1 of the trays had its LED light up. Out of chance, I pulled a drive out of one pool, and suddenly another tray LED lit up for one of the new SSDs. I disconnected an entire pool of 8x HDDs, and all the remaining SSD trays lit up.

Slowly I reconnected all 8x HDDs, and the last 3x HDDs would not light up and could not be detected.

I'm not a HBA expert, and I can't seem to find anything on this HBA. Is this an HBA issue?

 

[Arwen]

Could the part number have been 375-3609-03?

That does translate to an Oracle / Sun part number for an 8 port external 6 Gb/s SAS HBA.

I checked Oracle / Sun's system handbook for the card, and I did not find any drive limitation. It does specify that booting is not supported, (which is expected for this type of external card).

What is the total number of drives you can have activate at once?

Some LSI cards can have limited drive support, based on licensing to reduce cost. Sounds like you have such, though I don't see that in the documentation. But, I do see hints that this is not an LSI card...

 

[boggie1688]

Could the part number have been 375-3609-03?

That does translate to an Oracle / Sun part number for an 8 port external 6 Gb/s SAS HBA.

I checked Oracle / Sun's system handbook for the card, and I did not find any drive limitation. It does specify that booting is not supported, (which is expected for this type of external card).

What is the total number of drives you can have activate at once?

Some LSI cards can have limited drive support, based on licensing to reduce cost. Sounds like you have such, though I don't see that in the documentation. But, I do see hints that this is not an LSI card...

You are right:
375-3609-03-52

I tried:
sas2flash -list

Which returned:
NO LSI SAS adapters found!
Limited command set available!
ERROR: Command not allowed with an adapter!
ERROR : Couldn't create command -list
Exiting program.

I spoke with the dealer I purchased the entire bundle from, and he thinks maybe I should plug in every PSU. Thus far, I've had all PSUs installed but only powered the JBOD with one.

Also quick correction: 20 drives at once seems to be the limit.

 

[Arwen]

I have no further suggestions. That card seems limited to 20 drives at once. Don't know why, don't know if that can be changed.

 

[boggie1688]

I have no further suggestions. That card seems limited to 20 drives at once. Don't know why, don't know if that can be changed.

Thanks for replying and at least trying.

 

[jgreco]

HBA

That's the first problem here, this is a RAID card, not an HBA. and it's also the second problem, because it's not an LSI HBA, but rather I believe some awful PMC based card.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

It would be interesting if you could get the dmesg boot info to better identify what this actually is, since it seems to be somewhat recognized.

 

[boggie1688]

That's the first problem here, this is a RAID card, not an HBA. and it's also the second problem, because it's not an LSI HBA, but rather I believe some awful PMC based card.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

It would be interesting if you could get the dmesg boot info to better identify what this actually is, since it seems to be somewhat recognized.

I think you are spot on.

I get a lot lines in the log that refer to pmx80xx0: : mpi_ssp_completion. Googling pmx80xx0 driver, yields results talking about PMC based cards.

If I do a quick google search for PM8001 I can find documentation that suggests this controller is good for 128 drives.

Either way, I should grab a LSI card. Would this be a good replacement card?

LSI 9286CV-8e (9207-8e) HBA SAS2308 PCI-E 3.0 w/BOTH BRACKETS ZFS UNRAID FREENAS | eBay

HELLO, UP FOR SALE IS ALSI 9207-8E (LSI 9286CV-8E) HBA CARD, FLASHED TO THE LATEST9207-8E IT MODE FIRMWARE (P20).

www.ebay.com

 

[jgreco]

Either way, I should grab a LSI card.

Correct.

Would this be a good replacement card?

Um, not certain. The LSI 9286CV-8i is a 2208-based full RAID controller. Running its native firmware, it would be a terrible choice. However, in the last year or two, a glut of LSI 6Gbps RAID controllers has entered the used market. VMware deprecated compatibility with older controllers and over a period of a year or two, companies have done (often out-of-cycle) hardware refreshes because the stuff they bought just two years ago is not compatible. This pushed a metric crapton of high-dollar LSI RAID controllers with all the features, such as "CV" which is CacheVault, onto a market where no one wanted them -- the ESXi hobbyists don't want them for the same reason they were retired by their previous owners, and Proxmox/etc don't need RAID controllers for the same reason TrueNAS doesn't. So immensely little residual value.

But! The thing to understand is that these are basically just little PowerPC systems-on-a-chip and the high end cards can theoretically run the code of the low-end cards with just minor tweaking to enable that. This is not trivial, but LSI already had given the world an example of this because downgrading the 9240 to a 9211 was already officially supported. Some smart firmware hackers took that and used it to handle other OEM controllers such as the PERC H200/H310, the ServeRAID M1015, etc., often using the original LSI toolsets.

Then recently someone figured out how to do this for a PERC H710 (high-ish end RAID), lobotomizing the thing into a 9207-8i.

So we know that it is theoretically possible to bludgeon the 9287CV into a 9207. I just wasn't aware of anyone having done it, much less selling them in quantity on eBay. But it is logical that someone who maybe obtained a large batch of these cards for $20/ea might find it profitable and fun to do this.

 

[boggie1688]

Better just get one that is LSI SAS 9207-8e vs one flashed down into one.