9211-8i - Single Drive shown. Hangs sometimes on Boot.

[Salzig]

Hej all,
thanks for reading my post.

I'm absolute new to FreeNAS/TrueNAS, and maybe got a bit overcommited by building a custom system (see Spoiler in Signature).

Sadly i have a problem with the "LSI MegaRAID SAS 9211-8i (LSI00194)" card i got, which is suggested/recommended (?). Sometimes TrueNAS boots fine, but only a single of four installed WD drives is shown (all WD-Drives are attached to the LSI-Card, SSD are attached to the mainboard as boot, NVMe sofar unused).

But sometimes the board has problem loading the LSI-Card with (typical on cold boot):

Sometimes it hangs with:

Does anyone of you know about this behaviours? Is the card faulty? FW update needed?

Sadly i feel a bit lost about it, cause it's the first time in quite a while i build a system by myself.

 

[Salzig]

oh, and i'm using a "Mini SAS 36Pin (SFF-8087) Male to 4 SATA 7Pin female Cable, Mini SAS Host/Controller to 4 SATA Target/Backplane, 0.5M" cable to connect the four WD-drives.

 

[Salzig]

Bump. Some help/hint would be nice. I've really no clue.

 

[NugentS]

Try reflashing the card - that first error doesn't look good

 

[Ericloewe]

Ok, a few things here:

1. If the extension ROM intermittently fails, that doesn't bode well for the health of the card. At the very least, the card's firmware storage is in doubt.
2. Reprogramming the card may get a marginal cell back in shape, but see 1. above.
3. If you get lucky and the damage is restricted to addresses used by the extension ROMs, you can just remove them, assuming you don't need to boot from the card.
4. The missing disks could easily be due to damage to the controller, or just a bad cable. Hard to tell without further debugging.
5. If things aren't looking good for that card, it may simply be overheating. Make sure it gets a decent amount of airflow, replace the thermal compound with fresh one and maybe add a 40 mm fan to the card.

 

[Salzig]

Uhm, would be kind of strange for a new card i got on for ~100€? But i'll try flashing then.

 

[Ericloewe]

Well, where did you get it from? LSI SAS2 stuff has been out of production for a while now. In addition, there have been plenty of fakes and just plain defective cards in the LSI SAS HBA market.

 

[Salzig]

Got it via Amazon from a company called "KALEA INFORMATIQUE".

 

[Salzig]

yeah, that looks "promising".

 

[Salzig]

Is there any effective solution for two more SATA Slots? Tried a SATA Extension card before, which didn't worked, but i didn't really expected that after reading a bit, never the less wanted to give it a try. Now this SAS Card doesn't look that good too.

 

[Ericloewe]

Well, you have the right hardware, just less-than-fully working. I’d start by getting a replacement or a refund for the card, if possible.

 

[Salzig]

f*ck me. Just changed some things: As in removing the SSDs, removing the SAS-Expansion, switching the HDD to mainboard SATA, reinstalling TrueNAS onto the NVMe. Guess what? Still only one drive showing.

So, power off. Unpluged all SATA Cables. Started TrueNAS. Plugged first one in (yeah, not perfect). Shown in UI instandly. Plugged second one in. Not show, but Errors on console.

Nice.

 

[Salzig]

just for documentation. Errors rendered to the console when i plugin one of the non detected/working drives.

ATA Status Error
51 (DRDY SERV ERR), error: 04 (ABRT)

So, powered the TrueNAS Server down. Pulled all four HDDs. Connected them one by one via sata-usb-adapter to my computer. Sure, first one works, second failed - it didn't even spin up.

EDIT: Damn it. Removing that failing drive made two more drives apear (onboard sata). So, that really screwed the hole system.

 

[NugentS]

So one drive pulled the whole thing down?
That's ....... annoying


Hopefully it all works without that HDD

BTW - any reason you aren't using the onboard SATA for all the HDD's? Or aren't there enough

 

[Salzig]

Wanted to install TrueNAS on a dual SATA-SSD, and the MB only has four sata ports. So i thought it's maybe best to but the four HDD onto the expansion card. Was planing to use the NVMe as Cache drive.

 

[NugentS]

When you say cache - specifically what sort of cache? In ZFS terms?
I ask cos they generally don't work they way you probably think they work, and don't do the things you think they do.

 

[Ericloewe]

A disk brought down the controller? Talk about an obscure issue... So is the SAS controller working correctly without the bad disk?

 

[Salzig]

So, took a while, but now i had the time. Controller is still faulty, using the on board sata connector i get three drives, using the controller only one drive is shown. So definitly not what i was looking for ^^

A bit sad, but ok. Need to find another/better Source for a LSI card, just by looking at the card it doesn't seem to be a official LSI or any other recognizeable Brand card.

Something is really borked with this setup :D

On Cache: I suspect i would be able to use the NVMe as read/write cache?

edit: attached `sas2flash -list` output. Again, no mention of LSI on the board itself, and controller fails sometimes (as mentioned above) on boot. So, that one is going back.

 

[NugentS]

cache != cache
You haven't mentioned your use case. There are different kinds of cache, all of which may work dependant on your use case.
SLOG = iSCSI / NFS only (mostly)
L2ARC, 256GB of L2ARC will kill your primary ARC
Metadata vdev = possibly, depending on your use case
svdev = dangerous as its pool critical, so no resiliency

As for a controller, a safe bet is "Art of Server" on Ebay - but he is in the US. Buy one from a system dismantler rather than the cheapest chinese knockoff. An example that looks +ve at first glance is: Ebay.de Link

Damn things have gone up in price a lot.

Oh and get a refund from your amazon seller.