LSI HBA Sometimes does not initialize after a shutdown

[Luciano2572]

Hi all!
I am building a freenas server, after lurking around here for a long time, I've got the following hardware:

• motherboard: supermicro x9scm-f
• CPU: Intel Xeon e3 1230v2
• RAM: I bought 32gb of RDIMM only to find out it won't work in the super micro board so I'm running a 16gb ECC kit I have leftover from a mac pro upgrade until I can buy a 32gb UDIMM set, currently running ELPIDA 1866mhz 4x4gb sticks ECC
• hard drives: Only using test drives as of right now, old hard drives and such, but I have 11x WD Gold 10TB and the plan is to make a RAID Z3
• boot drive: 2x Crucial BX300 120GB SSD mirrored on freeNAS, connected to sata 0 and sata 1 on the motherboard
• hard disk controllers: 3x LSI 9211-8I flashed to v20 IT
• network cards: built in intel
• power supply: Corsair HX1000 Platinum
• chassis: TGC-4324 24-bay 4U chassis

Here's the thing:
When I built it, I only had 2 of the LSI cards (third one was still in the mail), so I booted it up and noticed there was no CTRL C option for controller settings. I noticed both amber lights of the HBA were lit.
I found out that if I cut the power to the power supply while it's running, then turn it on again, I have a 70% success rate on it booting correctly and shutting down the amber lights and flashing only the heartbeat green led.
After adding the third card (and thus flashing all three at the same time to v20 IT), I noticed the third card would always start up correctly, I attached a picture to clarify.
I've fiddled with "PCI lot SMB Enable"jumper, it made no difference.

This is my first time with a "server motherboard", but when I plug it in the wall and press the power button, it spins the fans for a few seconds, shuts down, and a few seconds later it spins the fans again and boots, is this normal?
I bought this board second hand, it seems to work fine I just can't isolate the problem to the HBA or motherboard. For now while I'm doing tests with the freenas environment I'm just cycling power until it initialises correctly.

PS: I'm not using the PCIE slot closer to the cpu because a) it interferes with the CPU cooler and b) previous owner cut the back of it so he could slide a 16x card in and it's a bit dodgy
PS2: This behaviour happened before me flashing the v20 IT firmware, so I don't think it's firmware related.

Thanks in advance for any help!!

Yours,
Luciano

 

[dlavigne]

Were you able to resolve this?

 

[Luciano2572]

Were you able to resolve this?

Nope, I have bought a Dell PERC H310 for personal use, will try it on the server to see if that makes any difference. For now, I have to power cycle a few times to get the LSI cards to boot :(


Thanks,
Luciano

 

[pro lamer]

Are you considering shuffling the HBA cards around the slots (to rule out dodgy slots) ?

Sent from my phone

 

[Luciano2572]

Yep, first slot always run fine no matter which card is on it.
I`ve got the server running since jan, when I have to shutdown/power on I found a workaround: power it on, it will try to power on - shut off - and go on again. When it goes on again, turn the power supply off on the switch after 5-10s. Then, turn everything back on and it will work. I don't know why this is, but it works..

I'm considering buying another motherboard