X11SSM-F and Perc H310 - Not playing nice and terribly slow

[joeschmuck]

So I'm placing this into the Off-Topic section because it doesn't pertain directly to FreeNAS.

My situation...
I have my ESXi machine which has the X11SSM-F motherboard (with the current BIOS) in which I desire to install a RAID card so I can run a mirrored pair of boot drives (SSD). But things are terribly slow. When I say slow, I mean it takes about 10 minutes to boot ESXi however on my older AMD system it takes about 1 minute with the same card and hard drives.

I received a Perc H310 running the P20 IR firmware. This card does work fine as I have tested it in another machine and it does work as one would expect.

On my X11SSM-F there was the typical Dell issue where most people would need to place tape across pins B5 & B6 (I originally did this) but I was able on the X11SSM-F board to disable to SMBus to PCI Slots by moving jumpers I2C1 and I2C2 to Pins 2-3. This allowed the system to boot up without memory issues.

Now I'm facing the super slowness of whatever this card is doing to my system and I'm not sure what, if anything, I can do with BIOS settings or other jumpers.

I'm all ears.

EDIT: The Perc card is now in Slot 4 of the motherboard meaning it's using the PCH-IO.

 

[BigDave]

With the addition of the new card, are all your PCIe slots now full of cards?

 

[joeschmuck]

With the addition of the new card, are all your PCIe slots now full of cards?

Nope, I have gutted the machine, only the one Perc card installed and only two hard drives connected to the card. It's a very bare bones setup.

 

[BigDave]

EDIT: The Perc card is now in Slot 4 of the motherboard meaning it's using the PCH-IO.

according to Dell specs for card, it's Host bus type 8-lane, PCI Express 2.0 compliant.
The board's #4 slot is listed as X4 (electrical) in an X8 (physical). Both slot #4 & #5 (PCH)
Try it in one of the first two slots (CPU controlled) which should be X8.
That's all I got!

EDIT: corrected for error, PCH slots are designated #4 & #5 (not #4 & #3 as previously stated).

 

[joeschmuck]

Yea, I tried it in slots 6 and 7, same results but I do understand the thought of placing it into a true X8 slot. I'll end up putting it back in Slot 7 after a few more attempts. Maybe I need to force PCI-E 2.0, it could be running at gen 1 speeds.

Some additional info, even when the card is initializing during a reboot, loading the ROM just takes forever.

Hey, thanks for the advice.

 

[joeschmuck]

I've submitted a request to Supermicro but I'm not expecting a favorable result.

 

[Mlovelace]

Out of curiosity, why aren't you installing ESXi to a flash drive/SD card and using the full SSDs for datastores and/or host cache? ESXi (much like freeNAS) loads into and runs in RAM. If you decide to install to a thumb drive, I would recommend moving the host scratch space and logs to a HDD/SSD backed datastore, so you don't burn up the USB thumb drive. If you keep an up-to-date backup of the ESXi config it's very easy to reload if the thumb drive dies.

Here's some VMWare KB links for the scratch space and config backup.
https://www.vmware.com/support/developer/PowerCLI/PowerCLI41U1/html/Get-VMHostFirmware.html
https://kb.vmware.com/selfservice/m...nguage=en_US&cmd=displayKC&externalId=2042141

https://kb.vmware.com/selfservice/m...nguage=en_US&cmd=displayKC&externalId=1033696

 

[BigDave]

While you wait for a response, you might tinker a bit with these BIOS options...
From page 87 of the manual.

PCIe/PCI/PnP Configuration

PCH SLOT4 PCI-E 3.0 X4 (IN X8) OPROM (X11SSM-F only)
Use this feature to select which firmware type to be loaded for the add-on card in this slot.
The options are Disabled, Legacy, and EFI.
PCH SLOT5 PCI-E 3.0 X4 (IN X8) OPROM
Use this feature to select which firmware type to be loaded for the add-on card in this slot.
The options are Disabled, Legacy, and EFI.
CPU SLOT6 PCI-E 3.0 X8 (IN X16) OPROM
Use this feature to select which firmware type to be loaded for the add-on card in this slot.
The options are Disabled, Legacy, and EFI.
CPU SLOT 7 PCI-E 3.0 X8 OPROM
Use this feature to select which firmware type to be loaded for the add-on card in this slot.
The options are Disabled, Legacy, and EFI.

I also found this mention of the H310 performance in use with ESXi and RAID

 

[joeschmuck]

While you wait for a response, you might tinker a bit with these BIOS options...
From page 87 of the manual.

PCIe/PCI/PnP Configuration

PCH SLOT4 PCI-E 3.0 X4 (IN X8) OPROM (X11SSM-F only)
Use this feature to select which firmware type to be loaded for the add-on card in this slot.
The options are Disabled, Legacy, and EFI.
PCH SLOT5 PCI-E 3.0 X4 (IN X8) OPROM
Use this feature to select which firmware type to be loaded for the add-on card in this slot.
The options are Disabled, Legacy, and EFI.
CPU SLOT6 PCI-E 3.0 X8 (IN X16) OPROM
Use this feature to select which firmware type to be loaded for the add-on card in this slot.
The options are Disabled, Legacy, and EFI.
CPU SLOT 7 PCI-E 3.0 X8 OPROM
Use this feature to select which firmware type to be loaded for the add-on card in this slot.
The options are Disabled, Legacy, and EFI.

I also found this mention of the H310 performance in use with ESXi and RAID

@BigDave I saw those BIOS options and gave them all a try (set to disable and legacy) already without any success. The link you provided, well that is a different type of problem than what I'm facing. the H310 is slow even before the OS starts. It literally takes several minutes just to have the H310 to get to the "initialize" display (where I suspect it's checking what drives and configuration is setup).

Thanks for giving it some serious effort.

 

[joeschmuck]

Out of curiosity, why aren't you installing ESXi to a flash drive/SD card and using the full SSDs for datastores and/or host cache? ESXi (much like freeNAS) loads into and runs in RAM. If you decide to install to a thumb drive, I would recommend moving the host scratch space and logs to a HDD/SSD backed datastore, so you don't burn up the USB thumb drive. If you keep an up-to-date backup of the ESXi config it's very easy to reload if the thumb drive dies.

Here's some VMWare KB links for the scratch space and config backup.
https://www.vmware.com/support/developer/PowerCLI/PowerCLI41U1/html/Get-VMHostFirmware.html
https://kb.vmware.com/selfservice/m...nguage=en_US&cmd=displayKC&externalId=2042141

https://kb.vmware.com/selfservice/m...nguage=en_US&cmd=displayKC&externalId=1033696

The entire reason for using a true RAID boot device and then add another RAID1 later for additional datastores was so if there was a drive failure, things would continue onward without the need for immediate user interaction. Why you say? Because when I'm not travel I will not be able to fix it and since I run the internet firewall (Sophos) on this box, it just needs to be as robust as possible. If I cannot get over this hurdle then I'll rethink how to make this robust and maybe I'll purchase a different RAID card that is compatible or just go back to what I had a few days ago.

 

[Mirfster]

Hmm, I have actually never used a PERC H310 in IR Mode but it taking that long to boot is all kinds of weird... Does the card have the Dell BIOS too and are you able to load the Configuration Utility?

/Edit: Noticed you mentioned that it took forever for the ROM to load, but curious as to if you are able to get into the Configuration Utility still..

 

[joeschmuck]

Yes, I was able to get to the Configuration Utility. I was able to create a RAID1 with two drives. The card works, it just works in slow motion.

I'm told that the firmware is the LSI 9211-8i P20 IR, not the Dell unique firmware (see attached screen capture), and I'm willing to try a different flash, even the Dell version if it helps but I am no expert on RAID cards which is why I'm asking for ideas. But it does run fine (normal speed) on a different motherboard, just not this oh so wonderful X11SSM-F.

 

[Mirfster]

But it does run fine (normal speed) on a different motherboard, just not this oh so wonderful X11SSM-F.

Odd, but I am not a SuperMicro user so my knowledge is a guess at best where that is concerned. However, I do have an offer for you...

I have a couple Perc H700 Raid Controllers that I can't use (don't work well in the C2100/FS12-TY with a Perc H200 at the same time... Weirdness on the C2100/FS12-TY, but that is another story..). Anyways, they are just sitting here (with a BBU) that are just "gathering dust".

If you want, I can send you one (along with a Perc 6i/R if desired) for you to test out. If the H700 works fine, I am willing to do an even trade for the H310. If the Perc 6i/R works (keep in mind it is limited to only seeing 2.2 TB of a single drive), you can keep it free of charge (not doing me any good anyways...). That should hopefully get you at least a couple other Raid Controllers to test out and see if they work.

I am no hurry with either of these cards as well, they are just *extras* that are sitting here.

 

[joeschmuck]

@Mirfster I do appreciate the offer and maybe I'll take you up on it or I'll more than likely just purchase a card known to work (my wallet just let out a scream). Let me see what Supermicro has to offer and I may try to reflash this card as well to see if there is any change. I downloaded a P20 firmware last night from both Dell for the H310 which had a minor change number from the current one installed, and one of the fixes was related to a speed issue. So now it's time for me to read up on how to upgrade this card properly. I also downloaded the P20 firmware from LSI just in case I need to give that a try.

So my concerns with reflashing this card is that there are what appears to be two parts, the firmware and the BIOS. I need to read the instructions so I do this correctly. There is no fun in bricking a device. I had bricked a router one time and it sucked bringing it back from the dead, but I did it. I still have that WRT-54GS wireless router (deep spare).

 

[Mirfster]

Not a problem, I am still willing to at least send you these two cards to try out if you want. You don't have to trade or anything. At the very least you can sure it works before deciding on buying one (if you decide to go that route). They are doing nothing here for me personally. The Perc 6i/R can't even properly fit in my C2100/FS12-TY systems with the cables attached (just too tall by a couple centimeters..).

 

[joeschmuck]

Fixed it! I flashed it to a very slightly newer version of the LSI firmware and bios. What a difference this makes. Here is a screenshot of the newer version. I'm just glad it was the card and not the motherboard. Also, I ended up placing a piece of tape across pins B5 & B6 and moved back the motherboard jumpers for I2C1 and I2C2. I am reinstalling ESXi right now. All should be good again soon.

 

[Rand]

If you flashed to 20.00.07 now then you maybe had the "known to cause issues" old version of p20 on it (04 or so) ...

 

[Mirfster]

Glad to hear it turned out well. Onward and upward... :)

 

[Mlovelace]

Good morning @joeschmuck I was going to point out that the Dell firmware is a newer version, but you fixed it. Great news!

Here a link to the Dell firmware just for grins anyway. Cheers!

http://www.dell.com/support/home/us/en/19/Drivers/DriversDetails?driverId=C19GP

 

[r0nski2000]

Sorry to resurrect and old thread, but I am in a similar situation and found some of the notes here helpful.

This is my first "server" build...
I am trying to flash an H200 to IT mode on x11ssm(bios 2.0a)/e3-1245v6 - 16gb ram and USB boot drive/kB/monitor is all that is connected.
It would boot only after I moved the I2C1 and I2C2 jumpers, as per the OP.
When I ran sas2flash -listall, i saw a sas2008 device, so I assume I would be able to flash it to IT mode
As per the guide in the "Resources" section.

Can someone explain to me in simple terms what is the meaning/usage of "SMBus to PCI slots"
And what are the effects of disabling it?
In the long run I am hoping to be able to run another H200 in IT mode and/or a third in IR mode.
I am hoping to be able to run an ESXi configuration with multiple vms, and maybe add a pcie nvme drive as a data store for ESXi/vms, although I don't think I would need to boot from it.

Does the "electric tape across B5 and B6" mod have the same effect as disabling "SMBus to PCI slots" -
Except of course it only applies to that specific card, instead of all of them?

Thanks in advance for any help.