Supermicro X9 questions

[zenith1]

Hi,

I got a Supermicro X9SCM-F motherboard, which is mainly used for virtualization and storage. I know that people use a lot of Supermicro boards in their builds here, so I thought it will probably not hurt to ask some questions, and maybe I should be careful to not run into any potential issues.

Current setup:
- Intel Xeon E3-1260L (Sandy Bridge?)
- 16 GB (4x4GB) of UDIMM ECC RAM
- LAN2 for network connectivity (Intel 82579)
- NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04) - probably a clone of some kind
- BIOS 2.2 and BMC/IPCMI firmware 3.38
- 4 WD Red 4TB drives, and two external USB 3.0 drives (WD My Passport 0748 2TB 2.5" and WD Elements 107C 5TB 3.5")
- Debian Stretch

Currently the only add-in card is the USB 3.0-controller (NEC clone?). I will add a Dell Perc H310 HBA soon and update firmware on it for pass-through. There are 4 PCI-E slots, the manual names them them as follows:

Slot 7 PCI-E 2.0 x8 (this one is nearest the cpu, currently in use by the USB 3.0-adapter)
Slot 6 PCI-E 2.0 x8
Slot 5 PCI-E 2.0 x4 in x8
Slot 4 PCI-E 2.0 x4 in x8 (at the bottom)

I think two of those ports are connected through the CPU, and the two other are through the PCH. Not sure which ones.

This older blog entry lists several issues: http://www.lucidpixels.com/blog/supermicrox9scm-fissues I think I haven't ran into the PCI-E issues and network lag yet, not sure how often those problems occur. I think with certain CPUs the x8 slots can support PCI-E 3.0.

1. What is your opinion on the PCI-E slots on this board. Should I be aware of certain issues, like putting the USB 3.0 controller and HBA in specific slots?

2. Should I bother with updating the BIOS to 2.3 and the BMC firmware? I think BIOS 2.3 only has Spectre-related fixes. The BMC firmware is at 3.52, but I use the remote control very rarely.

3. The external WD My Passport 2TB drive has a tendency to disappear with an I/O error. I have run disk-burnin.sh on this drive several times, always no errors. Essentially the drive has to be physically reconnected to the USB-controller to make the device reappear. This can happen a couple of times within two days, or it can stay connected for 2-3 weeks at least. Currently not sure if this is a drive issue, or a USB 3.0-controller or a motherboard issue with some of the PCI-E slots. I have done one change, and that is adding a power cable to the USB 3.0 controller before bothering with changing cables, slots, controllers etc.

 

[Johnnie Black]

I think two of those ports are connected through the CPU, and the two other are through the PCH. Not sure which ones.

3 top slots are CPU slots, and curiously you can use all 20 lanes even with non Xeon CPUs, bottom slot is a PCH slot, shares the bandwidth with all the other stuff going through the DMI.

 

[zenith1]

Thanks. I think I will stay with the top 3 slots for the moment being, if it all is stable. Just installed the Dell H310 controller and hopefully it will work without issues in addition to the USB 3.0-card.

 

[pschatz100]

I would expect that adding the power cable to the USB add-in card would help with stability. I have the same motherboard, and I use a Syba PCIe USB 3.0 add-in card. No problems for me, although I don't leave my USB devices attached all the time.

According to the motherboard manual, the top three PCIe slots (slots #5, #6 and #7) are PCI Express2.0 while the bottom slot (slot #4) is PCI Express. That might also make a difference for the add-in cards, if they require PCIe2.0.

Edit: As has been pointed out, the bottom slot is always PCIe2.0. The top PCIe slots are either PCIe2.0 or PCIe3.0, depending on the CPU and bios version.

 

[Johnnie Black]

the top three PCIe slots (slots #5, #6 and #7) are PCI Express2.0 while the bottom slot (slot #4) is PCI Express.

Top 3 slots are PCIe 2.0 or PCIe 3.0 with a Xeon E3 1200 V2 CPU, bottom slot is always PCIe 2.0.