So, you’ve decided to buy a Supermicro X11 Xeon E3 v5/6 board...

So, you’ve decided to buy a Supermicro X11 Xeon E3 v5/6 board...

Status
Not open for further replies.

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
Might be a FreeBSD thing, as you should ask FreeBSD for that, but you can get a PH10 driver for FreeBSD. Though, you need to use it at your own risk. Might want to submit a bug report, to FreeNAS, and maybe they can fix it.
 

RichTJ99

Patron
Joined
Sep 12, 2013
Messages
384
Think 65 hours on memtest is enough with 7 passes?
 

BigEndian

Cadet
Joined
May 21, 2016
Messages
3
Hi,

I've bought the Supermicro X11SSL-CF, a Xeon E3-1225v5 and 2 x M391A2K43BB1 (2 x 16 GB) in a kind of haste this week.

Origignally I planned to buy the X11SSL-F but in the last moment I went for X11SSL-CF without further researching. Now I've detected that the CF on-board SAS controller LSI 3008 is an early stage in FreeNAS and I don't wont to trust it with my precious data yet. Thus, this leaves me with 6 native SATA 3 ports but I need 7. :-(

Since all is in mint condition still, I wonder whether I should exchange this with the originally planned X11SSL-F or even get the full-blown overkill board X11SSL-CTF for twice the bucks.

Thus, a couple of questions to help me with this decision:
  • is it possible to hook up an optical SATA 1 drive to a SAS controller in general and to the LSI 3008 in current state specifially?
  • does SAS to SATA break-out cables, eg. AMCC 3ware CBL-SFF8087OCF support all SATA versions (1, 2 & 3)? When I research for these cable I don't find any information about the specific SATA version they support. I guess SATA 1 and 2 should be fine but what about SATA 3?
  • all the above listed Supermicro boards have the ASPEED AST2400 BMC on-board graphic controller which I plan to use only for troubleshooting and installation then disable it to save energy in production deployment. Does the integrated GPU on the Xeon E3-1225 interfere with disabling the ASPEED controller somehow? I guess theses boards could only use the ASPEED to drive the on-board VGA connector and the IGP on the E3-1225v5 should be no problem because it is not enabled / initialized by the BIOS. Right?
Thank you very much for any insight in these questions.
 

xhoy

Dabbler
Joined
Apr 25, 2014
Messages
39
Ok, our X550 is working fine ran some test and the worked great. so it looks that nothing can stop us from using a X11SSH-CTF board :)

The new memory stick we got was bricked as well. In 3 other systems (customer grade) we found 2 other broken kingston sticks, got a feeling they have a quality issue? I am doing this for 10 years and last moth more broken memory then in the last 10 years combined..... anyway.

We decided to by 10x4TB HGST deskstar disk (SKU 0S03356). We also got our sas to sata kabels (CBL-SAST-0631) We hooked everything up. Then we noticed there is an additional 8pin header on this cabels. It seems that these go on the I-SGPIO1/I-SGPIO2 headers with the "red" pin on facing the IO shield. We think this is for the "blinking"but we are not sure since we have an issue here...

We flashed the LSI3008 to IT mode with the supermicro firmware this worked as hoped.... But we cannot see any disks....
no_disk_support.jpg
So we played around with jumpers JPSAS1 / JPS1 but no luck so far. @Dudde what disks are you using and how did you connect them to your mainboard?
Somebody any thoughts?
 

xhoy

Dabbler
Joined
Apr 25, 2014
Messages
39
Great questions, and i am in the middle of answering them for you!

is it possible to hook up an optical SATA 1 drive to a SAS controller in general and to the LSI 3008 in current state specifially?
Yes it should be... but you need a breakout cable. see my list above.
Does SAS to SATA break-out cables, eg. AMCC 3ware CBL-SFF8087OCF support all SATA versions (1, 2 & 3)? When I research for these cable I don't find any information about the specific SATA version they support. I guess SATA 1 and 2 should be fine but what about SATA 3?
AFAIK (but working on that) all cabels should be working for all versions since there hasn't be any major cable changes in the standards. But you could just buy a supermicro breakout cable. since i hope my solution works :)
all the above listed Supermicro boards have the ASPEED AST2400 BMC on-board graphic controller which I plan to use only for troubleshooting and installation then disable it to save energy in production deployment. Does the integrated GPU on the Xeon E3-1225 interfere with disabling the ASPEED controller somehow? I guess theses boards could only use the ASPEED to drive the on-board VGA connector and the IGP on the E3-1225v5 should be no problem because it is not enabled / initialized by the BIOS. Right?

I asked SM exactly that! but never got an answer from them. I think (but this is my assumption) that the ASPEED is only used for the IMPI and doesn't do anything for displaying on the VGA port BUT i am not sure and i love to hear the anwser to this question.
 

Cornstick

Dabbler
Joined
May 16, 2016
Messages
12
OK.... Let me add some fresh new info for you. I have the X11SSH-LN4F. What a mess! If all you plan to use is the SATA RAID then go for it! But if you plan on adding ANY other boot drive (i.e. M.2 on the X11SSH-LN4F or a add on SATA card) forget it! The chipset can't handle it. I'm currently working with Supermicro to resolve the issue. Their might be a beta BIOS I'll be testing. Stay tuned!

Sent from my VS986 using Tapatalk
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
First of all, a note to everyone:

This thread is not appropriate for questions regarding specific problems you might be having, unless they're specific to these boards.

Discussions regarding these motherboards are fine, but if the thread continuously devolves into "I have problem X and I happen to be using an X11 board", I'll have to ask the mods to lock the thread, since I can't continuously ask them to fork off strings of messages into new thread - but that would be a pain in the ass, since I'd need a mod just to edit this and it needlessly dissuades discussion.

tl;dr - Please think if your post would be better off in its own thread before posting here.

Answers to some questions follow in the next post:
 

xhoy

Dabbler
Joined
Apr 25, 2014
Messages
39
@Ericloewe I tought this would be the appropriate place since it seems to be a setting ppl need to know about when using this board and SAS + sata drives. Ill repost it a separate thread :)
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
all the above listed Supermicro boards have the ASPEED AST2400 BMC on-board graphic controller which I plan to use only for troubleshooting and installation then disable it to save energy in production deployment. Does the integrated GPU on the Xeon E3-1225 interfere with disabling the ASPEED controller somehow? I guess theses boards could only use the ASPEED to drive the on-board VGA connector and the IGP on the E3-1225v5 should be no problem because it is not enabled / initialized by the BIOS. Right?
Note that VGA can only be enabled/disabled with a jumper.
Additionally, don't expect significant improvements in power consumption, since that only disables the VGA circuitry.
Actual graphics adapter functionality has to keep working to allow for IPMI's remote console.

CBL-SAST-0631)
Those sound like reverse breakout cables. You need forward breakout cables going from an HBA or expander with an SFF8087/whatever the new one is called to 4x SATA on the disks.

ASPEED is only used for the IMPI and doesn't do anything for displaying on the VGA port
Yes it does. The ASpeed 2400 BMC is responsible for IPMI, sensor management and basic video adapter duties. It cannot be disabled.

OK.... Let me add some fresh new info for you. I have the X11SSH-LN4F. What a mess! If all you plan to use is the SATA RAID then go for it! But if you plan on adding ANY other boot drive (i.e. M.2 on the X11SSH-LN4F or a add on SATA card) forget it! The chipset can't handle it. I'm currently working with Supermicro to resolve the issue. Their might be a beta BIOS I'll be testing. Stay tuned!

Sent from my VS986 using Tapatalk
Well, PCI-e cards are generally expected to provide their own extension ROMs to allow for booting and internal configuration. Like LSI SAS adapters do.
As for M.2 not booting properly, that doesn't surprise me. The BIOS is still very buggy and incomplete. The Windows 7 install compatibility option, for instance, does nothing and doesn't even work with Windows 7
 

xhoy

Dabbler
Joined
Apr 25, 2014
Messages
39
I was just reposting this?.... Shall i crop my post?

The cables we have are CBL-SAST-0631 and they look like this: I get it i need the "cross over cables..." ... THANKS!

EEUH we need the NOT cross over cables.....

SO new list:
When you want to use your SAS ports for sata disks (reverse breakout cable, SAS to SATA):
CBL-SAST-0556 => internal mini SAS HD to 4 SATA, 90/90/75/75cm w/75cm SB, S. 30AWG SAS/SATA Internal miniSAS HD to 4 SATA (€ 17)

When you like your sata ports to connect to a SAS backplane (forward breakout cable, SATA TO SAS):
CBL-SAST-0631 => cross-over mini SAS HD to 4 SATA w/SB, S. 55cm. 30AWG SAS/SATA Internal miniSAS HD to 4 SATA Cross-over 55 cm (~€ 9)
CBL-SAST-0591 => cross-over mini SAS HD to 4 SATA w/SB, S. 75cm. 30AWG SAS/SATA Internal miniSAS HD to 4 SATA Cross-over 75 cm (~€ 14)
 
Last edited:

BigEndian

Cadet
Joined
May 21, 2016
Messages
3
Yes it should be... but you need a breakout cable. see my list above.

Ok, so I have a 3rd option and keep the X11SSL-CF and hook-up the optical drive to the SAS controller. Then the 6 SATA ports would suffice.

OK.... Let me add some fresh new info for you. I have the X11SSH-LN4F. What a mess! If all you plan to use is the SATA RAID then go for it! But if you plan on adding ANY other boot drive (i.e. M.2 on the X11SSH-LN4F or a add on SATA card) forget it! The chipset can't handle it. I'm currently working with Supermicro to resolve the issue. Their might be a beta BIOS I'll be testing. Stay tuned!

Thanks for this information, in case I chosse to switch to X11SSL-F I also intended to order an M.2 SSD along. I guess this is also a little bit too early, a normal SATA HD must suffice for starting in this case.

Thanks to both of you.
 

BigEndian

Cadet
Joined
May 21, 2016
Messages
3
Discussions regarding these motherboards are fine, but if the thread continuously devolves into "I have problem X and I happen to be using an X11 board", I'll have to ask the mods to lock the thread, ...

Sorry, some of my questions were a "little" bit too general. I'll try to stay focused on questions/problems and hopefully soon experiences specific to SM X11SSL boards.

Note that VGA can only be enabled/disabled with a jumper.
Additionally, don't expect significant improvements in power consumption, since that only disables the VGA circuitry.
Actual graphics adapter functionality has to keep working to allow for IPMI's remote console.

Yes, this was my misunderstanding, I thought the ASpeed (or at least its graphic adapter) is to be disabled by the jumper JPG1 and I had feared that the integrated graphic processor of the E3-1225 CPU would then drive the VGA circuitry without possibility to disable it.

However after reading the manual again it is indeed as you say only the VGA connetcor is disabled by the jumper. So my understanding so far is and I hope it is correct: The VGA connector can only be driven by the ASpeed (on boards X11SSL-F, X11SSL-CF and X11SSL-CTF) and there is no way for an IGP of a CPU to use it (and waste engergy even if this should not bee too much).

Thanks.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
However after reading the manual again it is indeed as you say only the VGA connetcor is disabled by the jumper. So my understanding so far is and I hope it is correct: The VGA connector can only be driven by the ASpeed (on boards X11SSL-F, X11SSL-CF and X11SSL-CTF) and there is no way for an IGP of a CPU to use it (and waste engergy even if this should not bee too much).
Right, the server boards generally don't enable the iGPU (in fact, IIRC, Intel integrated graphics don't support VGA output anymore).

Unfortunately, this seems to also make it unusable for GPU acceleration workloads (not that such acceleration is even remotely widely supported under FreeBSD).
 

Cornstick

Dabbler
Joined
May 16, 2016
Messages
12
A couple things to add here. If you disable the ASpeed VGA via the jumper, it disables the entire function of the chip which also includes tempature controlled fan speed. So if your ok with the fans at full speed all the time then this isn't an issue. On the Supermicro server chassis I purchased the fans are extremely loud at full speed.

I wanted to also add that I got the M.2 drive and the C236 PCH RAID to play nice with each other. What I had to do is set the RAID to UEFI and then set the boot menu to DUAL (Legacy/UEFI) and that made it possible for me to choose the M.2 as boot and Windows 8 to see the RAID with Intel Rapid Storage Technology Enterprise software installed. Of course this would only work with a UEFI enabled OS which means Windows 8 and newer.

One other note that Ericloewe mentioned is the Windows 7 USB support setting. As he said it doesn't function at this time. However Supermicro said a BIOS update will be out soon to fix this issue. I'm working with them to also address the M.2 / RAID working in Legacy mode.

Sent from my VS986 using Tapatalk
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
A couple things to add here. If you disable the ASpeed VGA via the jumper, it disables the entire function of the chip which also includes tempature controlled fan speed. So if your ok with the fans at full speed all the time then this isn't an issue. On the Supermicro server chassis I purchased the fans are extremely loud at full speed.
Are you sure about that?
 

Cornstick

Dabbler
Joined
May 16, 2016
Messages
12
Are you sure about that?
Positive! I've moved that jumper so many times the plastic is messed up on it....

Because I am running Windows Media Center, I knew I needed an HDCP compliant graphics card to pass Digital Cable Advisor. Then I found a script that someone created for this exact reason the tricks DCA into a "PASS" rating.

So I removed the AMD graphics card and went back to the ASpeed.

Sent from my VS986 using Tapatalk
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Positive! I've moved that jumper so many times the plastic is messed up on it....

Because I am running Windows Media Center, I knew I needed an HDCP compliant graphics card to pass Digital Cable Advisor. Then I found a script that someone created for this exact reason the tricks DCA into a "PASS" rating.

So I removed the AMD graphics card and went back to the ASpeed.

Sent from my VS986 using Tapatalk
That's weird. My X11SSM-F has a jumper to enable the BMC and one to enable the VGA output.
 

Cornstick

Dabbler
Joined
May 16, 2016
Messages
12
That's weird. My X11SSM-F has a jumper to enable the BMC and one to enable the VGA output.
Ah yes. The X11SSH-LN4F has the two separate jumpers as well. However I don't think the BMC jumper does anything. I will explain.

After I disabled the onboard VGA / installed graphics card and noticed the fans not slowing down, I decided that the only reason I was leaving the BMC enabled was for the fans. So I disabled the BMC as well in hopes to skip all the pre-post checks and speed up boot time. Disabling the BMC did nothing that I could tell. Mind you that when external graphics are used all the pre-post stuff is not viewable. Nothing is displayed until the BMC turns it over to BIOS.

So I re-enabled the BMC. It wasn't until I re-enabled VGA that the BMC actively controlled fan speed.

I should add the only thing I didn't try was VGA enabled and BMC disabled. Maybe the BMC jumper would have had some effect then?

Sent from my VS986 using Tapatalk
 
Last edited:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Hmm, almost sounds like the documentation has them reversed. I'll add that to my list of things to investigate.
 

Cornstick

Dabbler
Joined
May 16, 2016
Messages
12
Hmm, almost sounds like the documentation has them reversed. I'll add that to my list of things to investigate.
I thought about that. So I disabled the VGA, hooked up a monitor, and powered up. No video signal.

Sent from my VS986 using Tapatalk
 
Status
Not open for further replies.
Top