So, you’ve decided to buy a Supermicro X11 Xeon E3 v5/6 board...

[Ericloewe]

Note: This thread is now the discussion thread for this Resource, the current version of which can be found at this link, in the Resources section.

The old version is archived below.

Show : Archived document

In case someone still doesn't get the reference, click here.

Now that Xeon E3 v5 is here, motherboard manufacturers have finally released their server lineups. Much like the Supermicro X10 guide, the purpose of this guide is to clarify the differences between the various options offered by Supermicro. Workstation boards and models designed for legacy applications (PCI and/or PCI-X slots) and boards with proprietary sizes are not considered, though some workstation models may be of interest, should a good deal be available.

The full comparison is presented below, for prospective purchasers and for troubleshooting. If you are familiar with the Supermicro X10 lineup, a shorter cheat sheet is available in this thread’s second post.

Baseline features​

Supermicro opted for a baseline closer to the X10SLL+-F/X10SLM+-F/X10SLH-F models for the X11 server motherboard lineup:

· Standard four DIMM slots, for up to four DDR4 16GB UDIMMs (64GB total RAM)
· IPMI provided by an ASpeed AST2400 BMC, with dedicated IPMI LAN port
· microATX form factor
· Dual Intel i210-AT GbE adapters
· Leftover PCI-e connectivity exposed in standard slots
· 5x USB 3.0 ports and 6x USB 2.0 ports
· C232 boards provide 6x SATA 6Gb/s ports
· C236 boards provide 8x SATA 6Gb/s ports and additional PCI-e connectivity

X11SSL-F​

The X11SSL-F employs a C232 PCH and is therefore limited to 6 SATA ports, like Haswell/Broadwell systems.

PCI-e 3.0 connectivity is as follows:

· One x16 slot (x8 electrical) from the CPU
· One x8 slot from the CPU
· One x8 slot (x4 electrical) from the PCH

A cost-reduced version without IPMI, X11SSL (no –F) is also available.

X11SSL-CF and X11SLL-nF​

Despite the name, these seem to not use the X11SSL-F layout. Feature-wise, they are similar to the X11SLL-F, though.

· X11SSL-CF routes 8x PCI-e 3.0 lanes to an LSI SAS 3008 controller, partially compensated by an additional x1 PCI-e slot
· X11SSL-nF routes a total of 8x PCI-e 3.0 lanes to two U.2 connectors, allowing for two PCI-e SSDs to be attached via cabling

X11SSM-F​

Unlike previous Haswell/Broadwell PCH families (Lynx Point), the Sunrise Point family has only two versions (C232 and C236). For this reason, the X11SSM-F is positioned differently than the X10SLM+-F.

By employing the C236 PCH, the X11SSM-F provides 8 SATA ports, allowing for a typical 6-drive pool plus two ancillary drives, like SATA DOMs or SSDs for booting, L2ARC, mid-range SLOG devices, or simply larger pools.

PCI-e 3.0 connectivity is as follows:

· One x16 slot (x8 electrical) from the CPU
· One x8 slot from the CPU
· Two x8 slots (x4 electrical) from the PCH

A cost-reduced version without IPMI, X11SSM (no –F) is also available.

The X11SSM-F seems to be based on the same layout of the X11SSL-F.

X11SSH-F​

The X11SSH-F is the high-end Xeon E3 v5 board from Supermicro. In practice, feature-wise, it’s very similar to the X11SSM-F. In fact, the only difference is that one PCI-e 3.0 x8 (x4 electrical) slot is exchanged for an M.2 connector (2x PCI-e 3.0).

For the sake of consistency, PCI-e 3.0 connectivity is as follows:

· One x16 slot (x8 electrical) from the CPU
· One x8 slot from the CPU
· One x8 slot (x4 electrical) from the PCH

X11SSH-LN4F​

The X11SSH-LN4F is very similar to the X11SSH-F. The only difference is the addition of two Intel i210-AT GbE controllers (total of 4 i210s), using the two PCI-e lanes leftover from the removal of a slot.

X11SSH-TF​

The X11SSH-TF uses a different layout than the X11SSH-F (it is similar to the X11SLL-C/F-nF layout, but different). Instead of Intel i210s, it uses the brand-new Intel X550 10GbE controller for two 10GBase-T ports.

The M.2

PCI-e 3.0 connectivity is as follows:

· One x8 slot from the CPU
· One x4 slot (x2 electrical) from the PCH (presumably)

X11SSH-CTF​

The X11SSH-CTF is similar to the X11SSH-TF, but adds an LSI SAS 3008 SAS3 controller.

This makes the X11SSH-CTF a very interesting board, since its connectivity is as follows:

· Dual 10GbE from an Intel X550 10GbE controller (no driver support yet)
· 8 SATA 6Gb/s ports
· 8 SAS3 ports (LSI SAS 3008, not yet guaranteed to work well with FreeNAS)
· 1 M.2 slot backed by PCI-e 3.0 x4

Driver support should now be in place for all hardware on these boards, with FreeNAS 9.10.
There have been no noteworthy complaints about LSI SAS3 controllers using P9 firmware/drivers and the Intel driver for 10GbE has been updated with support for the X550 controller.

-O vs -B model numbers​

Many people get confused with these model numbers when they can only find an X10SL7-F-O, instead of an X10SL7-F, or an X10SLM+-F-B instead of an X10SLM+-F.
All model numbers are exactly as printed in their respective sections. For real products, they get either an -O or a -B appended to them, for retail or bulk packaging, respectively.
An example:
A Supermicro X10SL7-F-O is a retail-packaged X10SL7-F motherboard.
A Supermicro X10SL7-F-B is a bulk-packaged X10SL7-F motherboard.


What's the difference between them? With retail packaging, you get a box, with proper packaging materials. With bulk packaging, you get something along the lines of an antistatic bag, a bit of protective foam and some cardboard to wrap it all up.


Edit 2016-04-25: Updated driver support notice for the X11SSH-(C)TF
Edit 2016-05-07: Added -B vs -O section, copied from the X10 thread. Should've been here all along. Fixed title apostrophe to proper serifed version, instead of mysterious apostrophe-like symbol.

 

[DaveFL]

I like these for a workstation. If you plan to run bare metal and will never need more than 64GB they are great.

As you mention. The support for the older hardware is better (not sure if his applies to OmniOS as well, it might not have issues).

I still like Xeon D as it has more cores and allows for more memory.

I'll probably pick one up for a Workstation - X11SSZ.

 

[nidur]

The X11SSL-CF would be my choice for a Fileserver today, but the X11SSH-CTF will hopefully get full support in the near future then it would be a definite winner.

For a workstation I would personally go for the X11SAE-M
M.2 (PCIe x4), PCIe x16 slot for large GPU, 2x USB 3,1 in M-ATX size.

 

[DrKK]

We got a guy with a successful setup on the X11SSH. I'm going to have him make a forum post to collect thoughts/experiences on it.

 

[religiouslyconfused]

Yes, one user in the US has a successful X11SSH-F setup and he uses an SSD though for his boot drive, as Skylake only supports XHCI and FreeNAS has that disabled in 9.3. Version 10 should support this when it comes out.

 

[religiouslyconfused]

@Raiz has successfully ran a X11SSH-F setup and it seems to work for him. Skylake C232 and 236 board should use the Intel i210 ethernet chipset, so that should be supported in freenas just fine.

 

[Dice]

Are there any other 'negative reports' on X11 boards as of now?
Other than what I know - the XHCI, the lack of support for boards with SAS3008, 10GbE not yet 'there'?

Cheers /

 

[Ericloewe]

Are there any other 'negative reports' on X11 boards as of now?
Other than what I know - the XHCI, the lack of support for boards with SAS3008, 10GbE not yet 'there'?

Cheers / Dice

No issues are currently known besides the obvious ones:

• USB only works with the XHCI driver
  • Some people have reported that IPMI HIDs do not work either, but I don't think it's a real issue. Others have used IPMI successfully. Besides, why would the BMC provide HIDs using the PCH USB ports? It would almost certainly present the OS with PS/2 or USB HIDs from a PCI-e controller - in the latter case, it would certainly be an EHCI controller.
• The SAS3008 controller used in some boards is unproven and potentially unstable.
• The Intel X550 10GbE controller used in some boards (at least some of them without any 1GbE controllers) is not yet supported in FreeBSD/FreeNAS. This should change "soon" and it'll be a heck of a controller. It'll also support 2.5GbE and 5GbE, when those start appearing.

 

[RchGrav]

The Intel X550 10GbE controller used in some boards (at least some of them without any 1GbE controllers) is not yet supported in FreeBSD/FreeNAS. This should change "soon" and it'll be a heck of a controller. It'll also support 2.5GbE and 5GbE, when those start appearing.

This can't happen soon enough...

 

[religiouslyconfused]

I believe FreeBSD is getting X550 support as there are some drivers that appear to be compatible with certain versions of that chip. I think it will definitely happen for FreeNAS 10 which is coming out sometime this year.

 

[Rand]

By employing the C236 PCH, the X11SSM-F provides 8 SATA ports, allowing for a typical 6-drive pool plus two ancillary drives, like SATA DOMs or SSDs for booting, L2ARC, mid-range SLOG devices, or simply larger pools.

Does that mean that utilization of 2 DOM's will reduce the number of usable SATA ports to 6 ? Or are these on top of the 8 ports for a C236?

 

[hugovsky]

Does that mean that utilization of 2 DOM's will reduce the number of usable SATA ports to 6

Yes.

 

[Ericloewe]

C232, like older PCHs, provides 6 SATA ports.
C236 provides 8 SATA ports.

The DOM ports are just two of the aforementioned SATA ports, with provisions to directly power Supermicro DOMs.

 

[Rand]

Bummer :/
Does anyone know whether (when?) support for the x550's will make it into FreeNas 9.x?

 

[religiouslyconfused]

Not sure, but I am sure you could get the X550 FreeBSD drivers and try to compile them for 9.x.

 

[religiouslyconfused]

Also, on Intel website I can only find X540 drivers for FreeBSD, but FreeNAS 10 is not that far off and I am sure Intel is working on a driver or a driver already exists.

Plus, that Supermicro board (X11SSH-TF/CTF) is not even out yet, so you will have to wait anyways and it might not come out until after FreeNAS 10. Then you will need to wait for a driver or Intel will have a driver for FreeBSD that you can use.

 

[KJaneway]

What about the X11SSZ-TLN4F for a NAS? It isn't included in the overview

 

[religiouslyconfused]

I don't think that would be a good server board as it contains unnecessary features such as display port and Realtek audio which is unnecessary for a Nas server. Would make a good workstation board. Plus it only has 4 sata ports from what I have read.

 

[religiouslyconfused]

Plus x550 chip is not compatible with freenas yet and I would recommend x11ssh-tf if you want a 10gbe board but nothing wrong with the x11ssh-f or x11ssm-f.

 

[KJaneway]

It was just my idea because the X11SSZ-TLN4F is easily available in contrast to the x11ssh-tf and cheaper. I want definitely a 10GBit NIC

But you've made a good point with the audio and displayport .