X11SCA or X11SRA-RF setup

[Okeur75]

Hello gentlemen,

(Follow-up of this thread which I hijacked)

I'm looking for a new system for my TrueNas. Here are my needs and my findings so far, I would appreciate any feedback!

New hardware requirements :

1. The motherboard must (at least):
   1. Support 2 M2 slots
   2. Support 2 PCIe (not necessarily 16x) but separated by at least 2 slot (the GPU is 2 slots wide)
   3. Fit in a desktop case (so no exotic format, must stick to xATX but the GPU width requirement almost excludes anything except ATX and E-ATX)
2. The CPU must handle very little workload...
3. RAM must be ECC (I've read too much on this forum now...)

Existing hardware:

1. 8 * 16TB SATA drives
2. 3 * SATA SSD 120G
3. 3 * NVME SSD 1TB
4. LSI SAS3008 IT mode
5. FAN, case, PSU, screws, cat hairs...

Usage :

1. 96TB of media server with extremely static data and large file (if you think of any tuning recommendation)
2. Encoding/Decoding supported by the GPU so no need for a powerful CPU
3. 5~ VMs : media server, windows gaming station, some containers... (64GB RAM may be enough)
4. TrueNas must be installed on a pair of SATA SSD connected to the MB
5. The VMs must run on a pair of NVME SSD connected to the MB

My findings so far :

• I have excluded X11SR1-F since it has only one M2 slot
• I have excluded all board with LGA3647 since none of them seems to have 2 M2 slots.
• I have excluded X12 and X13 boards

So I'm thinking either :
X11SCA + Xeon E-2136
OR
X11SRA-RF + Xeon W-2223.

For the X11SCA, it has a LGA1151 allowing 16 PCIe lines in total, and the MB specification says that the 2 PCIe slot are running at x8/x8. So how would behave the 2 M2 which are supposed to run at 4x each ? Are both x8/x8 links shrinked to x4/x4 to let some lines for the M2 ? Or are the M2 slots condemned ?

I have not checked X10 platform with LGA2011 socket though.

 

[Davvo]

About your question, take a look at the diagram in page 21 of the motherboard's manual.

You probably want to set the blocksize of your media dataset to 1M if we are talking about videos.

 

[Okeur75]

Thanks. I thought the connection between the CPU and the CPH was using PCI-e lines as well but it is using DMI3. Meaning the whole 16 PCIe line will be available for the GPU and the HBA (running at x8/x8) and the M2 SSD will be using the PCH connection.

So now that the 2 systems are "valid", do you have any recommendation ?

 

[Etorix]

My findings so far :

• I have excluded X11SR1-F since it has only one M2 slot

That's X11SRL-F (for "low-end", but it's not so low…).
Note that a PCIe slot can easily be used to host a M.2 drive, or even several if the board provides bifurcation.

adapter m2 pcie for sale | eBay

Get the best deals for adapter m2 pcie at eBay.com. We have a great online selection at the lowest prices with Fast & Free shipping on many items!

www.ebay.com

Given your requirements, I think that a X11SR board with lots of second-hand RDIMM is the best choice. You need RAM both to manage large storage (say 64 GB) and then for the VMs on top of that. Check pricing and availability, but for 96-128 GB RAM, X11SR + (low core, high frequency) Xeon W-2000 + RDIMM should come out better than X11SC + Core i3/Xeon E-2000 + ECC UDIMM (and server X11SRL should be cheaper than workstation X11SRA boards).

 

[Okeur75]

That's X11SRL-F (for "low-end", but it's not so low…).

My bad !

Note that a PCIe slot can easily be used to host a M.2 drive, or even several if the board provides bifurcation.

Right, but we go down the road where we add device to handle hardware some motherboard handles natively.
An extra PCIe card is an extra hardware that can fail, an extra driver that may be bugged etc.

Given your requirements, I think that a X11SR board with lots of second-hand RDIMM is the best choice. You need RAM both to manage large storage (say 64 GB) and then for the VMs on top of that.

I was aiming 64-96GB for a start, and given the server load I think it will be fine. So a motherboard with 4 RAM slots would be enough with 32GB stick to go up to 128GB. But we agree I have to exclude small board handling 64GB at max.

Check pricing and availability, but for 96-128 GB RAM, X11SR + (low core, high frequency) Xeon W-2000 + RDIMM should come out better than X11SC + Core i3/Xeon E-2000 + ECC UDIMM (and server X11SRL should be cheaper than workstation X11SRA boards).

The setups I mentioned in the intial message cost the following :

1. X11SCA + Xeon E-2136 : 640 USD
2. X11SRA-RF + Xeon W-2223: 800 USD

But the first setup becomes more expensive once you add the U-DIMM RAM, so you're right.

So I guess it narrows it down to either X11SRL-F or X11SRA, with a 20USD difference between the boards and same availability. But the X11SRL-F would require me to buy an extra PCIe card to connect one (or 2) M.2 drives, and given the link you provided and the cost of such card, we will end up at the exact same price...

So, is there a real difference between the 2 cards, or is it just branding to use the terms server and workstation ?
I can only think of the secondary gigabit controler on the SRA with the Aquantia chip, but I'm not using both ethernet ports.

 

[Etorix]

An extra PCIe card is an extra hardware that can fail, an extra driver that may be bugged etc.

In this case, it's just a small PCB with traces, no driver and not much chances it will ever fail.

So I guess it narrows it down to either X11SRL-F or X11SRA, with a 20USD difference between the boards and same availability. But the X11SRL-F would require me to buy an extra PCIe card to connect one (or 2) M.2 drives, and given the link you provided and the cost of such card, we will end up at the exact same price...

So, is there a real difference between the 2 cards, or is it just branding to use the terms server and workstation ?

X11SRA is meant to be a desktop, with on-board audio, a comprehensive set of USB ports and the consumer fancy Aquantia 10 GbE NIC which is not recommended for TrueNAS. None of these are relevant to a server (unless you expect to later turn it into a desktop!). Overall, the X11SRA should be more expensive than the X11SRL (which is a weird but nice twist of a worskstation plateform into a low-cost server), and certainly was at launch; maybe it's different now that this old platform runs out of stock.

At this point, my recommendation is to shop around, including the second-hand/refurbished market (eBay, ServeTheHome forum, local small-ads…), for ECC RAM (definitely!) and possibly CPU and/or motherboard and settle on whatever comes out cheaper overall.

 

[Okeur75]

I was not planning on using the Aquantia NIC don't worry

To be extra specific regarding audio, the SRA-RF does not have audio. And USB are extremely similar betweens these 2 boards if we look into the manuals...

But the difference that really matters is the IPMI port available on the SRL and not the SRA (due to the Aquantia NIC) if I understand correctly the manuals.

So I guess I will go for the SRL-F, and follow your last recommendation.

 

[Okeur75]

Extra question if I may about the RAM :

I found the following used RAM online and would like confirmation about the compatibility

RAM	X10 compatible	X11 compatible	Price for 32GB
M393A4K40CB2-CTD		yes	75$
M393A4K40BB1-CRC	yes	?	50$
HMA84GR7MFR4N-UH	yes	?	100$

There are some differences in the speed and Cas Latency, but they are all ECC, RDIMM, 32GB.

So the question is : Are the "old" RAM still compatible with a X11 board ?

If so, do we agree the CRC RAM module (2nd row) will be compatible with a Xeon W-21xx too ?
I've read in the supermicro motherboard documentation that a Xeon W-22xx is required to handle 16Gb density DRAM.

And looking at the spec sheet of the RAM it seems the density is at 8GB.

I just want to be extra sure before buying anything...

Regards,

 

[Etorix]

So the question is : Are the "old" RAM still compatible with a X11 board ?

In principle yes but 2400 MHz modules will, obviously, limit speed. The CB2 modules look like the obvious choice for a Xeon W-2000.