New Build - Supermicro Board Recommendations

[racielrod]

Hi,
I’m planning to build a new system to try TrueNAS Scale and eventually replace my existing TrueNAS Core.
My current system is based on X9SRL-F and I have never had an issue with it.
I would like to build something with newer hardware; and I was looking at the X11SRL-F board as an alternative.
Is this a good option? If not, what would yoy recommend?
I’m fine buying used and building my own system.

TIA,
R.

 

[Etorix]

What is your use case and what are your requirements?

 

[racielrod]

What is your use case and what are your requirements?

As a reference, this is a link to the post that describes my current build.
The only change is that I currently have 64GB of ram vs the original 32GB.

Current usage:

• NAS: Family media, backups, etc.
• Plex Media Server

In addition to that I would like to migrate some containers I have running in a different machine and consolidate them; hence my interest in TrueNAS Scale.
Examples of these containers are Home Assistant, Nginx Proxy Manager, NextCloud, etc.
My plan is to build a system with enough room for growth and experimentation, especially around memory, I would a minimum of 128GB.

My current system most likely will become my on-site backup.

Thanks,
R.

 

[Etorix]

The link is missing. But RAM is a fairly good reason to go for a RDIMM platform. A X11SRL-F has enough PCIe slots for any HBA, NIC (10 GbE?) and/or GPU (transcoding?) you may want to throw at it. Provided you can find a Xeon W-2000 at reasonable price, your plan appears to be sound.

 

[racielrod]

The link is missing. But RAM is a fairly good reason to go for a RDIMM platform. A X11SRL-F has enough PCIe slots for any HBA, NIC (10 GbE?) and/or GPU (transcoding?) you may want to throw at it. Provided you can find a Xeon W-2000 at reasonable price, your plan appears to be sound.

Apologies, I have included the link now.
I based my system on Chris' recommendation on this resource

I do use the X9SRL-F today, and I'm extremely happy with it, but I didn't see the X11SRL-F listed on this other resource about X11 boards.

I think Eric's point was that there were not many LGA2066 options out there to choose from and that they were not server boards.

My understanding is that the 2 NIC ports on the X11SRL-F are 1Gbps.
Is there any value in looking at X12 models? - not sure which generation offers the best value for a new build.

 

[jgreco]

any value in looking at X12 models? - not sure which generation offers the best value for a new build.

Best value is typically older hardware, especially if it is the stuff currently being purged out of data centers (3-5 years old). The biggest thing your X9 E3 system has working against it is that it's memory-limited to 32GB; if you were happy with its performance, you could even consider the X9 E5 gear that can easily handle half a terabyte of RAM. These are all getting somewhat long in tooth, though, so it's also fine to be considering moving on.

 

[Etorix]

I do use the X9SRL-F today, and I'm extremely happy with it, but I didn't see the X11SRL-F listed on this other resource about X11 boards.

This a resource about E3v5/6 (i.e. Core CPU with ECC), so not listing Xeon W-2000 boards is consistent.

I think Eric's point was that there were not many LGA2066 options out there to choose from and that they were not server boards.

C422 + Xeon W-2000 is officially a workstation platform. Many boards have IPMI and could be converted to server duty, but why bother when there are many more server options with C621 + Xeon Scalable at all price points (including cheaper if you go for Xeon Bronze)?

My understanding is that the 2 NIC ports on the X11SRL-F are 1Gbps.

Correct—and apparent from the lack of a "T" after the dash. "L" is for low-end.

Is there any value in looking at X12 models? - not sure which generation offers the best value for a new build.

X12STx would be 10/11th generation "Core + ECC UDIMM", so not for the amount of RAM you're looking at (or not at a good price).
X12SP/X12DP are the 3rd generation Xeon Scalable boards. Nice, but a NAS does not need the latest and greatest hardware. No "value" here!

The alternatives, for 128+ GB RDIMM RAM and not knowing how much computing power you need or want, would be:

• Old Xeon E5 on X9. Cheap but "getting long in the tooth". Bargain option.
• Xeon D-1500 (X10SDV). Small embedded boards, make good NAS. Tend to remain expensive even second-hand. :( May lack "enough room for growth and experimentation" if that involves multiple PCIe add-in cards.
• 1st/2nd gen. Xeon Scalable (LGA3647). Some of it is already being purged from data centres. Mostly too high-end for a home server, but with the right bargain…

If you do find a X11SRL-F and a second-hand Xeon W-2100 for cheap, it could be a good option. As @jgreco put it, the best value is in stuff that is old enough to be refurbished from data centres but not so old as to be useless. Until now, this pointed to X9 and X10 generations. But it may be time to put Skylake hardware and the X11 generation in the picture…

 

[jgreco]

Yeah, at this point, Nehalem/Westmere should definitely be put out to pasture.

X9 and X10 basically each cover a two year period.

X9 is probably near or at the end of its service life; I had a really big X9 system just die a few months back (though admittedly I manage dozens of them). ESXi CPU compatibility with X9 will be going away moving forward, soon, and even X10 is looking to be something that you should have a bit of a gameplan for. Part of it depends on what you plan to do with it, of course. If you can get X9 really super-cheap and it isn't in a critical role, then it's probably still fine. One of the biggest concerns is what happens when it one day goes <poof> and stops working. In my X9 dying example, we had other similar X9 systems available, so I moved memory and CPU and just spun the workloads back up. But now I've used a good bit of my spare capacity. I don't know that I want to put any more money into X9, but on the other hand, it would hurt to lose half a terabyte of RAM in the cluster. Just sayin': think about your overall gameplan.

 

[racielrod]

Thank you both @Etorix and @jgreco!
The only reason I was looking at X11SRL-F is because I’m very happy with the same X9 version (I didn’t know about being a workstation board).
I will check for prices and alternatives, but I think the minimum I will buy is X11 and keep it for longer.
This is for my home, no mission critical; but I would rather build something that gives me some peace of mind.
Will look into lga3647.

 

[Etorix]

X11SRL-F is a server board (no audio codec, few USB ports…), using what is normally a workstation platform.
It's down to your budget, your requirements… and trying not to overspend. (Or at least not too much.)

W-2100 CPUs should be available for cheap on eBay, at least those with low core counts. Same for Xeon Bronze—but these have terribly low clock, you may want to look also for second-hand Xeon Silver (and probably not higher).
Then there's the issue of motherboards. If you're not in a hurry, you may also monitor the sale forum at ServeTheHome.

Update. You're probably one of the first to come up with C422+Xeon W-2000 here. I thought it might actually be a good idea for a platform with lots of RAM and room for extension. After I posted the above I actually did some searches on eBay, to refresh my memories of when I browsed for similar parts as possible hackintosh workstations.

W-2102/2104 (4C@2.9/3.2 GHz) is indeed dirt cheap, and Chinese sellers offer "retail" (non ES/QS) W-2140B (8C@3.2 GHz, 120 W) under $150, which looks very sweet. As expected, it gets less interesting higher in the stack and for Cascade Lake Xeon W-2200.
But C422 motherboards are lacking, or I'm not looking the right way or in the right place. With a retail X11SRL-F, this becomes perhaps half of a good idea… But if size doesn't matter X11SRL-F + W-2140B (Skylake) could be a fair alternative to a Xeon D-1541 board (Broadwell).

After checking, Xeon Bronze and the bottom of the Silver range are both lacking in clock speed, and the upper part of the Silver range and the lower Gold range (5000) are not-that-great bargains. The best opportunities come from "off-roadmap" CPUs, but these are mostly in the Gold 6000 and Platinum 8000 ranges: Perfect for a big-workstation-on-the cheap, but complete overkill for a NAS. Don't spend much time pondering LGA3647.

 

[racielrod]

Thank you for the update! A lot of insights.
I have been looking at LGA-2066 and LGA-3647 CPUs, their CPU and thread marks, TDP, and overall performance here

PassMark - CPU Benchmarks - CPU Mega Page - Detailed List of Benchmarked CPUs

I used the parameters of my existing E5-2650 v2 as a reference because I don't want to build a system that is less powerful than what I have now (if anything I'm considering new hardware to get similar or more power and added efficiency, in addition, to have a system that has a longer expected lifespan than my current system) and one thing is obvious:
This X9SRL-F and E5-2650v2 setup recommended by Chris is a huge bang for the buck for sure - too bad it is "getting long in the tooth" :)

W-2102/2104 (4C@2.9/3.2 GHz) is indeed dirt cheap, and Chinese sellers offer "retail" (non ES/QS) W-2140B (8C@3.2 GHz, 120 W) under $150, which looks very sweet. As expected, it gets less interesting higher in the stack and for Cascade Lake Xeon W-2200.

Yeah, I saw good deals on the 2140B, and it seems to be in a sweet spot in terms of performance (passmark of 17,549 vs 9,938 of my current setup)

But C422 motherboards are lacking, or I'm not looking the right way or in the right place. With a retail X11SRL-F, this becomes perhaps half of a good idea… But if size doesn't matter X11SRL-F + W-2140B (Skylake) could be a fair alternative to a Xeon D-1541 board (Broadwell).

Sorry, I'm afraid I'm confused here.
What are the limitations of the C422 boards?
What's your point with the size comment?

After checking, Xeon Bronze and the bottom of the Silver range are both lacking in clock speed, and the upper part of the Silver range and the lower Gold range (5000) are not-that-great bargains. The best opportunities come from "off-roadmap" CPUs, but these are mostly in the Gold 6000 and Platinum 8000 ranges: Perfect for a big-workstation-on-the cheap, but complete overkill for a NAS. Don't spend much time pondering LGA3647.

What are considered off-roadmap CPUs? - apologies for my ignorance.
What would these be an overkill for a NAS if my goal is to use TrueNAS Scale for virtualization as well? - granted, I need to check prices and have a reality (budget) check ASAP.

Thanks again!

 

[Etorix]

Sorry, I'm afraid I'm confused here.
What are the limitations of the C422 boards?

The price, if you have to buy the motherboard new at retail price…

What's your point with the size comment?

C422 boards are mostly ATX, with some micro-ATX. Xeon-D boards are mostly mini-ITX, with some flex-ATX (Supermicro) or micro-ATX (AsRockRack), so they allow building smaller systems, which is often fine for home. (Of course, small boards fit in large cases, and a Xeon-D with a SAS HBA can power a big NAS with lots of drives.)
If you don't mind the larger ATX case, or need many PCIe slots, just ignore the comment on size.

What are considered off-roadmap CPUs? - apologies for my ignorance.

These are CPUs you do NOT find in ARK. They are custom-made by Intel for large customers and not sold at retail.
The Xeon W-21x0B are the CPUs which powered Apple's iMacPro, with 120 W TDP instead of 140 W for the regular W-21x5 parts they are based on.
There are lot of them in the Xeon Scalable range. For instance, look at this table and note the many lines with part numbers ending in 'W' or 'C' (custom?), or some odd-digit, shown in black because there's no link to ark.intel.com and only with partial information because there's no public documentation for them. And the good thing with these parts is that the hyperscalers they were made for are on relatively short upgrade cycles and one can already find these 'off-roadmap' Cascade Lake Xeon on eBay for relatively low prices compared to the related 'on-roadmap' parts.

What would these be an overkill for a NAS if my goal is to use TrueNAS Scale for virtualization as well? - granted, I need to check prices and have a reality (budget) check ASAP.

The overkill is likely to be price. 8 cores/16 threads from a W-2140B is already a lot of power.
If you want even more threads than that, you can of course go higher in the stack, albeit at higher cost and lower clock, and you can look at Xeon Scalable, if only to compare with high-core Xeon W-2000. But with the motherboard, case, PSU and RAM, not to mention 10 GbE NIC and HBA, the cost will quickly go up—and that's before putting drives in the NAS.

Thanks again!

You're welcome. Thanks for making me realise that, if one does not mind the size and the likely higher power draw at idle, a X11SRL-F could be a competitive alternative to the much-loved X10SDV.

 

[racielrod]

The price, if you have to buy the motherboard new at retail price…

Got it!

C422 boards are mostly ATX, with some micro-ATX. Xeon-D boards are mostly mini-ITX, with some flex-ATX (Supermicro) or micro-ATX (AsRockRack), so they allow building smaller systems, which is often fine for home. (Of course, small boards fit in large cases, and a Xeon-D with a SAS HBA can power a big NAS with lots of drives.)
If you don't mind the larger ATX case, or need many PCIe slots, just ignore the comment on size.

My preference is to use Fractal Design cases, so an ATX mobo would be fine.

These are CPUs you do NOT find in ARK. They are custom-made by Intel for large customers and not sold at retail.
The Xeon W-21x0B are the CPUs which powered Apple's iMacPro, with 120 W TDP instead of 140 W for the regular W-21x5 parts they are based on.
There are lot of them in the Xeon Scalable range. For instance, look at this table and note the many lines with part numbers ending in 'W' or 'C' (custom?), or some odd-digit, shown in black because there's no link to ark.intel.com and only with partial information because there's no public documentation for them. And the good thing with these parts is that the hyperscalers they were made for are on relatively short upgrade cycles and one can already find these 'off-roadmap' Cascade Lake Xeon on eBay for relatively low prices compared to the related 'on-roadmap' parts.


The overkill is likely to be price. 8 cores/16 threads from a W-2140B is already a lot of power.
If you want even more threads than that, you can of course go higher in the stack, albeit at higher cost and lower clock, and you can look at Xeon Scalable, if only to compare with high-core Xeon W-2000. But with the motherboard, case, PSU and RAM, not to mention 10 GbE NIC and HBA, the cost will quickly go up—and that's before putting drives in the NAS.

Thank you for the education on CPUs, very informative!