Yet Another Hardware Request: Picking a CPU Gen

[t3ch]

Hello all, firstly apologies for an additional hardware-help thread, but I'm having a serious case of decision paralysis and figured I might as well make a post versus entering another 10 hours of circular googling. Maybe this will also help others?

Use-Case:​

System will see relatively light usage.

• Media server (<=2 concurrent, will be an occasional use thing)
• adblock
• VPN
• ebook server
• linux vm
• + whatever misc container/vm I decide to play with

Considerations:​

• Happy to overbuild a bit for the headroom and reliability (and fun of it)... buy/cry once
• but also want to be mindful of needless power draw
• ECC a hard-requirement
• ICMP sure would be nice
• intel NIC
• would prefer not to deal with gfx card if feasible

Hardware (Current):​

• Fractal R6 (can handle EATX)
• platter drives (lets say 5x 14tb to keep things simple)
• ssd (2x 256gb)

(TLDR - Start here)​

Hardware (Needed):​

OK finally to the actual question. Having a hard time putting my finger on which gen strikes the best balance of value, efficiency, and power. I know I'd probably be happy with any combo of these, but figure I might as well get my moneys worth. I initially wrote this as 'example builds', but I thought it made more sense to make use of the notes I've already taken and provide a more complete list of available options.

LGA1700​

+Newest architecture
+Efficient
+W680 allows for consumer cpu ECC
+ gfx
+QSV
- everything expensive/inflated/OOS, other than the CPUs
- possible weirdage with P/E cores

• Mobo
  • AsusRack E3C256D4U-2L2T (DDR4)- $450
  • Supermicro X13SAE-F (DDR5) - $450
• CPU
  • Core 12600k (10C(6p/4e)) - $200
  • Core 12700k (12C(8p/4e)) - $250
• RAM
  • 64gb ECC UDUMM DDR4 - $170
  • 64GB ECC UDIMM DDR5 - $280

LGA 1200
+ still relatively new but with used availability
+ onboard gfx
+ QSV
- ECC UDIMM DDR4 is expensive

• Mobo
  • Supermicro X12STH-F - $270
• CPU
  • Xeon W-1250 (6C, 12T) - $100
  • Xeon W-1270 (8C/16T) - $180
  • Xeon W-1290 (10C/20T) - $290
  • Xeon W-1350p (6C, 12T) - $300 (new)
  • Xeon E-2324G (4C, 4T) - $180
• RAM
  • 64GB ECC UDIMM DDR4 - $170
  • 128gb ECC UDIMM DDR4 - $340

LGA1151
+ cheap mobo/cpu
+ QSV
+ GFX
- ECC UDIMM DDR4 is expensive

• Mobo
  • Supermicro X11SSM-F -$170
• CPU
  • Xeon E-2124G (4C/4T) - $60
  • Xeon E-2174G (4C/8T) - $125
  • Xeon E-2226G (6C/6T) - $250
• RAM
  • 64GB ECC UDIMM DDR4 - $170
  • 128gb ECC UDIMM DDR4 - $340

LGA 4189
+getting into actual-server hardware!
- I can't afford this platform lol

LGA 3647
+ actually affordable server hardware
+ cheap RDIMM !!
+ more horsepower
- no QSV
- no gfx
- efficiency numbers harder to come by for home-use situations? don't know much about scalable

• Mobo
  • X11SPL-F (C621, 8 sata, 1GbE) - $300 ($380 new)
  • X11SPI-TF (C622, 10 sata, 10GbE) - $400 ($450 new)
• CPU
  • Bronze 3106 (8C/8T) -$15
  • Bronze 3204 (6C, 85w) - $20
  • Silver 4108 (8C, 85w) - $10
  • Silver 4109T (8C, 70w) - $25
  • Silver 4114 (10C, 85w) - $12
  • Silver 4116 (12C, 85w) - $15
  • Gold 5115 (10C, 85w) - $30
  • Gold 5118 (12C, 105W) - $20
  • Gold 5120 14C, 105w) - $30 Gold 6126 (12C, 135w) - $30
    Gold 6130 (16C,125w) - $40
    Gold 6132 (14C, 140w) - $30
    Gold 6142 (16C, 150w) - $75
• RAM

 

[t3ch]

Well crap I accidentally hit submit before being able to clean things up. Will do in the morning, hopefully post is approved.

 

[sfatula]

Given your very light usage, I don't see the point of going expensive other than it's shiny new and cool. Your cheapest option would be more than adequate. I built using an LGA2011-3 and average over a week is 97.5% idle, despite extensive media server/dvr usage and a lot of other usage. My max load is 3, which is nothing out of 32 threads.

 

[ChrisRJ]

The VPN should run on your router/firewall, no the NAS.

As to the power you need, I agree with @sfatula . Even my system would be more than sufficient for what you describe. On the other hand, your description allows for several orders of magnitude of leeway in terms of performance, if I want to be picky. Media server can mean anything from simple SMB server up to Plex with 4k transcoding. VMs can range from Home Assistant to a Kunbernetes cluster or a database server that needs loads of IOPS. If you plan nothing special, it should be ok, though.

 

[sfatula]

As far as transcoding goes, in our 2 person house, I've done 4k transcoding in software via Emby (merely as a curiosity) and it does make the Xeon work a good bit, but it handles it fairly easily. I don't know if I could do 2 at once though, I think it could. But the idea for me is the client shoudl transcode and it does, so, I never transcode unless remote or testing, and if I'm remote, no one is home so 1 user is fine. Heck, I even run Handbrake conversions on the NAS. I've even run as Raspberry Pi previously as an Emby server, and with no transcoding clients, it worked fine.

My impression from his original post was very light usage, words like light, occasional, etc.

@ChrisRJ is spot on, VPN belongs in firewall/router for a variety of reasons. I think from your comments you may be surprised what it can do.

 

[Davvo]

Go 1151 without going below Coffe Lake.

 

[Etorix]

would prefer not to deal with gfx card if feasible

TrueNAS itself has no use for a GPU. You can strike out all your bullet points about graphics. Basic needs for BIOS settings and installation are covered by IPMI on real server boards (which need not be for Xeon Scalable).
Have you looked into second-hand RAM?

LGA1700​

Too new, not that efficient; DDR5 is expensive in general, but DDR5 ECC UDIMM is super expensive.

• Mobo
  • AsusRack E3C256D4U-2L2T (DDR4)- $450
  • Supermicro X13SAE-F (DDR5) - $450

Two wrong choices: C256 is for LGA1200 Xeon E-2300; X13SAE is a workstation motherboard.

LGA 1200
Mobo

• Supermicro X12STH-F - $270

C256 chipset: uses the same Xeon E-2300 as AsRockRack E3C256D4U-2L2T above.
Xeon W-1200 are strictly for W480 workstation motherboards (much higher power than Xeon E-2300 which derives from Tiger Lake laptop CPUs).

LGA1151
Mobo

• Supermicro X11SSM-F -$170

• CPU
  • Xeon E-2124G (4C/4T) - $60
  • Xeon E-2174G (4C/8T) - $125
  • Xeon E-2226G (6C/6T) - $250

There's LGA1151 (Skylake) and LGA1151-2 (Coffee Lake) and they are NOT compatible.
X11SSM is for Skylake, and requires a Xeon E3 v5/v6, or a Core i3-6100/7100 (these do ECC!).
Xeon E-2100/2200 is Coffee Lake, and requires a X11SC_ motherboard. (Again, Core i3-8100/9100 supports ECC.)

You need to decide how many cores you want for your VMs, and how much RAM is needed.
128+ GB RAM is a strong pointer towards RDIMM, which may be 1st/2nd gen. Xeon Scalable (overkill for your stated use, but at the quoted prices you may indulge yourself), the older Xeon E5, or even a Xeon W-2100/2200 if you can find one of the rare server boards for it (Supermicro X11SRL-F, Gigabyte MF51-ES0).
With less RAM, the reasonable choice is a C246 or C256 board with matching Xeon E for LGA1151-2 or LGA1200. Which one is entirely down to availability at decent price. The last available C246 boards can now command anything from $200 to $500, so a X12STH at $270 may be a decent alternative.

 

[t3ch]

Just realized I've had this response sitting since this morning, thanks all I'll chime in in a bit

Given your very light usage, I don't see the point of going expensive other than it's shiny new and cool. Your cheapest option would be more than adequate. I built using an LGA2011-3 and average over a week is 97.5% idle, despite extensive media server/dvr usage and a lot of other usage. My max load is 3, which is nothing out of 32 threads.

thanks for chiming in. I can't seem to edit, so I appreciate you slogging through the wall of text and lack of final context, lol.

While you say that any of my examples would be adequate (which I totally concede), I guess the TLDR here is that I'm just kind of confused at what I'm missing when some items don't seem to be as popular as others. I know we had a large jump in the latest gen intels, there are lots of i3 builds, etc., but by the time you add in the expensive pieces of the equation (an appropriate server-oriented motherboard and a pile of ram), it's all generally around the same price as options with 2x/3x the cores, or access to server rdimms, etc... I'd have to go full consumer to really save a ton of money. If the answer is, "eh it's all just kinda fine, pick whatever, stop overthinking", that wouldn't surprise me.

 

[sfatula]

If the prices are close enough, go with the later gen for sure. I paid < $100 for an x10 motherboard that was new in box and very little for CPU so that made my choice. Those are often 1 off deals though. Make sure whatever CPU has a sufficient number of threads, not like a 4 core. I'd be more inclined to take whatever deal I can find on any given day. But definitely watch the memory prices, just because a MB/CPU combo is cheap doesn't mean the memory is, esp on some of the more recent CPUs. And do get a server motherboard with IPMI. For me, I like knowing if while I am in Fiji for 4 weeks, if something happens to the Truenas server I can VPN in and use IPMI to fix things if need be. That and I can hide the server in a small area with no KB or screen so IPMI becomes necessary.

 

[t3ch]

sfatula --
to clarify, each system is around the same price, but with lowered raw performance with each newer gen. i.e., a complete system with a newer 4C/4T i3 is the same ballpark as a 16C skylake xeon gold. I assumed there were power implications or some such as to why these configs might not be more common; want to make sure I'm not overlooking anything.

ChrisRJ --
you're right "VM" wasn't particularly helpful. the container/vm's won't be heavy (small windows/linux homelab, random bs) and jellyfin/plex would only need to handle single 4k transcoding. I just want to ensure I don't have to worry about performance down the line (without running a 200w idle server). The plan is for this to be a set and forget system until I can go full SSD, in however many years when that becomes feasible.

Etorix --
thanks for all the corrections (will edit when able). I'm not sure if I screwed up my notes or if I just got a bit too overzealous with all the copy/pasting.

the pricing is second hand; I've just been doing a quick check on ebay to get general pricing info. fb/craigs didn't turn much up in my area last I checked.

The VM's won't need multiple dedicated cores or huge pools of ram, though I'm not sure how to accurately quantify it. I'd planned to just get 128gb to ensure there's enough for ZFS + everything else (thought I might start with 64gb if I have to buy expensive UDIMM).

Re: the graphics, to be completely frank, I'm a linux putz and I'd just feel more comfortable having the option of plugging in a monitor if needed. I am ashamed!

definitely a lot of good food for thought there, will explore, really appreciate your detailed response.

 

[sfatula]

For server, better to have more cores, Xeon Gold over i3. Regarding power usage, my Xeon machine with drives and busy much of the day averages over the course of the day to just over 100 watts. It uses 2.5 kwH/day. Since that's about a dime of electric here (mostly solar and this is cost over 10 years/day), not a concern for me at least. I doubt the XEON has ever been 100% busy. Probably never 50% busy.

Many server motherboards have vga ports, at least Supermicro. I am only using 64GB ECC Ram and it's way more than enough for my 20 containers and heavy Emby use, along with 3 VMs including 1 Win 10. I do think you want a server motherboard even if you don't think you want IPMI.

 

[Etorix]

The VM's won't need multiple dedicated cores or huge pools of ram, though I'm not sure how to accurately quantify it. I'd planned to just get 128gb to ensure there's enough for ZFS + everything else (thought I might start with 64gb if I have to buy expensive UDIMM).

Any VM will require some reserved core(s) and RAM.
With this amount of RAM to begin with, go for RDIMM platforms. A second-hand complete server (Dell R700 series) may be the cheapest option, if you can accommodate a rack—and its potential noise.
Otherwise, make a home build with a X11SPL/i and a sub-$100(!) Xeon Gold.

In theory, you'd still need to clarify which class of NIC you want and how many SATA/M.2 (or U.2?) ports you need, including boot drive and a small SSD pool for VM/apps, but this class of platform has typically 8-10 SATA and PCIe lanes to waste.

Re: the graphics, to be completely frank, I'm a linux putz and I'd just feel more comfortable having the option of plugging in a monitor if needed. I am ashamed!

A BMC gives you basic video output from the VGA port as well as in a browser window from wherever you like. It hardly gets more comfortable than managing your server from the couch in the living room…

 

[t3ch]

For server, better to have more cores, Xeon Gold over i3.

I do think you want a server motherboard even if you don't think you want IPMI.

Well yeah exactly, that's I'm confused I don't see more people talking about these low cost cpu's! I figured there had to be a catch.

I 100% want IPMI, I am actively picking out boards with it. I've never actually used it before though so apologies if some of my comments are confusing.

With this amount of RAM to begin with, go for RDIMM platforms. A second-hand complete server (Dell R700 series) may be the cheapest option, if you can accommodate a rack—and its potential noise.
Otherwise, make a home build with a X11SPL/i and a sub-$100(!) Xeon Gold.

Rack is a nonstarter. But I think we agree on what the move might be otherwise.

In theory, you'd still need to clarify which class of NIC you want and how many SATA/M.2 (or U.2?) ports you need, including boot drive and a small SSD pool for VM/apps, but this class of platform has typically 8-10 SATA and PCIe lanes to waste.

NIC I'm not too focused on because I can just add a faster card if needed, more worried about getting the base platform right. I don't imagine being too network limited after the initial backups are uploaded (though faster than a gig is a clear bonus).

As mentioned I've got maybe 6 platter drives and 2x SSD's, so a fair number, but most of these server boards should support me from the start. If NVMe is supported I'll grab two of those as well and use the ssd's for caching.

A BMC gives you basic video output from the VGA port as well as in a browser window from wherever you like. It hardly gets more comfortable than managing your server from the couch in the living room…

Oh, derp. That makes it easy then.

 

[Etorix]

If NVMe is supported I'll grab two of those as well and use the ssd's for caching.

"If"? Typically, all x16 and x8 slots can bifurcate all the way down to x4x4x4x4 and x4x4, so you could use simple adapters to host quite a few M.2 or U.2 drives from there.
Mind what you mean by "caching" though. L2ARC has its rules, and is not always useful. SLOG is NOT a write cache, and is only helpful if synchronous writes are absolutely required.

 

[sfatula]

No "catch" to using Xeon over consumer. They are not gaming CPUs so maybe some are thinking that way, not sure, but I personally do not see a lot of discussion about consumer CPUs. Most I see are using Xeon. I suppose a Xeon doesn't have a integrated GPU (I think a few do?) so maybe some might want an integrated GPU for Plex/Emby or whatever, not realizing value of cores. Or maybe they already had a machine or motherboard and couldn't afford a new one. It can work without a ton of cores.