Ryzen vs Xeon

[Davvo]

Toshiba laptops were indeed amazing.

 

[da_da]

I recommend going with AMD.

 

[Ericloewe]

I recommend going with AMD.

Well, that's hardly a persuasive argument. And I'm saying that as the guy who's mostly specified AMD for new servers for three years now.

 

[jgreco]

Well, that's hardly a persuasive argument. And I'm saying that as the guy who's mostly specified AMD for new servers for three years now.

So, persuade me.

 

[Ericloewe]

Well, tell us what you're looking to accomplish. For me, the shining example is getting to choose between a quad-socket monstrosity from Intel and a two-socket mainstream platform from AMD.
I save rack space, idle power and cash. In return, I give up bragging rights about needing a four-socket machine (yay?).

 

[Whattteva]

I have been digging for a while now and feel like I am being blind here. I am planning a home server build in 2023, this will boot proxmox with a VM for TrueNAS, the high level requirements are:

• TrueNAS - ~100tB available data for media storage (anything up to and including 4k blu ray rips uncompressed)
• Plex Media Server (will add a GPU to the build for hardware encoding)
• Pi hole/Graphana etc. (other relatively lightweight tasks)
• Home assistant
• a Linux VM playground machine
• ability to scale to add more machines as and when I think of something useful or want to have a play

(I am here as Truenas is my most important part here for reliability and longevity)

My issue is there appear to be 2 routes I can go, Xeon E v2388g or Ryzen 7900. (there are some other options too i.e. AMD 5900X or something)
The former appears to be more expensive and comes in at half the cores
The latter is a consumer chip with more nuanced support for things like ECC memory

It seems most people go the Xeon route for their builds today, so I guess I am just hoping for a summary as to why (of they go completely consumer parts, no ECC etc.)

I will chip in here because I've built a server that has basically similar use case as yours (Proxmox + TrueNAS VM) though I don't run a GPU because while I do run Jellyfin, I have no need for transcoding and I only have 12 TB worth of data.

As much as I generally like to go with the underdog (AMD), I decided to go with Intel for a few reasons:

• ECC is kind of a hard requirement for me and I wanted official documented support over unofficial maybe'ish support out of the Ryzen's.
• I wanted to be able to have the ability to have 1TB RAM and not have to pay too much.

I also wanted a more mature and stable platform (it is a server after all) so I didn't go for latest and greatest. It also comes with the advantage that it costs way cheaper too!

The platform I decided to go with is in my sig (Supermicro X11-SPi-TF + Xeon Silver 4210T (Cascade Lake) + 224GB RAM (currently). I got mostly used parts for a grand total of ~$1500. I'm very pleased with the system. It runs cool and quiet (quieter than my gaming PC surprisingly). It has plenty of spare CPU/RAM capacity even with 13 VM's, 9 of which run 24/7. It currently serves as my router, NAS, media/transmission server, and my personal daily driver machine.

I suppose my use case is simpler than yours due to having no need for transcoding, but hopefully this will at least be a little help.

 

[da_da]

Well, that's hardly a persuasive argument. And I'm saying that as the guy who's mostly specified AMD for new servers for three years now.

I prefer AMD, since most intel architecture is in the business of Sharing when it comes to RAM/Memory and CPU. It's always best to separate all this and let the multi threading beasts work their magic... Again, not here to say why some apps are optimized for intel chips and run faster there... :)

 

[Ericloewe]

I prefer AMD, since most intel architecture is in the business of Sharing when it comes to RAM/Memory and CPU.

I don't follow. The memory model across the x86 PC landscape is the same, with fairly minor differences (more cache here, fewer levels there, HBM as L4 are all examples, and all pretty minimal from an architectural point of view).

 

[da_da]

Oh I am sorry, you were talking about the dual Socket as opposed to four. In those scenarios AMD has each CPU with its own RAM as opposed to intel which is shared.

 

[Ericloewe]

No, Intel CPUs in four-socket systems get their own memory, too. It's not too different from dual-socket systems, just scaled up. Of course, all memory can be accessed by any CPU on both Intel and AMD, at the cost of extra latency (and bandwidth constraints).

 

[rbentley100]

I will chip in here because I've built a server that has basically similar use case as yours (Proxmox + TrueNAS VM) though I don't run a GPU because while I do run Jellyfin, I have no need for transcoding and I only have 12 TB worth of data.

As much as I generally like to go with the underdog (AMD), I decided to go with Intel for a few reasons:

• ECC is kind of a hard requirement for me and I wanted official documented support over unofficial maybe'ish support out of the Ryzen's.
• I wanted to be able to have the ability to have 1TB RAM and not have to pay too much.

I also wanted a more mature and stable platform (it is a server after all) so I didn't go for latest and greatest. It also comes with the advantage that it costs way cheaper too!

The platform I decided to go with is in my sig (Supermicro X11-SPi-TF + Xeon Silver 4210T (Cascade Lake) + 224GB RAM (currently). I got mostly used parts for a grand total of ~$1500. I'm very pleased with the system. It runs cool and quiet (quieter than my gaming PC surprisingly). It has plenty of spare CPU/RAM capacity even with 13 VM's, 9 of which run 24/7. It currently serves as my router, NAS, media/transmission server, and my personal daily driver machine.

I suppose my use case is simpler than yours due to having no need for transcoding, but hopefully this will at least be a little help.

Perfect summary for my small brain to follow :) thank you and I think also encapsulates the ideas elsewhere in the thread.

Intel, better history at being predictable and stable, therefore also better supported in the space that values stability and predictability.
AMD has shiny features but harder to pin down what combo they all work on etc. (i.e. the maybe'ish ECC support you reference)

If you want 128 GB RAM you may as well look into "real" Xeons, i.e. those which use RDIMM rather than UDIMM: More RAM capacity for a lower price—which will offset the higher price of the platform.
Second-hand/refurbished Xeon Scalable, or Xeon W-2000 if you can find a Supermicro X11SRL-F or X11SRM-F.
Or EPYC on the AMD side of things.

Does sound like going up into more real Xeons might be the best play, just means I need to go learn that naming scheme to work out what generation/tier I am looking at :) I dont want to go too old and spend a small pot of gold each day running the thing, or too new and paying through the nose for the new chip.


Also I do enjoy a good ol' AMD vs Intel debate, its been a good decade since I have witnessed one of those! so thank you for the background everyone - this post has got a lot more input and genuine help than I expected!

 

[Whattteva]

Intel, better history at being predictable and stable, therefore also better supported in the space that values stability and predictability.
AMD has shiny features but harder to pin down what combo they all work on etc. (i.e. the maybe'ish ECC support you reference)

Well not exactly in that way. What I mean is that AMD doesn't intentionally disable ECC on their chips the way Intel does, but they also don't officially support it so it's one of those "your mileage may vary" type of situations and if that's how it's going to be supported, I may as well opt for cheaper non-ECC chip. I just feel that maybe'ish your mileage may vary type of support kinda' defeats the purpose of going ECC in the first place (enterprise-grade predictability and reliability). No sane enterprise will use a feature that "most people think will work" without the manufacturer's express support.

Of course, I'm just referring to their Ryzen chips. It's different for their server-grade EPYC and ThreadRipper chips which actually DOES list ECC support officially clearly on their product pages. Truth be told, if AMD announced tomorrow that all Ryzen chips have ECC support, sign me up for an updated Ryzen build.

TL;DR: For ECC to be viable for me, it has to have first-party official support denoted clearly on their product page.

 

[Arwen]

Speaking of learning naming scheme, that is one thing I dislike about Intel. Finding an Intel processor with a specific set of features is painful. They have improved with Intel Ark, but I remember in the old days hunting & pecking trying to find something suitable. Then have to see what it's price at retailers was, and if even available.

Not saying AMD is any better. In fact, AMD removed a really nice CPU filter page, back before Ryzen & Epyc. That is how I found my Athlon upgrade from Sempron on my old desktop.

 

[Ericloewe]

Speaking of learning naming scheme, that is one thing I dislike about Intel. Finding an Intel processor with a specific set of features is painful.

In the Xeon E5 days, things were complicated, but not excessively so. When Xeon Scalable arrived, they decided to make the portfolio completely impossible for anything but the most dedicated consultants to understand.

 

[jgreco]

When Xeon Scalable arrived, they decided to make the portfolio completely impossible for anything but the most dedicated consultants to understand.

Credit where hard work was done and credit is due, STH has a pretty good breakdown of the dozens of Scalable 3rd Gen parts.

3rd Gen Intel Xeon Scalable Ice Lake SKU List and Value Analysis

The 3rd Gen Intel Xeon Scalable Ice Lake SKU List and Value Analysis shows which parts are the best values and what to look out for

www.servethehome.com

 

[Whattteva]

3rd Gen Intel Xeon Scalable Ice Lake SKU List and Value Analysis

The 3rd Gen Intel Xeon Scalable Ice Lake SKU List and Value Analysis shows which parts are the best values and what to look out for

www.servethehome.com

Code:

Q – Liquid Cooling Optimized

I'm assuming that's code for "Warning, you're gonna need a firehose to cool this badboy of a space heater".

 

[jgreco]

Code:

Q – Liquid Cooling Optimized

I'm assuming that's code for "Warning, you're gonna need a firehose to cool this badboy of a space heater".

Q - Drool Cooled

 

[oncdoc]

Ok I'll give this a shot. This is a HUGE topic and I cannot do it 100%-accurate justice in less than a hundred pages, so all of those of you who would pick nits, nuts to you. This is just to show a bigger picture which is not immediately obvious to many people. If I miss a major point, you are of course welcome to comment.

Throughout the '90's, Intel and AMD were in a battle for the hearts of users, and you could toss either an Intel, AMD K6, or Cyrix "Pentium-class" processor, all of which were socket-compatible. These were mostly all desktop-targeted systems but could also be used for small servers if you had a server-oriented board.

Intel's Xeon and and its predecessor the Pentium Pro designs were targeted at servers and workstations; these were never socket-compatible with anything else. As we entered the 2000's, dual and then quad core CPU's became a thing, as did different designs for memory controllers (MCH, northbridge, etc) where it quickly became clear that the next evolutionary target was going to be integrating the memory controller into the CPU itself, and designing a quick interconnect between cores on the CPU or even between CPU's.

AMD evolved its systems in a somewhat different way, with the Athlon, Duron, and Opteron lines; the latter being AMD's server-oriented Xeon-like offering. You could get server-y dual Athlon boards such as the Tyan S2882, but in reality it was more a nod towards the workstation users, and while it did offer a BMC and onboard SCSI and other server-y like things, it was also poorly designed and gave some of us endless problems. I had units where the PCI bus had to be clocked down in order to be stable, for example.

The Opteron represented a very popular platform towards the end of the aughties and it WAS aimed squarely at servers. Some people thought that AMD was going to become dominant in the server world, but it was not to be. Instead, what happened was Intel made a bunch of advances that resulted in the Core and Xeon lines we know from the early 2010's, and one of the genius things they did was they leveraged their designs so that some i3 could be used as a low end server processor in some E3 Xeon boards. This killed AMD for some of us, because there was really little consistency in AMD platforms; you were always getting crappy Broadcom or other alternative ethernet chipsets, random SATA/SCSI/SAS controllers, etc. Intel plowed their roads clear with reference designs that offered a large degree of consistency, and during almost the entirety of the 201X's, Intel was a clear winner, both for desktop and server.

Meanwhile, AMD was licking its wounds, but they have some really smart people over there, so they re-evaluated where they were going, and aimed Ryzen at desktop/gaming, and EPYC at servers. There is no doubt that some of these are killer products, but they are targeted at certain market segments. There is relatively little call for a low-core-count "server" platform for Ryzen; Ryzen core counts are mostly driven by gaming and desktop demands, so it remains focused on lower core counts and relatively little I/O (even if the threadrippers offer 64 cores and a hundred PCIe lanes for six thousand dollars).

Most server needs these days involve virtualization, large I/O loads, and heavy consolidation ratios, so AMD has focused on making EPYC competitive there. They're in a battle for their life, though, and actually, so is Intel. ARM has made minor inroads into markets once exclusively the realm of Intel/AMD, and that trend is likely to continue. AMD is unlikely to put out reference designs for Ryzen-based servers because it would undercut EPYC, and because they're too busy trying to evolve their existing platforms to maybe recapture more of the intended market share in each segment. There's not a whole lot of demand for Ryzen based servers, which would be "smaller" servers.



In short, it's because Ryzen is a desktop targeted platform.

love the one sentence summary at the end.