Intel Xeon Quick Sync

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
Hey All,

Currently have an X10 system which has been running for many years and has been such rock stable. I also have a separate Blue Iris machine which I recently setup. I am considering building a system with ESXI to run both Truenas Core and Blue Iris to save on power cost. Currently we have some of the highest energy costs in the country and my cost per kwh just doubled on Jan 1st. This is just a home media/data server currently, so it does not get that much activity.

I will need Intel Quick Sync for Blue Iris and it needs to handle x265, which is Gen 6th and newer I believe. I currently only have 1 4k camera, but will be adding more in the future and looking to keep video files sizes down when possible. I also could use the 1050 Ti, although it seems Blue Iris is more stable with Quick Sync, not sure about that though?

So looking for some recommendations on Intel Xeon CPU's that have Quick Sync. It would be nice to stick with older hardware like the X11, trying to keep costs down and X12 is still very new. Something with at least 64GB of memory support for future expansion or even 128GB. I would also want the ability to add a 10GB NIC to the machine as I am upgrading to a Brocade ICX-6450-48P switch so I would like to take advantage of the SFP+ 10GB links for the server.

I was also throwing around the idea of maxing out my current X10 board with 32GB of memory and running ESXI on that and use a NVIDIA 1050 TI which I already have for Blue Iris. But this would be pushing the limits with the memory I suspect. Although currently I run 8 jails and 10 different apps on 24GB of memory with no issues and it has been running for 7 years at this point. I would consider this a stop gap until upgrading at a later date though, as the cost would be almost nothing.

Storage wise I doubt I would go over 100TB in the future. I know I am at the limits of the 24GB and even 32GB of memory already so this is also why I am considering upgrading for future expansion and 64GB of memory.

Current System:
Supermicro X10SL7-F
Intel Xeon E3-1230V3
24GB Crucial DDR3 ECC
5x14TB WD Drives
NORCO RPC-4020
TrueNAS-13.0-U3.1
 

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
Intel Xeon's generally don't come with QuickSync, but you can use Intel Ark to find this kind of stuff.

 

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
Intel Xeon's generally don't come with QuickSync, but you can use Intel Ark to find this kind of stuff.


Yeah that is why I was asking here as I know there is very limited Xeon's that have Quick Sync, so I was wondering if anyone had any experience with certain models. Also not sure how well NVIDIA GPUs work vs Intel Quick Sync with Blue Iris when passing it through ESXI. Maybe I would be better off just using the 1050 TI, which would give me many more options.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
So looking for some recommendations on Intel Xeon CPU's that have Quick Sync. It would be nice to stick with older hardware like the X11, trying to keep costs down and X12 is still very new. Something with at least 64GB of memory support for future expansion or even 128GB. I would also want the ability to add a 10GB NIC to the machine as I am upgrading to a Brocade ICX-6450-48P switch so I would like to take advantage of the SFP+ 10GB links for the server.

This was literally just the previous message I wrote in this thread:


X12 is now a generation old, as X13 has just been released. The Supermicro X12ST class is intended for low end servers and perhaps workstation-ish type stuff, and the E-23xx CPU lineup includes a whopping two models (I think) with iGPU support and QuickSync. Primarily the E-2388G CPU. Typically the heavy-hitter Xeons have not included iGPU or QuickSync; I can't think of any exceptions in the E5 lineup. The E3's, being workstation-oriented, usually had iGPU's on part numbers ending with E3-xxx5, so for example the E3-1585 v5 has Intel Iris Pro P580 and QuickSync. That's only Skylake though.

quicksync.png


The E-2388G is Rocket Lake, the E-2288G is Coffee Lake. I didn't bother learning/retaining whatever notation Intel may have used for iGPU units between Skylake and Coffee Lake, so you're a bit on your own there. And I'm not sure what "x265" is, perhaps H.265? If so, isn't that a subclass of HEVC? If so, you might be able to get by on a Skylake, which should be relatively readily available on the used market. Do your own research, though. You don't want my advice before I've even had any caffeine.
 

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
This was literally just the previous message I wrote in this thread:


X12 is now a generation old, as X13 has just been released. The Supermicro X12ST class is intended for low end servers and perhaps workstation-ish type stuff, and the E-23xx CPU lineup includes a whopping two models (I think) with iGPU support and QuickSync. Primarily the E-2388G CPU. Typically the heavy-hitter Xeons have not included iGPU or QuickSync; I can't think of any exceptions in the E5 lineup. The E3's, being workstation-oriented, usually had iGPU's on part numbers ending with E3-xxx5, so for example the E3-1585 v5 has Intel Iris Pro P580 and QuickSync. That's only Skylake though.

View attachment 62847

The E-2388G is Rocket Lake, the E-2288G is Coffee Lake. I didn't bother learning/retaining whatever notation Intel may have used for iGPU units between Skylake and Coffee Lake, so you're a bit on your own there. And I'm not sure what "x265" is, perhaps H.265? If so, isn't that a subclass of HEVC? If so, you might be able to get by on a Skylake, which should be relatively readily available on the used market. Do your own research, though. You don't want my advice before I've even had any caffeine.

The X12 is quite a bit more expensive then the X11, about 2x it seems, mainly the cost of the CPU. At least compared to the used market. Yes sorry I was referring to H.265 or HVEC decoding, which technically goes back to Skylake, although Coffee Lake and up seems to have the full HVEC support.

It looks like the E-2356G supports Quick Sync and has an iGPU. Still 6C12T part as well and much cheaper then the E-2388G.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
It looks like the E-2356G supports Quick Sync and has an iGPU. Still 6C12T part as well and much cheaper then the E-2388G.

I'm not seeing any significant availability; the ARK MSRP is like $350 but I only see a few available from distrs for about $450.

The E-2324G is a 4c/4t part with MSRP of $250, and I see a few tray parts for about that, or a boxed part from CDW at $276. So if you aren't feeling constrained by the lack of cores, that might be an option for you.

I did discuss the shortage of parts with a few vendors including iXsystems last year, and indications at the time were that supplies are highly constrained with no easing in the short term. I was able to buy an E-2388G at the time via Supermicro, but it was a BTO dealie (ugh) and it took about four months from P.O. to box-on-dock. I'm not seeing retail purchase options for most of these except for the E-2324G from CDW.

From my perspective, the E-2388G is fine because when you take the TCO of the server and then divide that up by the compute capacity it provides, the E-2388G is far cheaper.

E-2388G at $3650 divided by 25.6 works out to $143 per GHz.
E-2324G at $3300 divided by 12.4 works out to $266 per GHz.

It probably helps that it is a tiny CPU in the grand scheme of things.
 

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
@jgreco What kind of energy use does your X12 system use at idle? Trying to get an idea of energy use compared to my X10 E3-1230V3 setup I currently have. Currently my system idles around 60-70W with 4x 14TB 5400 RPM drives and no other PCIE devices. Wondering if I can lower overall idle energy use, while also upgrading to newer hardware. Our energy rates doubled on Jan 1st.

Also I have a separate Blue Iris box and a spare GTX 1050 TI. So I am throwing around the idea of upgrading from Core to SCALE and running a win 11 VM for BI to eliminate the second machine. Currently I have 24GB of memory in the system, but I can pick up another 8GB to max out to 32GB for cheap. This would give me another 8GB of memory to dedicate to BI and then 1050 TI would handle the decoding. The system is just a home server, so little use and the 24GB has been running great with a bunch of jails.

Also maybe you can confirm if GPU passthrough is possible with the X10SL7-F and SCALE, as the above would not be possible if its not.

I would then upgrading down the road to X12 when prices go down and availability is better.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
What kind of energy use does your X12 system use at idle?

This one time, I'll be nice. :smile: This unit is a Supermicro SYS-510T-WTR, which is a dual-PSU chassis, along with an E-2388G (95W max TDP) CPU, loaded with four sticks of DDR4-2933 UDIMM, four installed of six chassis fans, a Samsung 970 Evo Plus, and one PSU energized.

power.png



So you can see there where I yanked the GPU and PERC H740p out. It claims 64 watts. It is sitting at the NSH prompt because it has no boot device, so it might not be as idle as possible.

The E3-1230 v3 is about a 3600 Geekbench while the E-2388G is around 8000. Twice as many cores and up to four times the memory. The newer system probably uses a little more power, but is capable of at least twice the work and also has the iGPU available.

has been running great with a bunch of jails.
throwing around the idea of upgrading from Core to SCALE

SCALE doesn't do jails, just so you know. You would want to set your jail up on TrueNAS CORE. You can do PCIe passthru but it isn't as polished on CORE.

Hope this is somewhat helpful. I know it isn't exactly what you wanted to hear. If you had additional workload that you could toss on the machine, it might be a clearer win.
 

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
The E3-1230 v3 is about a 3600 Geekbench while the E-2388G is around 8000. Twice as many cores and up to four times the memory. The newer system probably uses a little more power, but is capable of at least twice the work and also has the iGPU available.

Yeah not surprised with the performance difference since it has twice the core count, I am actually surprised its not even higher with the newer tech, nvm the increase core count.

SCALE doesn't do jails, just so you know. You would want to set your jail up on TrueNAS CORE. You can do PCIe passthru but it isn't as polished on CORE.

I did not think Core had GPU pass through for a Windows VM.

So you can see there where I yanked the GPU and PERC H740p out. It claims 64 watts. It is sitting at the NSH prompt because it has no boot device, so it might not be as idle as possible.

The energy use is impressive for the core count.

Outside of Blue Iris everything else that I run on my server works great on the 1230V3, so I really do not think I would get much benefit from the newer CPU architecture. There is a lot of idle time, so I figured mise well use the CPU cycles for Blue Iris and get rid of that box. Also gives me an excuse to play with SCALE.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
Yeah not surprised with the performance difference since it has twice the core count, I am actually surprised its not even higher with the newer tech, nvm the increase core count.

In the last decade, probably longer, I've never noticed a significant difference in performance from "newer tech". I know many people like to crow about how the latest CPU is so hot, but it's so not.

Here's me whining about this more than five years ago:

Bent pins, design errors, etc.

Honestly, the generation-to-generation improvement of CPU's since Sandy Bridge are quite modest. Let's review the last 5 years.

Sandy Bridge E3-1230 v1 - 3.2GHZ, 8MB - Geekbench ~11000

Ivy Bridge E3-1230 v2 - 3.3 GHz, 8MB - Geekbench ~12000 - note MHz increase

Haswell E3-1230 v3 - 3.3 GHz, 8MB - Geekbench ~13000

Haswell Refresh E3-1231 v3 - 3.4 GHz, 8MB - Geekbench ~13500 - note MHz increase

Broadwell v4 - effectively skipped(!)

Skylake E3-1230 v5 ~ 3.4 GHz, 8MB - Geekbench ~13000

All of these CPU's are effectively in the "entry level" ~$220-240 price range for the Xeon, so I deem them approximately equivalent.

My perspective is that there haven't been any significant changes since Sandy, except for some gradual evolutionary improvements, and finally with Skylake, the big RAM jump. These new parts do not consume "significantly" less power for meaningful values of the word; I benched a complete small Sandy system idling at 44 watts in 2011.

When you start to consider that, then all of a sudden it does not seem so urgent to jump on the "latest and greatest" bandwagon.

Bear in mind that those are earlier version Geekbench numbers. Geekbench5 for an E3-1230 is about 3000, released in 2011 Q2 (almost 12 years ago). So two of those would equal eight cores at about 6000; the E-2388G is only about 8000.

It's probably worth noting that Geekbench is a bit skewed in that it looks more at a mix of results rather than picking the fastest compute problems. PassMark will give you a different answer, probably more like 4:1, but for all the years I've heard "the new model's 30% faster!" ... here, and here, for example, you'd think modern CPU's should be so much faster.

I did not think Core had GPU pass through for a Windows VM.

Is GPU passthru special in some way? I was under the impression that it's just PCIe passthru. It looks like CORE does PCIe passthru but that's just observation of what seems to be there. I don't do any serious VM stuff on TrueNAS. Or much of anything with GPU's. Something for you to explore. :smile:
 

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
Is GPU passthru special in some way? I was under the impression that it's just PCIe passthru. It looks like CORE does PCIe passthru but that's just observation of what seems to be there. I don't do any serious VM stuff on TrueNAS. Or much of anything with GPU's. Something for you to explore.

According to this its technically not supported.


t's probably worth noting that Geekbench is a bit skewed in that it looks more at a mix of results rather than picking the fastest compute problems. PassMark will give you a different answer, probably more like 4:1, but for all the years I've heard "the new model's 30% faster!" ... here, and here, for example, you'd think modern CPU's should be so much faster.

Yeah its more like 7-10% at most between generations, so when you get to 4-5 gen newer its when you notice the difference.

I just don't run anything really heavy in my jails for the most part. Even with Emby most of what I watch is at home so no transcoding is required. So it makes it hard to justify $1000 upgrade, with very little benefit. Over the years I found jails to be very efficient and generally was able to find freebsd versions of all the different apps I use so I did not need any VMs., hence less overhead.

I also still prefer to keep my pFsense box separate, don;t like the idea of a single point of failure.

Truenas Core has also been rock stable over the years, I have been with Truenas/Freenas since version 8, so been through a lot and it has come a long way.

My goal now has been trying simplify and automate everything I can, the less maintenance the better. i heavily invested in Pushover and setup notifications for everything so I don;t have to baby sit my equipment, apps, automatons etc.
 

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
So I am throwing around the idea of upgrading from Core to SCALE
I'm not trying to pull your leg on this, but it kinda' triggers me (just slightly) whenever I see this framing and I see this time and time again in the forums here or on Reddit. I really wonder who it was (must be some friggin' YouTuber) that first popularized this concept that SCALE is an "upgrade" to CORE. They're different products that have different design goals. And that is true whether we are talking about CORE vs SCALE or the underlying OS's (Linux vs FreeBSD). Each one has pros and cons and tradeoffs. NEITHER is a straight up "upgrade" to the other. CORE has an edge in stability and performance (particularly in memory management and network stack) for pure NAS functions. SCALE has the advantage in HW support (particularly newer gamer gear types), virtualization, and apps (TrueCharts). Though I'm starting to doubt that last part being an advantage seeing all the "apps failing to start" posts on the forums over the last week or so.

Personally for me, I don't really see the point of SCALE because NAS is what I need out of TrueNAS. When I need virtualization, I use Proxmox and when I want additional apps, I install it directly on a jail (or rarely in a Linux VM if there's no FreeBSD port). So SCALE really doesn't offer much of a value (if at all) for my use case.
 
Last edited:

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
Personally for me, I don't really see the point of SCALE because NAS is what I need out of TrueNAS. When I need virtualization, I use Proxmox and when I want additional apps, I install it directly on a jail (or rarely in a Linux VM if there's no FreeBSD port). So SCALE really doesn't offer much of a value (if at all) for my use case.

In my case I am trying to simplify my setup, while also saving on energy. Energy costs in my area are some of the highest in the country and they just doubled last month.

Currently I run everything on CORE in jails, but unfortunately Blue Iris only runs on Windows and its really hard to beat the features for security cameras. Currently I have BI running on its own system and would prefer to virtualize it on Truenas if possible to eliminate that system.

In a home environment I really have no need for a hypervisor since I am not running enterprise critical apps. To me a simple Jail/Container with low maintenance would be ideal.

With SCALE I could run all the existing apps I do currently in my Jails and then also spin up a Windows VM for BI.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
According to this its technically not supported.

One line in a two year old document would not be sufficiently convincing to me. GPU passthru, especially for non-display purposes, is generally an "it works or it doesn't" type thing.
 

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
In the last decade, probably longer, I've never noticed a significant difference in performance from "newer tech". I know many people like to crow about how the latest CPU is so hot, but it's so not.
I feel the same way. Basically, once they've hit a ceiling in clock speeds, generational performance leaps basically started to stagnate.
 
Top