epyc cpu needs turbo boost turned on (if possible)

[gus4b]

Hello, do you know a way to activate the turbo boost of the cpu inside truenas scale v22.12.2 and passing thru to a windows 10 vm machine ? I am running a epyc 7d12 on a supermicro h11 ssl-i,,, the cpu is running on the base clock of 1.1ghz full time, seems a little slow... even when i run cinebench or cpu-z.
thanks,

 

[NickF]

Ok - So I have a Epyc 7302 in an H12SSL-I board so my setup is similar to yours. I cannot say I've seen similar behavior.

Under heavy Load:

Code:

root@prod[/mnt/optane_vm/test]# lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   43 bits physical, 48 bits virtual
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              2
Core(s) per socket:              16
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC 7282 16-Core Processor
Stepping:                        0
Frequency boost:                 enabled
CPU MHz:                         3075.339
CPU max MHz:                     2800.0000
CPU min MHz:                     1500.0000
BogoMIPS:                        5600.18

Under small load:

Code:

root@prod[/mnt/optane_vm/test]# lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   43 bits physical, 48 bits virtual
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              2
Core(s) per socket:              16
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC 7282 16-Core Processor
Stepping:                        0
Frequency boost:                 enabled
CPU MHz:                         1500.000
CPU max MHz:                     2800.0000
CPU min MHz:                     1500.0000
BogoMIPS:                        5600.18

Where are you getting this information that it seems to be running at 1.1ghz? From Linux CLI on SCALE or the Windows VM underneath?

 

[gus4b]

Ok - So I have a Epyc 7302 in an H12SSL-I board so my setup is similar to yours. I cannot say I've seen similar behavior.

Under heavy Load:

Code:

root@prod[/mnt/optane_vm/test]# lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   43 bits physical, 48 bits virtual
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              2
Core(s) per socket:              16
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC 7282 16-Core Processor
Stepping:                        0
Frequency boost:                 enabled
CPU MHz:                         3075.339
CPU max MHz:                     2800.0000
CPU min MHz:                     1500.0000
BogoMIPS:                        5600.18

Under small load:

Code:

root@prod[/mnt/optane_vm/test]# lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   43 bits physical, 48 bits virtual
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              2
Core(s) per socket:              16
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC 7282 16-Core Processor
Stepping:                        0
Frequency boost:                 enabled
CPU MHz:                         1500.000
CPU max MHz:                     2800.0000
CPU min MHz:                     1500.0000
BogoMIPS:                        5600.18

Where are you getting this information that it seems to be running at 1.1ghz? From Linux CLI on SCALE or the Windows VM underneath?

im having 1.1ghz in windows under trunas scale, i ran cinebench and it stays in 1.1ghz not boosting

 

[gus4b]

how can i run a benchmark in truenas scale to see if the problem is in windows 10 ?

 

[jgreco]

im having 1.1ghz in windows under trunas scale, i ran cinebench and it stays in 1.1ghz not boosting

So you do understand, in a virtualization environment, if you are passing through lots of cores, not only are you not going to be turbo on all those cores, but also the virtualization takes out its own tax and you are not going to get what it says for MHz on your CPU's box either, right?

Cinebench above says you have a 32-core processor but you are running 60 threads at 1.1GHz. This seems relatively reasonable. The 7D12 base clock is 1.1GHz and if you are running many of them at the same time, it may not have enough headroom to turbo anything.

 

[gus4b]

yes i know the tax penalties, but the turbo on all core should exist even if in a small amount, looks like its locked not beeing able to turbo boost... if i want to use the computer for a nas and meanwhile leaving 2 cores (intentionaly) for truenass and the rest (30 cores) to render 3d images such as 3dsmax with vray etc, how is that possible.?

the specs on my cpu the epyc 7d12 are 2.4ghz turbo on all cores by default.

my onle concern here is to get a return on my investment, uniifying as a nas server and as a render server if you can follow me up ?

 

[jgreco]

What mechanism does Windows use to gather this data? How do you know you can rely on it?

Perhaps it would be worthwhile to gather independent benchmarks on both virtualized and bare metal configs to see what the reality is.

 

[gus4b]

yes i did it already, the difference in a windows native is 30% faster i tested it in 3dsmax with vray identical instalations same windows and same archive.

 

[HoneyBadger]

Based on my understanding of some search results, the EPYC 7D12 is a CPU that was OEM for Dell servers, and has a relatively low TDP (thermal design power) of 85W.

According to a user at ServeTheHome, the all-core turbo is designed to limit the chip to 1.1GHz when AVX2 instructions are in use. Using a custom KVM/qemu command line to mask the AVX2 feature may allow the chips to boost higher, but at the cost of that feature.

https://forums.servethehome.com/ind...me-what-is-epyc-7d12.28654/page-2#post-373859

I actually have one of it running in my Homelab.
The CPU itself is actually quite custom and has the following specs:
32 Core
1.1GHz base with 3.0GHz boost clock.
85W TDP (125W limit)
only 4 Memory channels, the others are fused off or non existent.
CPU Multiplier is unlocked and can be overclocked. But due TDP Limit, it is not useful
No multi CPU support
Infinityfabric clock is locked at 1200MHz
There are some memory controller limitations with this CPU:
SK Hynix memory is not supported and memory self test fails.
DDR4-2133 memory does not boot.
CPU can only run with one DIMM per channel because of TDP limit.
Also I played a bit around and found out that XFR can be enabled as well as other OC and TR only stuff.
This it requires huge BIOS modifications and I did it only experimental.
I think a Threadripper IO DIE is used but with 128PCIE lanes enabled (The IO DIE internal sku is the same as the TR one not like Epyc).
Also it worked out of the box on a Supermicro H12SSL-C mainboard with bios version 2.3.

At 85W cTDP and PPT the all core clock is 2.3GHz(1.1GHz with AVX2)
At 105W cTDP and 125W PPT the all core clock is at 3.0GHz(2.4-2.7GHz with AVX2)
When the cTDP is set higher than 105W the performance tanks in most Benchmarks.

 

[jgreco]

yes i did it already, the difference in a windows native is 30% faster i tested it in 3dsmax with vray identical instalations same windows and same archive.

Then I'm not really convinced that anything wrong is happening here. You say that your CPU is "2.4GHz turbo on all cores" but I can't really find any specs for that CPU. Typically a turbo boost results on focusing of resources on a small number of cores. When you use a high number of cores in a virtual manner, the hypervisor spends a lot of time in busywait for exclusive access to all the requested cores. That could easily account for the 30%. If your CPU has a baseline of 1.1GHz, it's unlikely to be able to thermally tolerate and sustain (for ex.) twice that on all cores.

Basically, I think you're misunderstanding something here, but I'm not clear on exactly what it might be. The numbers don't make sense.

 

[NickF]

I'd bet there isn't an actual problem and Windows is mis-reporting the frequency to you.

What settings do you have here?

Do you have VirtIO Guest drivers installed?

Index of /groups/virt/virtio-win/direct-downloads

If you run

Code:

lscpu

In the SCALE CLi while you have your Windows box under load, what does it say? Try multiple benchmarks, not just Cinebench.

But as @HoneyBadger has said, it may be your specific CPUs expected behavior as well with the workload you are simulating.

 

[gus4b]

this cpu single boost clock is 3.0ghz all core is 1.7 - 2.4ghz... base clock is 1.1ghz...

AMD EPYC 7D12

Descriptions of AMD EPYC 7D12: frequency, number of cores, socket, TDP, max. temperature, supported memory, cache and others.

www.chaynikam.info

on the atach same passmark test with windows baremetal x truenas / windows 10

any chance of believing now ?

 

[NickF]

Please re read my last response. What does Linux say the cpu frequency is while you are running the benchmarks?
I still believe number 1 that AVX2 workloads are part of your problem and number 2 that windows is misreporting the cpu frequency.

 

[gus4b]

here my windows 10 vm config

 

[gus4b]

im running both tests with remote desktop connection in windows

 

[jgreco]

So a CPU with 64 threads and a PassMark rating of 40459, but a single thread rating of 1870.

By my math, 32 CPU's at 1870 plus 32 threads at a handicapped 50% of 935 puts us around 90,000. Your CPU is rating less than half that. Something's not adding up. I'm fine with believing it's @NickF and his AVX workload theory, but that's only one possibility. I'd just as readily believe misreported numbers.

any chance of believing now ?

Not really. The CPU Mark tests seem to suggest a heavy virtualization tax, as I said before, likely due to an overallocation of cores/threads.

 

[gus4b]

other benchmark

 

[gus4b]

lscpu print screen

 

[gus4b]

Please re read my last response. What does Linux say the cpu frequency is while you are running the benchmarks?
I still believe number 1 that AVX2 workloads are part of your problem and number 2 that windows is misreporting the cpu frequency.

 

[gus4b]

print linux under benchmark stress test