Understanding VM CPU settings

[Louis2]

Despite reading a couple of things related to the VM CPU-settings, I must admit I am still ^completely lost^.

At VM setup CPU-section you can assign:
- Virtual CPU's
- Cores
- Treads
And you define the CPU-mode. That seems to be the easy one, where I assume that that should normally be ^Host Pass through^

Coming back on the hard part "Virtual CPU's, Cores and Treads" Looking at the physical side of TrueNas scale:
- it has one or more physical CPU's (in my case 1)
- each having one or more core's (in my case 8)
- each core can handle one or more threads (in my case 2)

But than .... what for the hell are "virtual CPU's" !!??
And what is the relation between VM-Cores to Physical-Cores !?
And between "VM-threads to Physical-treads" !?

I also assume that:
- TrueNas Scale itself should always have at lease one core !!???????
- some how the physical CPU can in my case not deliver more that 8 cores and 16 threads
- that if a VM is running, it can never use more than the assigned resources
- that if a VM is not activated by the scheduler, it does not use any CPU-resource
- than it seems to be that a VM-is not made active by the scheduler, if the number of requested resources is not available at that time
(I think)
- how to prevent the scheduler to assign all CPU resources to VM's not leaving a resource for TrueNas Scale itself !?

So I am really lost especially related to Virtual CPU's which I can in no way map on the physical CPU.
As example what is happening when I assign two virtual CPU's, where there is only one physical CPU ???????

 

[sretalla]

what for the hell are "virtual CPU's" !!??

CPUs that are assigned to a Virtual Machine. It doesn't necessarily need to match your hardware as KVM can tell the guest that you have 2 CPUs with 1 core each and assign the jobs that come for those CPUs to cores on your hardware's single CPU.

You can assign more than one if you wish, but ultimately, a lot of OSes don't support multiple CPUs very well, so best to keep that at 1 and set cores/threads only in a VM.

Short answer, forget about them and set it to 1.

how to prevent the scheduler to assign all CPU resources to VM's not leaving a resource for TrueNas Scale itself !?

That can't really happen, but oversubscribing your hardware isn't recommended in hypervisors like KVM (you need VMware or Xen if you want to do that kind of thing)

As example what is happening when I assign two virtual CPU's, where there is only one physical CPU ???????

As long as the ultimate thread count (taking into account CPUs x Cores x Threads) doesn't exceed what you have, it doesn't matter.

 

[Louis2]

Yep, but that is strange ....

My CPU has 8 core's and each capable of 2 threads. So that makes '16 concurrent processes' at a time.

But your definition does not really clarify things. That sounds if all those settings do not matter at all !!! ??? :
4 - 4 - 4 equals 1 - 16- 1 equals 1 - 8 -2 etc.

It leaves lots of questions open

 

[sretalla]

4 - 4 - 4 equals 1 - 16- 1 equals 1 - 8 -2 etc.

Nope.

4 x 4 x 4 = 64

1 x 16 x 1 = 16

1 x 8 x 2 = 16

 

[sretalla]

And what is the relation between VM-Cores to Physical-Cores !?
And between "VM-threads to Physical-treads" !?

Maybe it will help to answer these questions...

No specific relationship unless you lock it to specific cores with the setting:

Optional: CPU Set (Examples: 0-3,8-11)​

Specify the logical cores that VM is allowed to use. Better cache locality can be achieved by setting CPU set base on CPU topology. E.g. to assign cores: 0,1,2,5,9,10,11 you can write: 1-2,5,9-11

 

[Louis2]

sorry 4 x 4 x 4 was intended as 4 x 4 x 1 = 16

I really do not understand any thing!! Related to these settings

If this whole thing is true, those three parameters are nonsense !!
And one could replace them.
What I can not believe.

 

[sretalla]

How do you answer the question:

What do you want your Virtual Machine to show when you look at the CPU, Core and Thread counts?

It's a question that all guest operating systems need to have answered before they will work as if they ran on hardware (which must have real CPUs with fixed numbers of cores and threads).

If you don't have those settings (like VMware doesn't), all you have is vCPUs, which equate to CPUs with one core and one thread each... which you can assign even beyond the number of your physical threads and is managed by the hypervisor to ensure oversubscription can work if all machines aren't fully using all assigned resources (for too long).

I don't see what's not to understand here.

Virtualization is 100% about doing exactly this... taking some hardware, abstracting it and presenting it to some software as different hardware.

In terms of performance, there's the setting I mentioned above which can help to ensure hardware CPU cache is really used how it should be, but otherwise, it's just about how many threads are available... where they come from isn't the business of the VM, only the hypervisor (which you are controlling with those settings).

If you don't care to control it, just set it to 1 CPU, however many cores you want as threads and 1 thread (per core).

 

[Heracles]

First layer is CPU. Instead of CPU, other documents call that concept "socket". That is, how many processing chips can you have on a motherboard.

When you do virtual, that does not really matter. As @sretalla told you, just keep that to 1 to ensure you will not end up with poor performance related to code (operating system or application) that would not manage multiple sockets properly.

Second layer is core. That means how many processing units a single processing chip contains.

In the old days, that was only 1. But for long now, a single chip can hold many processing units (many cores). These cores are closer to what people think of a CPU.

Last layer are threads.

In the old days, a single CPU had a single Core and that Core could do only 1 thing. Then came hyper-threading, a way for a single physical CPU to split itself in two independent halves. That way, it can run two independent process. It helped increase performance because most of the time, the CPU is way too fast compared to the rest of the system (RAM, Disk, Network, PCI, ...). By doing 2 things at once, it achieves more in the same amount of time.

All of that is 100% related to physical concepts. As such, it does not matter that much when you virtualize your hardware.

So go for 1 single CPU.
Put in as many Core as you wish
Keep them at 1 thread per Core

That way, you have only a single value to adjust.

 

[sretalla]

I also didn't mention the setting (because it's not important in your case, but in the interest of completeness for folks with multiple physical CPUs):

Pin vcpus​

When number of vcpus is equal to number of cpus in CPU set vcpus can be automatically pinned into CPU set. Pinning is done by mapping each vcpu into single cpu number in following the order in CPU set. This will improve CPU cache locality and can reduce possible stutter in GPU passthrough VMs.

And you define the CPU-mode. That seems to be the easy one, where I assume that that should normally be ^Host Pass through^

That's not necessarily the case.

Host model can be appropriate a lot of the time as can Custom and then setting to KVM64 or matching a particular generation of Intel or AMD CPU if an app or guest OS needs to see that.

 

[Louis2]

Yep, I understand that it can be necessary to emulate a certain CPU, however I never run that is the minority of applications. However emulating a certain CPU will always cost performance, so I assume that ^Host Passthrough^ is normally the best setting.

Back to my initial question, the reason is of course that I need to assign something to a VM and I need like to understand what I am doing. And I still do not have that feeling.
- I really do not understand the impact from assigning more cores that there physically are. However my feeling is, that it is a bad idea
- I think that the hypervisor try's to emulate CPU-sockets (virtual CPU), Cores and threads towards the VM application. I also think that the closer that is to reality, the better the performance will be. Given that feeling the best setting will be:
1 Virtual CPU since my computer has only one physical CPU and I am not running applications explicit requiring more than 1
x Number of cores where "x" should in my case between "1" and "8"
2 Number of number of threads since my CPU has 2 threads per CPU

A situation where I would like half a core (one thread) so 1,1,1 is also an option in case many VM's which need to run in parallel and are consuming little CPU.

Thinking this way, I tend to use:
1:1:1 for low requirement frequently running / real-time running VM's
1:1:2 for low requirement VM's
1:2:2 for average VM's without GUI
1:4:2 for VM's with GUI
1:6:2 for heavy VM's (heavy DB or such)
1:8:2 never since that would push the first two classes of VM's away

Just thinking

 

[sretalla]

As already suggested above and based on your own posts, differentiating cores/threads makes no sense here.

Just use 1:x:1 where x is the number of "cores" (or whatever term you want to use to describe the units of computing power you will assign).

Higher x is higher multi-core compute power.

I see no reason nor benefit to your proposed structure.