This is mainly for information, I have found a work-around. I also don't know if this makes any difference in practice, it may be just a theoretical problem.
There is an unexpected mismatch with hyperthreads between the host and guest when pinning the CPUs. I use an Ubuntu 22.10 guest, and it doesn't get the right CPUs as hyperthreads.
As an example, I wanted to pin all of the cores and hyperthreads of the second socket CPU in a two-socket system to a VM.
On the host, the first threads of CPU socket 1 (second socket) are from CPU # 12-23. Then the second threads of the same cores are from 36-47.
So I put Pinned VCPU as: 12-23,36-47
The Ubuntu guest saw the first two CPUs as two threads of core 0, the next two CPUs as core 1, etc.
I confirmed that this is wrong by forcing a program to run on processors 0 and 1 in the guest, and viewing the bar chart in the TrueNAS. On the host it acually runs on CPUs 12 and 13, which are independent cores.
I get worse performance in sysbench when using cores 0 and 12 in the guest, which according to Ubuntu would be independent cores, but are actually two threads on the same core. In the bar chart, on the host, it's CPUs 12 and 36 that are used; the same core.
The solution is to put the pinned CPUs in this order - interleaving the first and second threads of each core
12,36,13,37,14,38,15,39,16,40,17,41,18,42,19,43,20,44,21,45,22,46,23,47
I hope this can be of help to someone, even though it probably doesn't make a huge difference.
There is an unexpected mismatch with hyperthreads between the host and guest when pinning the CPUs. I use an Ubuntu 22.10 guest, and it doesn't get the right CPUs as hyperthreads.
As an example, I wanted to pin all of the cores and hyperthreads of the second socket CPU in a two-socket system to a VM.
On the host, the first threads of CPU socket 1 (second socket) are from CPU # 12-23. Then the second threads of the same cores are from 36-47.
So I put Pinned VCPU as: 12-23,36-47
The Ubuntu guest saw the first two CPUs as two threads of core 0, the next two CPUs as core 1, etc.
Code:
$ grep 'processor\|core id' /proc/cpuinfo processor : 0 core id : 0 processor : 1 core id : 0 processor : 2 core id : 1 processor : 3 core id : 1 processor : 4 core id : 2 processor : 5 core id : 2 processor : 6 core id : 3 processor : 7 core id : 3 processor : 8 core id : 4 processor : 9 core id : 4 processor : 10 core id : 5 processor : 11 core id : 5 processor : 12 core id : 6 processor : 13 core id : 6 processor : 14 core id : 7 processor : 15 core id : 7 processor : 16 core id : 8 processor : 17 core id : 8 processor : 18 core id : 9 processor : 19 core id : 9 processor : 20 core id : 10 processor : 21 core id : 10 processor : 22 core id : 11 processor : 23 core id : 11
I confirmed that this is wrong by forcing a program to run on processors 0 and 1 in the guest, and viewing the bar chart in the TrueNAS. On the host it acually runs on CPUs 12 and 13, which are independent cores.
Code:
taskset -c 0,1 sysbench cpu --threads=2 run
I get worse performance in sysbench when using cores 0 and 12 in the guest, which according to Ubuntu would be independent cores, but are actually two threads on the same core. In the bar chart, on the host, it's CPUs 12 and 36 that are used; the same core.
The solution is to put the pinned CPUs in this order - interleaving the first and second threads of each core
12,36,13,37,14,38,15,39,16,40,17,41,18,42,19,43,20,44,21,45,22,46,23,47
I hope this can be of help to someone, even though it probably doesn't make a huge difference.
