CPU Choice for best performance

[berkbal]

Hello, im running over 700+ VM's from proxmox with TrueNas storage. İn current setup i have 256GB ECC DDR3 Ram With 2x 2670 v2 as TrueNas server and it performs too slow. I want to upgrade server for next storage with better performance. Which cpu should i use ? im thinking about using xeon v4 but is core number or base clock will be better? ( I don't have any problem on storage side i have 24x SAS SSD * 2 Z2 raid with 2 Storages)

 

[ChrisRJ]

Is it actually clear that the CPUs are the bottleneck?

 

[berkbal]

Is it actually clear that the CPUs are the bottleneck?

Which CPU i should use? Do you know any good CPU for this im thinking for E5 v4 Family

 

[berkbal]

Which CPU i should use? Do you know any good CPU for this im thinking for E5 v4 Family

Sorry i got it wrong, yes i think its CPU because im using 10G fiber and switchs storages etc looking good. When i check TrueNas server htop i see that nfsd service consuming %200 CPU so i think its because of cpu

 

[HoneyBadger]

When i check TrueNas server htop i see that nfsd service consuming %200 CPU so i think its because of cpu

How many NFS server threads do you have your system configured for? Look under the NFS configuration at the top for "Number of servers" - I believe it's defaulted to 4 in versions before 13, and 16 from that point on.

A few other things that bear mentioning, based on your described use case:

1. Even with SSDs, mirrors are still preferred for maximum storage throughput.
2. Are you using any other CPU-intensive settings such as compression stronger than LZ4/zstd, or deduplication?
3. NFS writes will use synchronous writes, which could benefit from an SLOG device. All-SSD pools will be less impacted by this.

 

[ChrisRJ]

Sorry i got it wrong, yes i think its CPU because im using 10G fiber and switchs storages etc looking good. When i check TrueNas server htop i see that nfsd service consuming %200 CPU so i think its because of cpu

This usually means that 2 CPU cores are fully utilized by the process. It does not mean that 2 CPU are completely maxed out.

 

[berkbal]

How many NFS server threads do you have your system configured for? Look under the NFS configuration at the top for "Number of servers" - I believe it's defaulted to 4 in versions before 13, and 16 from that point on.

A few other things that bear mentioning, based on your described use case:

1. Even with SSDs, mirrors are still preferred for maximum storage throughput.
2. Are you using any other CPU-intensive settings such as compression stronger than LZ4/zstd, or deduplication?
3. NFS writes will use synchronous writes, which could benefit from an SLOG device. All-SSD pools will be less impacted by this.

I have edited it to 40 because my server has 40 CPU total. also im using SAS SSD on storages . i don't use settings like LZ4/zstd, or deduplication

 

[HoneyBadger]

I have edited it to 40 because my server has 40 CPU total.

Did you start with this number initially, or increase to there based on earlier performance testing?

That 40 threads is also counting your hyperthreaded/logical cores; I can't say for certain that HT might be slowing you down, but in a heavily CPU loaded scenario you might benefit from limiting it to 20 and having the scheduler hit only the purely physical cores.

But as @ChrisRJ says, the "200%" means "2 cores fully loaded" by that service. Are there 40 nfsd processes running, if you do something like ps auxH | grep nfs?

i don't use settings like LZ4/zstd, or deduplication

I'd suggest using LZ4 compression, as it's basically "free" for most VM workloads - but stay away from deduplication here.

 

[berkbal]

Did you start with this number initially, or increase to there based on earlier performance testing?

That 40 threads is also counting your hyperthreaded/logical cores; I can't say for certain that HT might be slowing you down, but in a heavily CPU loaded scenario you might benefit from limiting it to 20 and having the scheduler hit only the purely physical cores.

But as @ChrisRJ says, the "200%" means "2 cores fully loaded" by that service. Are there 40 nfsd processes running, if you do something like ps auxH | grep nfs?


I'd suggest using LZ4 compression, as it's basically "free" for most VM workloads - but stay away from deduplication here.

Yes, i was using 10 cores at first then because of performance problem i set it to 40. With command you send i can see 40 nfsd processes