SMB and CPU

[Fastline]

Hello,

So, i've come to know that SMB is single threaded process and therefore one needs to use the highest possible Core Clock for the best performance, considering every other hardware is up to the mark. I've got a few questions regarding that

1. Is that really the case? Being a single threaded connection, does it mean that one needs to use the highest Core clock? For example, 4.2GHz wins over 3GHz in SMB.

2. If that's the case, if you use dual CPU, does TrueNAS double the Core Clock for the SMB transfers? For example, 2x3.0GHz = 6.0GHz.

Thanks

 

[Arwen]

Here is my understanding, (someone else correct me if I am wrong);

1. Yes. Use the highest core clock speed to get the best per user performance.
2. No, a single user does not benefit from dual core, (or dual threads per core). Additional users likely benefit from additional cores.

There is a new SMB feature involving dual network connections, for a single user. Though the network has to be planned carefully, and is generally not relevant to SOHO users. In reality, for single users that feel 1Gbits/ps Ethernet is "too slow", jumping to 10Gbits/ps Ethernet is the way to go. Not dual path. Plus, their is of course the Samba single threaded limit to take into consideration as well.

 

[danb35]

Being a single threaded connection, does it mean that one needs to use the highest Core clock? For example, 4.2GHz wins over 3GHz in SMB.

Yes, but there's only so much you can do there--e.g., if your array can only sustain 1.5 Gb/sec transfers, there's no point holding out for a CPU that can handle 10 Gb/sec/thread. That, of course, leads to the question of what kind of clock speed you'd need to handle a given transfer speed, and that's a question I can't answer.

if you use dual CPU, does TrueNAS double the Core Clock for the SMB transfers? For example, 2x3.0GHz = 6.0GHz.

No, if it could do that, it could spread the work across multiple cores (or threads) on the same physical CPU.

SMB is single threaded process

To be specific, it's single-threaded per client.

 

[Fastline]

Yes. Use the highest core clock speed to get the best per user performance.

Cool!

No, a single user does not benefit from dual core, (or dual threads per core). Additional users likely benefit from additional cores.

Yes, i get that. But here, I'm talking about the core clock only, not the CPU cores.

There is a new SMB feature involving dual network connections, for a single user. Though the network has to be planned carefully, and is generally not relevant to SOHO users. In reality, for single users that feel 1Gbits/ps Ethernet is "too slow", jumping to 10Gbits/ps Ethernet is the way to go. Not dual path. Plus, their is of course the Samba single threaded limit to take into consideration as well.

Of course, what you said is true but my question is in a Dual CPU setup, where multiple users will access the NAS, does TrueNAS doubles the Core Clock or it uses the single CPU Core clock speeds for the transfers?

 

[Fastline]

Yes, but there's only so much you can do there--e.g., if your array can only sustain 1.5 Gb/sec transfers, there's no point holding out for a CPU that can handle 10 Gb/sec/thread.

Indeed. That definitely makes sense.

That, of course, leads to the question of what kind of clock speed you'd need to handle a given transfer speed, and that's a question I can't answer.

Hmm. I think for server grade CPUs, the max one could go is 4.0GHz. Already, these have lower core clocks than normal consumer grade CPUs.

However, now my question is how does one select the CPU? Is it based on Storage capacity, or network setup? I mean for example, if a user uses like 16-20 HDDs and has a 10GbE network setup, a Pentium CPU is definitely going to be a bottleneck. So, how does one calculate the required number of CPU Cores, when one knows that ECC is a must for which one has to choose Xeon or EPYC and probably newer CPU generation, not the latest of course (if not possible).

No, if it could do that, it could spread the work across multiple cores (or threads) on the same physical CPU.

Makes sense. So, this means no matter if you have a Quad CPU, only a single thread will be utilized and even if one has a Dual Socket setup, TrueNAS will only utilize a single CPU and its Core Count due to the SMB technology which limits the use of dual CPU or Dual CPU Core Count and Core Clock. Do i understand it right?

So for that single socket CPU will be best. But i have seen some configurations with Dual Socket CPUs. What's the purpose then? VM and other work within TrueNAS, not just the Storage and transfers using SMB?

To be specific, it's single-threaded per client.

Umm, does that mean if another user connects to the NAS, another thread will be used for the transfers? If so, what's the workflow in Dual CPU setup? Would that utilize a second CPU?

Also, when doing multiple transfers to the NAS from multiple clients, does TrueNAS use both the CPU's Core Clock to serve each client in a Dual CPU setup?

 

[Patrick M. Hausen]

It's irrelevant if you have a single socket system with e.g. 16 cores or a dual socket system with 8 cores in each CPU. SMB will use one core per client up to a maximum of 16 concurrent transfers. More clients will not be queued but simply have to share cores.

This simplifies things a bit, because there's hyperthreading and NUMA considerations in a multi socket system (or even some AMD multi chip single socket ones) etc.

As to why one would use such a system: file servers frequently serve hundreds of clients of which dozens might have an active operation at the same time.

 

[Fastline]

It's irrelevant if you have a single socket system with e.g. 16 cores or a dual socket system with 8 cores in each CPU. SMB will use one core per client up to a maximum of 16 concurrent transfers. More clients will not be queued but simply have to share cores.

This simplifies things a bit, because there's hyperthreading and NUMA considerations in a multi socket system (or even some AMD multi chip single socket ones) etc.

As to why one would use such a system: file servers frequently serve hundreds of clients of which dozens might have an active operation at the same time.

Yet another valuable information. Thanks man.

So, like i said, a single CPU is the go when one does not VM and such needs right? And probably, should pick a CPU with the highest Core Clock. Is that correct?

 

[Patrick M. Hausen]

Depends on your priorities ... what about energy efficiency? Why do you think you need the highest SMB performance for a single thread possible? Is a low power Atom or Xeon-D CPU even a bottleneck for common 1 Gbit/s speed ...?

 

[Fastline]

Depends on your priorities

Like?

what about energy efficiency?

Under 200W works for me for the CPU.

Why do you think you need the highest SMB performance for a single thread possible?

Umm, cause I'm going to pick the right hardware i guess so want maxed out performance as there will be multiple clients accessing the NAS.

Is a low power Atom or Xeon-D CPU even a bottleneck for common 1 Gbit/s speed ...?

Not at all. However, here I'm trying to have 10GbE minimum.

 

[danb35]

But i have seen some configurations with Dual Socket CPUs. What's the purpose then?

Patrick's already mentioned "lots of clients" as one. "More RAM" and "More PCIe lanes" would be two more, although RAM limits are high enough any more that that's not much of a factor (my motherboard, IIRC, can handle up to 2 TB of RAM). Availability on the secondary market would be a fourth. And, of course, you're already aware that VMs and apps can add significantly to the CPU load.

Umm, does that mean if another user connects to the NAS, another thread will be used for the transfers?

Yes.

If so, what's the workflow in Dual CPU setup? Would that utilize a second CPU?

It'd use another core, or at least another thread (keeping hyperthreading in mind). A second (or third, or fourth) physical CPU isn't, for these purposes, significantly different from another core on the same physical CPU.

 

[Fastline]

Patrick's already mentioned "lots of clients" as one. "More RAM" and "More PCIe lanes" would be two more, although RAM limits are high enough any more that that's not much of a factor (my motherboard, IIRC, can handle up to 2 TB of RAM). Availability on the secondary market would be a fourth. And, of course, you're already aware that VMs and apps can add significantly to the CPU load.

Gotcha

Yes.

Cool!

It'd use another core, or at least another thread (keeping hyperthreading in mind). A second (or third, or fourth) physical CPU isn't, for these purposes, significantly different from another core on the same physical CPU.

Thank you for explaining it well. So, in short, it can be said that if a user has only storage needs, no VM, no databases, no Apps, a single CPU with Hyperthreading would do the job and there's no need of Dual CPU unless you have lots of clients who will access the NAS.

Regarding the CPU selection, i think besides the ECC and lanes, one needs to have 4 cores+ for a single client and probably more for more clients. Is there any criteria for selecting a CPU depending on the needs? I mean how many cores one should choose? Also, does NIC speed make any impact on the CPU selection?

 

[Davvo]

If a home user only has storage needs low-core count, low-power CPUs are their target. Embedded are your first option. Pentiums your second.

Sadly there doesn't yet exist a 5GHz 4C CPU. At least, I wasn't able to find it.

 

[Fastline]

If a home user only has storage needs low-core count, low-power CPUs are their target. Embedded are your first option.

Cool. Would a 6 Core CPU overkill for two clients doing 10G?

Sadly there doesn't yet exist a 5GHz 4C CPU. At least, I wasn't able to find it.

Me neither. The max i could find is 4.2GHz for the one which supports ECC.

I guess 2 cores for the TrueNAS itself and then the rest for the clients. Is that calculation fine? or i would need a more powerful CPU?

 

[Davvo]

For SMB you have to look at clock speeds.
Then look at HyperThreading.
Then at CORE number.
Finally you look into power efficency.

 

[Fastline]

For SMB you have to look at clock speeds.
Then look at HyperThreading.
Then at CORE number.
Finally you look into power efficency.

I'm just stuck at point number 3. Also, wanted to add that need to use 2xLSI 9400-16i so also need to make sure that there are sufficient lanes. It will also have a NIC.

 

[Davvo]

I'm just stuck at point number 3.

A 4C will suffice if you find one with clock speeds you like.
A 6C is indeed overkill for two simultaneous users in SMB.

 

[danb35]

a single CPU with Hyperthreading would do the job

Probably. If:

• Your only application is file-sharing
• You're only using SMB
• You have relatively few users who'd be accessing at one time
• Highest possible network throughput is your only priority

then go for a 4+ core CPU with the highest clock speed you can find. Hyperthreading will likely be a given in this case.

As a semi-relevant point of comparison, pushing 10 Gbit/sec (9.4 Gbit/sec actual throughput) through iperf3 comes close to maxing out one core of my Xeon Gold 6132. SMB is going to have more i/o to deal with, of course.

 

[ChrisRJ]

I would like to point out that this discussion, at least in my perception, is a bit narrow. Perhaps I missed the part where a precise use-case description was done. That would include details on the workload in terms of throughput, IOPS, etc.

Not that the points above are irrelevant. But they seem to focus on one aspect only. At least as important are the underlying disks and their configuration, the network, the clients (hardware and applications), etc. It is also worth noting that those clients actually need to be able to saturate the server (not only in the context of TrueNAS/storage but in general).

 

[danb35]

this discussion, at least in my perception, is a bit narrow.

Agreed. But that seems to be characteristic of this OP.

 

[Fastline]

A 4C will suffice if you find one with clock speeds you like.
A 6C is indeed overkill for two simultaneous users in SMB.

Thank you friend.

Would there be a lot difference in 3.7GHz and 4.2GHz?