Xeon D-1541 too much power consumption?

[macx979]

Hey guys,

i recently upgraded my TrueNAS server since my mainboard as well as my power supply died due to a power outage.

I replaced the supermicro X9SCM-F running a Xeon E3-1240v2 with Asrock Rack D1541D4U-2T8R running Xeon d-1541. Besides that I had to replace the "be quiet! pure power 430 Watt" power supply with a "be quiet! BN253 Dark Power Pro 11 (850 Watt)" which I reused from a spare pc. Furthermore I removed the Mellanox ConnectX-3 10Gbit pcie card, since the new mainboard has build in 10Gbe.

The replacement went fine and my new server is up and running but I am really dissapointed regarding the power consumption of this new system.
The old one was using roughly 112 Watts at idle while the new system is at around 130 Watts. Of course, the xeon d-1541 is about 1/3 more powerful than the e3 cpu but I was under the impression that xeon d cpu and mainboard bundles are insanely power efficent.

I already set the c-state in TrueNAS as I did on the old platform (sysctl hw.acpi.cpu.cx_lowest=C3) but this didn't change a watt.

Is 130 watt during idle just normal or did I miss some important setting?

Best

 

[Samuel Tai]

For a server, 130W idle is already very good. Try turning off one of the onboard 10G NICs.

 

[demon]

Also, you went from a 430W power supply to an 850W. Why? Did you do the math for your power budget? If the total average power draw is low relative to the 850W rating, you're probably getting below its efficiency curve, thus the power supply has to draw more power to convert it down for what the system needs. My NAS (see signature) idles around the 100W mark, with an 8-core Xeon E CPU and a 450W supply. Even if everything were at full tilt, the draw would be about half of the rated output. I can't be 100% certain, but I'd suspect that might at least be contributing.

 

[macx979]

Also, you went from a 430W power supply to an 850W. Why?

As I said, the power supply died as well due to the power outage and the 850W power supply was the only one I had lying around.
But indeed, I also suspected the bad efficeny at ~15% load to be a reason. But then I read that this 80 Plus Platium is rated at 92% efficiency at 20% load.

Your system looks quite similar to mine in terms of number of spinning disks and something like 100W during idle was what I was expecting to be realistic for my new system.

I still have a couple of things to check out:
- As Samuel proposed, I'll see if I can disable one 10G NIC in BIOS
- I haven't seen any option to disable the onboard SAS controller. Maybe it's consuming power without beeing utilised.

besides that, I don't have any more ideas

EDIT:
- it is not possible to deactive only one NIC. Either both or none.

But I figured out that somethings odd regarding CPU speedstep.

The CPU could step down to 800MHz:

Code:

root@freenas[~]# sysctl -a | grep dev.cpu.0.freq_levels
dev.cpu.0.freq_levels: 2101/45000 2100/45000 2000/42264 1900/39583 1800/36956 1700/34381 1600/32283 1500/29808 1400/27383 1300/25010 1200/23093 1100/20816 1000/18588 900/16409 800/14666

But even with 0% load the cpu cores are stuck at 2100MHz:

Code:

load  54%, current freq 2101 MHz ( 0), wanted freq 3025 MHz
load   0%, current freq 2101 MHz ( 0), wanted freq 2930 MHz
load   0%, current freq 2101 MHz ( 0), wanted freq 2838 MHz
load   3%, current freq 2101 MHz ( 0), wanted freq 2749 MHz
load  16%, current freq 2101 MHz ( 0), wanted freq 2663 MHz
load   0%, current freq 2101 MHz ( 0), wanted freq 2579 MHz
load   0%, current freq 2101 MHz ( 0), wanted freq 2498 MHz
load  17%, current freq 2101 MHz ( 0), wanted freq 2419 MHz
load  62%, current freq 2101 MHz ( 0), wanted freq 3999 MHz
load   4%, current freq 2101 MHz ( 0), wanted freq 3874 MHz
load   0%, current freq 2101 MHz ( 0), wanted freq 3752 MHz
load   3%, current freq 2101 MHz ( 0), wanted freq 3634 MHz
load   6%, current freq 2101 MHz ( 0), wanted freq 3520 MHz
load   0%, current freq 2101 MHz ( 0), wanted freq 3410 MHz
load   0%, current freq 2101 MHz ( 0), wanted freq 3303 MHz
load   7%, current freq 2101 MHz ( 0), wanted freq 3199 MHz
load   3%, current freq 2101 MHz ( 0), wanted freq 3099 MHz

 

[Hinterwaeldler]

Have you tried to set powerd to throttle your cpu more aggressively by using the adp mode?

I have set two rc.conf tunables for this:
powerd_enable=YES
powerd_flags="-n adp"

Now my cpu idles at 800mhz and scales up only if I have load

 

[macx979]

Have you tried to set powerd to throttle your cpu more aggressively by using the adp mode?

I have set two rc.conf tunables for this:
powerd_enable=YES
powerd_flags="-n adp"

great, I didn't know about this.
Unfortunately, in terms of power consumption this doesn't make a change in my situation. But I still need to double check.

Furthermore I replaced the 850w power supply with a 450w one and this reduced the power consumption in total by ~6 Watts - most likely due to a better efficiency.
Replacing the HBA PCIe card by the onboard SAS Controller gave me an impressive reduction of another 13 watts.

So I am around 109 Watts for the entire system. This looks quite promising already.

 

[tio]

Hey guys,

i recently upgraded my TrueNAS server since my mainboard as well as my power supply died due to a power outage.

I replaced the supermicro X9SCM-F running a Xeon E3-1240v2 with Asrock Rack D1541D4U-2T8R running Xeon d-1541. Besides that I had to replace the "be quiet! pure power 430 Watt" power supply with a "be quiet! BN253 Dark Power Pro 11 (850 Watt)" which I reused from a spare pc. Furthermore I removed the Mellanox ConnectX-3 10Gbit pcie card, since the new mainboard has build in 10Gbe.

The replacement went fine and my new server is up and running but I am really dissapointed regarding the power consumption of this new system.
The old one was using roughly 112 Watts at idle while the new system is at around 130 Watts. Of course, the xeon d-1541 is about 1/3 more powerful than the e3 cpu but I was under the impression that xeon d cpu and mainboard bundles are insanely power efficent.

I already set the c-state in TrueNAS as I did on the old platform (sysctl hw.acpi.cpu.cx_lowest=C3) but this didn't change a watt.

Is 130 watt during idle just normal or did I miss some important setting?

Best

How did you manage to set the C state on the unit? Got the same board here and looking to throttle down usage if possible