CPU Temp Reporting LARGE difference: IPMI/BIOS vs. FreeNAS - Supermicro X11 SSL-F & Intel E3 1245 v6

Status
Not open for further replies.

appoli

Dabbler
Joined
Mar 22, 2017
Messages
44
Hi All,

Since there are a number of people that know the Supermicro X11 series motherboards I hope that you guys can help, but I wouldn't call this a FreeNAS problem per se since I personally believe that FreeNAS is the only thing on the machine reporting the correct CPU temp...

I build a machine to run FreeNAS so I chose parts that were recommended by the forum - since it was to run headless the IPMI stuff really intrigued me. My original processor, Intel Sky Lake i3 6100, showed essentially the same temp - the temp in IPMI every now and then went up a couple degrees for 1 or 2 reporting cycles, but I never paid much attention to it. Since I wanted to keep the budget low I cut corners with the hard disks (got those fancy enterprise grade disks that apparently die faster, but got refurbished ones for less than new regular NAS drives... it's a home server where the only stuff that is located only on the FreeNAS box are movies & it's running raidz1) I wanted to keep the temps in the right area to do what I could to keep my disks alive. I set the IPMI temp thresholds, but when I realized that the SM controller doesn't really care much about peripheral temps I added the fan spin script.
However, I did notice that the thresholds for the CPU temp never 'stuck'. After every reboot they would reset to the original threshold.... in addition the highest threshold doesn't seem to actually show up in the IPMI dashboard. But that didn't really both me much so I didn't pay attention.

Since I ended up using the machine for media playing/transcoding & I was told that the issue myself and others were experiencing with Emby was due to a limit in resources I upgraded the CPU to a Kaby Lake Xeon E3 1245 v6. The other day, after I had set the CPU threshold again (I don't usually set it after reboots, but I felt like it a few days before) the physical alarm tone on the motherboard went off while the machine was doing some light transcoding. It went off because the IPMI controller was reading the CPU temp as 60 degrees C, the second upper limit I had set.

I was shocked that a light workload had gotten the temp so high (I have the fancy Intel copper cooler that has a heat pipe in it) so I checked the temps. IPMI and the Fan Spin script, which I believe gets the temp from IPMI, were showing the CPU as being at 60 degrees C while FreeNAS was showing the CPU being at something like 33 degrees C. I turned the machine off, opened it up and put my hand on the cooler - it was pretty cool to the touch, cooler than my hard disks which were a couple ticks under 40 degrees C.

The IPMI firmware I am running is 1 revision older than the latest, and I will try installing the latest if that is what is suggested however the Supermicro literature says to not install new firmware unless there is an issue...and there is not IPMI changelog so I can't determine if the firmware even addresses this issue.
But I figured I should ask the community and see if anyone has experienced something like this/has an idea as to what I should do?

I've included some screen shots of the output of the Fan Spin script, the IPMI web interface and the reporting tab in FreeNAS (showing what I personally believe to be the correct temp). I know there are multiple CPU temp sensors, but I think they get the temp from approximately the same place... so I didn't think the issue was poor application of thermal paste since, but I suppose that could be an issue? If that's what you guys think I could get the CPU hot, pull the heatsink off and take it's temp using my IR gun. However, I figured the temp probes were close enough that there would be a 30 degree C different just because the temp probes are a few mm apart.

Any help is greatly appreciated - the high temp is causing the fan spin script to rev the heatsink fan way higher than it needs to be which makes a LOT more noise than necessary (at idle it's quiet, but revved up its loud - and every other fan I have is a noctua so I was hoping for a relatively quiet machine when it is just coasting) and there is a tiny bit more power use heh.
thanks!
(my apologies for the long winded post - I'm way too verbose and I try to be as detailed as possible...)
 

Attachments

  • Screen Shot 2017-11-21 at 8.11.50 PM.png
    Screen Shot 2017-11-21 at 8.11.50 PM.png
    1.4 MB · Views: 646
  • Screen Shot 2017-11-21 at 8.11.59 PM.png
    Screen Shot 2017-11-21 at 8.11.59 PM.png
    291.7 KB · Views: 623
  • Screen Shot 2017-11-15 at 9.44.03 PM.png
    Screen Shot 2017-11-15 at 9.44.03 PM.png
    765.2 KB · Views: 609
  • Screen Shot 2017-11-15 at 9.43.53 PM.png
    Screen Shot 2017-11-15 at 9.43.53 PM.png
    731.8 KB · Views: 645

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,871
Do you have the current IPMI firmware and BIOS firmware installed? The Kabylake CPU was not supported ou tof the box and required a firmware upgrade. I'm not saying this is the issue but it could be.

Do not take the heatsink off the CPU and power up the machine. If you fell the heatsink is making poor contact then ensure you have some thermal compound available before removing it, clean up the old compound and place a new thin layer across the CPU, then place the heatsink on the CPU.

In my opinion you should not have to adjust any values in the IPMI to make things work correctly. Reset to factory defaults and ensure your CPU temp tracks what FreeNAS reports, or within a few degrees. I myself do not use these fan scripts but rather I run all the fans at slow speed by adjusting the voltage to each fan to 7VDC (direct wired power) so they all rotate at a constant slow speed ensuring my system is always passing enough air. The CPU fan is controlled by the motherboard of course.

So as I'm thinking about this, what script are you using and what threshold valuses did you set for the CPU fan? I'm thinking you set the CPU threshold temps too low and the script will tell IPMI that the CPU is hot and needs to be cooled thus the high CPU fan speeds. But since I don't know the script then I can't really say for sure.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,071
I'd heard of this before, but I haven't looked into the discrepancy yet.
 
Status
Not open for further replies.
Top