SOLVED SuperMicro SC836 HDD Temps Too High / Fan Replacements / Your Experiences?

Status
Not open for further replies.

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
You know, there are only two modes which spin the peripheral fans at 100%, Max and Heavy IO.

All the other modes spin the fans in relation the CPU temperature, which is not what you want.

If a supermicro system can only be cool enough on max, you should check what fans are connected to where, and run a custom fan controller script.

Here's mine: https://forums.freenas.org/index.php?threads/script-hybrid-cpu-hd-fan-zone-controller.46159/

Should work on x9 but will probably need a BIOS update.

I was of the understanding that fan scripts, example: Fan Scripts for Supermicro Boards Using PID Logic wouldn't work with an X9 board because they attempt to use ipmitool to set fan speed at a granular level (0 - 100%, with 64 increments), but a very brief glance at your script suggests it might work with an X9 board as you take a different approach, setting fan speed by changing the applied duty cycle. Is that a correct assertion? Sorry, I don't have time to review in more detail as I'm getting yelled out to go eat dinner. The server is almost loud enough such that I don't hear that dinner bell though. :)

$hd_fan_duty_high = 100 = "Full Speed" Profile
$hd_fan_duty_med_high = 80 = ??? "Heavy I/O" = 75%
$hd_fan_duty_med_low = 50 = "Standard Speed" Profile
$hd_fan_duty_low = 30 = "Optimal Speed Profile"

Two complications I see:
  • $ipmitool = "/usr/local/bin/ipmitool" = With an ESXi install, impitool in FreeNAS shell can only be used by remote log in, i.e. ipmitool -I lanplus -H $IP -U $user -P $pass cmd and that directory don't exist.
  • sysctl -a dev.cpu = Also an ESXi complication, cpu temperature is not passed through to FreeNAS. The only way to snag it that I'm aware of is by impitool, i.e. ipmitool -I lanplus -H $IP -U $user -P $pass sensor | grep "CPU1 Temp" | awk '{print $1, $4}'
Is there a work around for these items? Other than to revert to bare metal?
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,419
I was of the understanding that fan scripts, example: Fan Scripts for Supermicro Boards Using PID Logic wouldn't work with an X9 board because they attempt to use ipmitool to set fan speed at a granular level (0 - 100%, with 64 increments), but a very brief glance at your script suggests it might work with an X9 board as you take a different approach, setting fan speed by changing the applied duty cycle. Is that a correct assertion? Sorry, I don't have time to review in more detail as I'm getting yelled out to go eat dinner. The server is almost loud enough such that I don't hear that dinner bell though. :)

$hd_fan_duty_high = 100 = "Full Speed" Profile
$hd_fan_duty_med_high = 80 = ??? "Heavy I/O" = 75%
$hd_fan_duty_med_low = 50 = "Standard Speed" Profile
$hd_fan_duty_low = 30 = "Optimal Speed Profile"

Two complications I see:
  • $ipmitool = "/usr/local/bin/ipmitool" = With an ESXi install, impitool in FreeNAS shell can only be used by remote log in, i.e. ipmitool -I lanplus -H $IP -U $user -P $pass cmd and that directory don't exist.
  • sysctl -a dev.cpu = Also an ESXi complication, cpu temperature is not passed through to FreeNAS. The only way to snag it that I'm aware of is by impitool, i.e. ipmitool -I lanplus -H $IP -U $user -P $pass sensor | grep "CPU1 Temp" | awk '{print $1, $4}'
Is there a work around for these items? Other than to revert to bare metal?

1) I think Xeon-e5 x9 boards do work. You may need a BIOS update. I could be wrong.
2) ipmitool will work from inside FreeNAS if you specify host etc
3) the script has an alternate way to read CPU via IPMI.

Alternate is a script which fluctuates max/standard/Optimal
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
You know, there are only two modes which spin the peripheral fans at 100%, Max and Heavy IO.

All the other modes spin the fans in relation the CPU temperature, which is not what you want.

If a supermicro system can only be cool enough on max, you should check what fans are connected to where, and run a custom fan controller script.

Here's mine: https://forums.freenas.org/index.php?threads/script-hybrid-cpu-hd-fan-zone-controller.46159/

Should work on x9 but will probably need a BIOS update.
Love those scripts, @Stux! But the X9 series motherboards don't support the raw IPMI commands for changing the fan mode duty cycle (raw 0x30 0x70 0x66 0x01 x000 0x??).

So I just keep things simple: I've pulled the 4th wire from the two CPU fans - an easily reversible operation if you know what you're doing - and plugged them into the FANA and FANB headers; they therefore run at full speed, ~3600 RPM, and keep the CPUs nice and cool, typically 34-37C and seldom rising above the mid-40s under load. I've experimented with the various fan mode settings, and have settled on the Standard or Heavy IO modes, both of which run the CPU-zone fans (FAN1-6) at ~50%. This spins the case fans at ~2300-2600RPM and keeps my HDDs at no more than ~37C without sounding like a flock of screaming banshees.

And this is good 'nough! :cool:
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
4. Improve: In process. 5. Control: To be completed.

I received the "correct" fans for my SC836E16-R1200B and I would like to share updated HDD temperature measurements with the community. I hope that my efforts here will benefit someone else who encounters a similar issue or that other community members learn something from from my analysis.

Before I begin, I would like to offer a sincere and genuine thank you to @Spearfoot , @Stux , @BigDave , and @melloa for their input and support. :):):) I've proposed "Next Steps" and a single "Question" at the end of this post - I would be appreciative if anyone is able to call out any flaw in those proposed next steps and/or answer that question.

As a refresher, my proposed explanation for HDD temperatures being high was my supposition that this was due my SC836E16-R1200B (purchased used via ebay) not being populated with the fans specified for this chassis on Supermicro's website. In my measurements, reference to "NEW" refers to those "correct" (now installed, as specified) fans, and reference to "OLD" refers to the "incorrect" (previously installed, as purchased) fans, specifically as follows:
  • "NEW" (Now Installed, Specified per Supermicro's website): Chassis Middle = 3 x FAN-0126L4 + Exhaust = 2 x FAN-0125L4
  • "OLD" (Previously Installed, as Purchased) = Chassis Middle = 3 x FAN-0094L4 + Exhaust = 2 x FAN-0062L
Note: A summary of the fan's specifications is included at the end of this post.

Results - Summarized, Qualitative (mostly)
  • The results are favorable and attributable to the extreme importance of using fans with sufficient static pressure. With the "OLD" / "incorrect" fans (vs. the "NEW" / "correct" fans), I was missing static pressure = 0.24 in. H20 at the backplane (per fan x 3) and static pressure = 0.19 in. H20 at the rear (per fan x 2).
  • Temperatures have decreased -2.9°C or better for each ambient temperature and fan profile combination.
  • My target was/is sub-40°C HDD temperatures at an ambient temperature of 78°F using the "Optimal" Fan profile was and I'm very close now (with a mean of 40.3°C).
  • As measured by an un-calibrated sound level app on my phone, there is no difference in noise level with the "NEW" fans.
OK - to the data …

Results - Summarized, Quantitative
  • Ambient @ 78°F: Mean reduction of -3.1°C with "NEW" fans. ("Full" = -3.1°C, "Standard" = -3.0°C, "Optimal" = -3.2°C)
  • Ambient @ 75°F: Mean reduction of -3.0°C with "NEW" fans. ("Full" = -3.0°C, "Standard" = -2.9°C, "Optimal" = -3.1°C)
  • Ambient @ 72°F: Mean reduction of -3.8°C with "NEW" fans. ("Full" = -3.0°C, "Standard" = -3,9°C, "Optimal" = -4.4°C)
    Summary.jpg
Results - Ambient @ 78°F
  • "Full" Fan Speed Profile: Reduction of -3.1°C with "NEW" fans.
  • "Standard" Fan Speed Profile: Reduction of -3.0°C with "NEW" fans.
  • "Optimal" Fan Speed Profile: Reduction of -3.2°C with "NEW" fans.
    Compare - Full.jpg
Results - Ambient @ 75°F
  • "Full" Fan Speed Profile: Reduction of -3.0°C with "NEW" fans.
  • "Standard" Fan Speed Profile: Reduction of -2.9°C with "NEW" fans.
  • "Optimal" Fan Speed Profile: Reduction of -3.1°C with "NEW" fans.
    Compare - Standard.jpg
Results - Ambient @ 72°F
  • "Full" Fan Speed Profile: Reduction of -3.0°C with "NEW" fans.
  • "Standard" Fan Speed Profile: Reduction of -3.9°C with "NEW" fans.
  • "Optimal" Fan Speed Profile: Reduction of -4.4°C with "NEW" fans.
    Compare - Optimal.jpg
Next Steps
  • Decouple CPU and HDD cooling and manage via fan script (my understanding is that with a X9 board, I will only have the luxury of the script adjusting fan speed between low / medium / high).
  • Install the Supermicro 2U Active CPU Heat Sinks (SNK-P0048AP4) that I have purchased as recommended by @Spearfoot (again, thank you).
  • Connect 3 Backplane fans to FANA using PWM splitter.
  • Connect CPU fans to FAN1-2.
  • Connect Exhaust fans to FAN3-4.
Question
  • I'm observing significantly higher temperatures for "Slots" 12 and 13, which are the 3.5" HDD Drive Bays at bottom, right. Reference one of the above diagrams for my labeling schema.
  • +5.3°C higher than lowest HDD temperature @ 78°F Ambient using the "Full" Speed Fan Mode (40.0°C v. 34.7°C)
  • +6.0°C higher than lowest HDD temperature @ 78°F Ambient using the "Standard" and "Optimal" Speed Fan Modes (44.0°C v. 38.0°C)
  • Is this to be expected and simply because those bays are directly in front of the PDU and there is no backplane fan directly behind them? If that is the explanation, it doesn't quite explain why there is still a temperature variance between the top and bottom bays in that column.
Column 4 Temps.jpg

Thank you for all of your input and assistance. :):):)


____

Fan Specifications per Supermicro's website:

[Chassis Middle Fans] / {Now Installed, Specified per Supermicro's website} or (Previously Installed, as Purchased)
  • {"NEW"} = 3 x FAN-0126L4: 80 x 80 x 38 (mm), Fan Speed (RPM) = 7.0K, Airflow (CFM) = 72.5, Static Pressure (in. H20) = 1.09, Noise (dBA) = 53.5
  • ("OLD") = 3 x FAN-0094L4: 80 x 80 x 38 (mm), Fan Speed (RPM) = 6.3K, Airflow (CFM) = 90.3, Static Pressure (in. H20) = 0.85, Noise (dBA) = 51
[Exhaust Fans] / {Specified, per Supermicro's website} or (Installed, as Purchased)
  • {"NEW"} = 2 x FAN-0125L4: 80 x 80 x 32 (mm), Fan Speed (RPM) = 6.7K, Airflow (CFM) = 59.6, Static Pressure (in. H20) = 0.68, Noise (dBA) = 47
  • ("OLD") = 2 x FAN-0062L4: 80 x 80 x 32 (mm), Fan Speed (RPM) = 5.0K, Airflow (CFM) = 59.1, Static Pressure (in. H20) = 0.49, Noise (dBA) = 47
____

[Cross Posted @ STH - I've discussed the issue at length with members from both forums and would like to give all the opportunity to view replies]
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,419
I'm not sure I understand how the optimal setting can possibly get better results than full :-/

I've found top bay drives are hotter than bottom bay drives. I figure because heat rises.

It would not surprise me if the HD column with no fan behind it is hotter than the rest. It might be worthwhile putting your cooler running drives there.
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
I'm not sure I understand how the optimal setting can possibly get better results than full :-/

What are you looking at precisely? I'm assuming you mean on a relative basis as compared to my "baseline"? On an absolute basis, at 78°F ambient, HDD temp for Full = 36.4°C and for Optimal = 40.3°C. But on a relative basis, the result that sticks out most to me is at 72°F ambient with Optimal showing a -4.4°C improvement over baseline (v. -3.0°C for Full).

Do you think the fan curves come into play here? More of an academic question than an actual one.
I've found top bay drives are hotter than bottom bay drives. I figure because heat rises.

So we would both expect to see higher temps for the top drive bays, right? But I see exactly the opposite, with the coolest temps at the top. If I had to guess, this could be because the fans don't extend all the way to the floor of the chassis? I've sealed off that top row of DVD-ROM/FDD height bays, so that variable can be removed from determining the cause.

Compare - Full.jpg
It would not surprise me if the HD column with no fan behind it is hotter than the rest. It might be worthwhile putting your cooler running drives there.

That is exactly what I see. Good call on moving cooler drives there.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,419
What are you looking at precisely? I'm assuming you mean on a relative basis as compared to my "baseline"? On an absolute basis, at 78°F ambient, HDD temp for Full = 36.4°C and for Optimal = 40.3°C. But on a relative basis, the result that sticks out most to me is at 72°F ambient with Optimal showing a -4.4°C improvement over baseline (v. -3.0°C for Full).

That'd be it.
 

TXAG26

Patron
Joined
Sep 20, 2013
Messages
310
Try this: I have a SC836 as well and did the following with mine since I was not using all 16 slots for HDD's. Mine is a single-socket board, and I added a Noctua NH-D9L (3U compatible). You may have to measure to see if there is enough room for dual processor boards.

http://noctua.at/en/products/cpu-cooler-retail/nh-d9l

Pull all 4 empty HDD trays in column 4 (slots 12-15) and cut a piece of clear packing tape about 1" x 3.5". Apply the clear packing tape to the silver metal grill on the inside (right behind the black plastic vents). This will completely seal up the dummy drive trays on the unused slots and force more air through your remaining 12 drives.

You're noticing a rise in HDD temps as you get closer to slots 12-15 because the static pressure drops due to there being less air resistance in slots 12-15 (even with the dummy trays). Completely sealing those unused slots will force more air across the occupied slots. Give it a try, worst case, it takes 2 minutes to remove the tape. Also, the dummy trays that my used SC836 came with had much more room all the way around them compared to a modern HDD, which I think is a potential design flaw when you're fighting ambient temps and trying for a quiet case. :D
 

melloa

Wizard
Joined
May 22, 2016
Messages
1,749

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
Pull all 4 empty HDD trays in column 4 (slots 12-15) and cut a piece of clear packing tape about 1" x 3.5". Apply the clear packing tape to the silver metal grill on the inside (right behind the black plastic vents). This will completely seal up the dummy drive trays on the unused slots and force more air through your remaining 12 drives.
  • 16/16 bays have drives in them, but before I populated the last 4 I did something similar and did notice a decrease in temp for the 12/12 existing drives at the time.
  • Taking your idea a step further, I have dummy trays with SSDs in them above the HDD bays, and I sealed off the "grates" with velco (from the inside, best material I had on hand) to ensure air wasn't coming through there.
Appreciate the kind advice nevertheless. :)
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202

melloa

Wizard
Joined
May 22, 2016
Messages
1,749
Damn you are fast! Beat me to it.

The suggestion is good, don't take me wrong, you just can't do it with your # of disks you have, but he has a point here:

I think is a potential design flaw when you're fighting ambient temps and trying for a quiet case.

I'd get it on the closet and install the A/C ducts you are planning, to cool it there and let is be noisy far way from you :D
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
The suggestion is good, don't take me wrong, you just can't do it with your # of disks you have, but he has a point here:
I'd get it on the closet and install the A/C ducts you are planning, to cool it there and let is be noisy far way from you :D
  • The suggestion is better than good ... it worked when I tried it before putting the last 4 disks in.
  • The problem is that we are talking about a 1200 square foot condo. The closet requires an air intake to the living area which is a matter of feet from the bedroom. Not sure how much the closet will deaden the noise, but that intake will allow some through. Moral of the story, these guys don't belong in condos if you care about noise.
 

TXAG26

Patron
Joined
Sep 20, 2013
Messages
310
  • The suggestion is better than good ... it worked when I tried it before putting the last 4 disks in.
  • The problem is that we are talking about a 1200 square foot condo. The closet requires an air intake to the living area which is a matter of feet from the bedroom. Not sure how much the closet will deaden the noise, but that intake will allow some through. Moral of the story, these guys don't belong in condos if you care about noise.

You could also go to the other extreme and get a large case from Caselabs that holds 20-40 HDDs, fill it up with quiet 120mm fans and have a case significantly cooler and quieter than a 3U rack mount. The Caselabs case in my signature holds 36+ 3.5" HDDs using four 3x5 and four 3x4 HDD racks. I have nowhere near that many drives, but the future proofness and nearly unlimited fan configuration options make you quickly forget about the modest price premium, and it turned out being less expensive and quieter than a new 4U 846 SM rackmount.

http://www.caselabs-store.com/magnum-th10a/

http://www.caselabs-store.com/content/CaseLabs Tech sheet2.pdf
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
Vive y aprende

The situation will ultimately be rectified when the fiancé forces me to sell / rent out the bachelor pad and buy a single family home. Which is OK, as holding a real estate license on the side, I won't show her any home that can't fit less than a datacenter in the basement. In the interim, I placed the server on my balcony and built a "house" for it, so it doesn't get wet. I ran an extension cord off the UPS and 5 drops of Cat6 to it. Not sure why I didn't think of this before. It is whisper quiet now, or at least I can't hear it. ;) Inside joke @melloa ...

On a serious note, I ponder whether one of the following "ideas" may have any merit. They are certainly unorthodox and hate even having to contemplate them.

Preface
  • I'm so close to being sub-40°C with the Optimal Profile @ 78°F ambient with a mean of 40.3°C across all drives. But those bottom, right drives still hit 44°C at that ambient. If you reference the attached picture, you will see what is causing this. The inlets on the backplane at bottom, left of the the picture (bottom, right drive bays) are tiny! It isn't until ambient is 72°F that the max drive temperature is <= 40°C (exactly 40.0°C) where the mean across all drives is 37.2°C.
  • I prefer not to run the A/C at 72°F, thereby allowing the Optimal profile to be used, for two reasons: (a) Most importantly, that is uncomfortably cold for me while lounging around at home in shorts and an undershirt and would have to wear a sweatshirt, (b) Less importantly, I don't want to incur the expense of running the A/C 24/7 from Spring to Fall.
  • As such, I believe that means I will need to continue to use the deafening "Full Speed" fan profile.
  • If I could stop making useless posts, I could finish the closet where the server will reside. The airflow accommodations to be made include intake vent at floor level and exhaust vent at ceiling. The intake will connect the closet to the living area and exhaust will connect the closet to a utility closet where the air handler will ultimately evacuate the exhaust. I will start with a passive approach, but if convection by itself doesn't produce cool enough temperatures, fans can be added to those vents.
  • Surely containing the server in a closet will reduce some noise, somewhat offset by that air intake allowing noise to escape to the living area, which is about ~10 feet from my bedroom. I of course won't know what the sound level is until I put more holes in the wall, but want to have a plan to mitigate excessive noise if need be.
Note
  • What I'm after is a reasonable sound level, that isn't obnoxious and doesn't interrupt watching movies or keep me awake at night. I'm by no means pursuing whisper quiet.
Ideas
  • Application of Dynamat to the server.
  • Application of sound dampening material such as foam or acoustic panels inside the closet.
Does anyone have experience / advice on either idea?

Optimal Profile - New Fans.jpg


Backplane.jpg
 

melloa

Wizard
Joined
May 22, 2016
Messages
1,749
I placed the server on my balcony and built a "house" for it, so it doesn't get wet.

hmmm ... did you take the idea from the picture I've sent? You got to pay royalty to Bryan...
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
Update: I'm happy to share updated results that allow the high temperature issue to be considered fully resolved. As more thoroughly described below, addition of 2 x Noctua NH-D9L CPU fans, connecting the Chassis Middle Fans to Zone 1, sealing the previously open PSU slot with a Dummy PSU cover, and use of the Heavy I/O fan mode allow for all HDD temperatures to remain under 40.0°C at a very reasonable sound level. A summary of the HDD temperatures, by fan mode, at 78°F ambient, follows:

EDIT: The fan script can be crossed off the list of open items thanks to @Stux. I edited an old script created by Kevin Horton, which is rather simple in nature - it changes fan mode based on drive temperature (as noted below). I believe between all hardware optimizations documented herein and that script, my HDD temps are as optimized as they will get.

  • 40°C or higher = Full
  • 38°C-39°C = Heavy IO
  • 37°C or lower = Optimal
Reference: https://forums.freenas.org/index.ph...-fan-zone-controller.46159/page-5#post-384923

Code:
Fan Mode	Tavg	Tmin	Tmax	Tavg Δ  Tmax Δ
Standard	38.8°C  37.0°C  41.0°C  -1.5°C  -3.0°C
Full		34.4°C  33.0°C  36.0°C  -2.1°C  -4.0°C
Optimal	 39.8°C  38.0°C  42.0°C  -0.6°C  -2.0°C
Heavy I/O   36.6°C  35.0°C  39.0°C  *	   *
* Heavy I/O Measurements not previously recorded
Δ = Compared to prior measurements without optimization

Configuration
  • Prior
    • CPU Cooling: Passive Heat Sink - SNK-P0048P, Air Shroud - MCP-310-39001-0N
    • Rear Exhaust Fans: 2 x FAN-0125L4 connected to FAN5/6
    • Chassis Middle Fans: 3 x FAN-0126L4 connected to FAN2/3/4
  • Updated
    • CPU Cooling: Removed Passive Heat Sink - SNK-P0048P and Air Shroud - MCP-310-39001-0N, Replaced with 2 x Noctua NH-D9L connected to FAN1/2
    • Rear Exhaust Fans: 2 x FAN-0125L4 connected to FAN5/6 (no change)
    • Chassis Middle Fans: 3 x FAN-0126L4 connected to FANA by splitter
    • Added Dummy PSU Cover - CSE-PT0130L
Fan Mode
  • With the "Prior Configuration" CPU cooling and HDD cooling were lockstep with all fans in Zone 0.
  • Now, with the "Updated Configuration," CPU cooling and HDD cooling have been decoupled with the addition of 2 CPU fans and a fan splitter to connect the 3 Chassis Middle Fans to FANA (Zone 1).
  • Previously, the Chassis Middle Fans, connected to Zone 0, were controlled based on CPU temp with targets of 30% (Optimal) or 50% (Standard and Heavy I/O). Full fixes them @ 100%.
  • Now, the Chassis Middle Fans, connected to Zone 1 are controlled based on PCH temp with a target of 50% (Standard) OR fixed at ~30% (Optimal) or 75% (Heavy I/O). Full fixes them @ 100%.
  • In conclusion, using the Heavy I/O profile allows a sub-40.0°C average (of 36.6°C), with no drive exceeding 40.0°C, at a very reasonable sound level.
Importance of Proper Airflow (Dummy PSU Cover)
  • My prior assertion that the bottom, right HDD temps are much higher than the temperature of their brothers and sisters due to the small cutout in the backplane behind these HDDs holds true, but is not responsible for the entire variance between the coolest HDD and the warmest HDD.
  • I removed the bottom PWS-1K21P-1R to reduce noise and simply left that slot open (during testing with the old and new fans). It didn't occur to me that this could have such a negative impact on temperatures. The chassis middle fans were pulling a considerable amount of air through that open slot, so I tried covering it with painter's tape and noticed a very nice improvement. Ultimately I purchased a Dummy PSU Cover - CSE-PT0130L to replace the painter's tape. I also sealed off all other vents on the chassis.
  • My assumption as to why this had a favorable impact:
    • The fan closest to the right HDD bays is going to always move X CFM of air (for purposes of this argument);
    • There are two areas where that air could come from;
      • Around the HDDs by way of cutouts through the backplane (p1), or
      • Through the open PSU slot (p2);
    • With the pressure differential, p1>p2, most of the air moved by that fan is going to come from the open PSU slot and not around the HDDs as we want.
    • See illustration below.
  • This resulted in a temperature reduction of -3°C / -4°C / -2°C (Standard/Full/Optimal) for the two bottom, right HDDs,
  • which helped to drive a reduction in the mean temperature of -1.5°C / -2.1°C / -0.6°C (Standard/Full/Optimal).
Column 4 Temps - PSU Slot.jpg CSE-PT0130L.jpg

CPU Fan Impact
  • While I don't have a plethora of empirical data as is the case with the HDDs temperatures, with the passive heatsink and air shroud in place, my CPU temps were typically: CPU1 40-45°C and CPU2 57-62°C.
  • As I type this, they are currently CPU1 42.0C and CPU2 56.0°C (full speed fan profile). CPU2 (rear) always runs hotter as it is picking up second-hand air from the HDDs and third-hand air from CPU1. Reference below image.
  • I expected some reduction in temperatures by adding CPU fans; however, that doesn't seem to have occurred. I believe this suggests that the passive heatsink with air shroud is rather well engineered and CPU Fans aren't needed.
  • I did see a nominal increase in RAM temperatures with the removal of the air shroud (average +2°C); however, that increase could be due to the addition of +48GB RAM. I believe they are fine as is and removing the air shroud isn't detrimental to RAM.
    • CPU1 - Left Bank: 38°C
    • CPU1 - Right Bank: 39°C
    • CPU2 - Left Bank: 45°C
    • CPU2 - Right Bank: 45°C
CPU Fans.jpg

Comparison of HDD Temperatures, Prior v. Updated Configuration
Standard Speed Fan Mode
Standard.png

Full Speed Fan Mode
Full.png

Optimal Speed Fan Mode
Optimal.png

Heavy IO Fan Mode
Heavy IO.png


Reference - Fan Mode
  • 0x0 - Standard: Zone 0 = BMC control based on CPU temp (target = 50%) | Zone 1 = BMC control based on PCH temp (target = 50%)
  • 0x1 - Full: Zone 0 = Fixed @ 100% | Zone 1 = Fixed @ 100%
  • 0x2 - Optimal: Zone 0 = BMC control based on CPU temp (target = 30%) | Zone 1 = Fixed @ ~30%
  • 0x4 - Heavy I/0: Zone 0 = BMC control based on CPU temp (target = 50%) | Zone 1 = Fixed @ 75%
  • where Zone 0 = CPU / System Fans = FAN1, FAN2, etc. and where Zone 1 = Peripheral Fans = FANA, B, etc.
  • Citation: https://forums.servethehome.com/index.php?resources/supermicro-x9-x10-x11-fan-speed-control.20/
 
Last edited:
Status
Not open for further replies.
Top