Out of my league: How do I adjust fan speed (Supermicro CSE-846, X9DRi-F) - fans quiet with hot disks

esamett

Patron
Joined
May 28, 2011
Messages
345
Please excuse my inexperience. This is my first excursion into using server hardware. I just "finished" moving my FreeNAS from consumer hardware to the server case. 2x11-Z3 disk arrays, 2 spares. dual SSD boot drives inside case powered from extra Molex connector from power distributor. Set up network NIC.

When I turn on system the 5 big case fans spin up loudly initially and then get rather quiet when Motherboard boots. I promptly got disk overheat errors from FreeNAS, even when I replaced the case cover to promote air flow. I rebooted several times and got similar disk overtemperature warnings. Since the system fans currently spin slowly and quietly I want to adjust the speed to increase cooling. Its my guess that the fans can cool my disks when spun up to speed. I looked through the BIOS menus and found no fan control options. I search the forum and found discussions about "scripts" and "IPMI" of which I have no knowledge.

Please guide me in the right direction about fan control. I would hate to have to go back to my consumer frankenstein build.
 

jon_2112

Explorer
Joined
Feb 8, 2016
Messages
52
Your motherboard has IPMI. This is a web interface for managing the motherboard. Make sure it's plugged in, figure out the IP address, and log into it via a web browser. You might need to reboot and go into the BIOS to figure out the IP address. The BIOS may or may not have a way to adjust the fan mode as well.

Here is what the settings mean:
You might want "HeavyIO"
 

Evertb1

Guru
Joined
May 31, 2016
Messages
700
and found discussions about "scripts" and "IPMI" of which I have no knowledge.
Those discussions about scripts and IPMI are most likely related to forum members with motherboards from Supermicro or so that have IPMI on board. Within IPMI you can set a couple of standard profiles for your fan control. If those are not sufficient you can enter and run a script in the console for finetuning your fan behavior. See this thread for more information. I could not find IPMI in the specs of your motherboard but it has the WfM 2.0 interface that was the predecessor of IPMI.

One note about your system. Your motherboard supports ECC ram but as far as I know your CPU not. To have the ECC functionality both the CPU and the motherboardf must support it.
 
Last edited:

nanodec

Dabbler
Joined
Jan 14, 2015
Messages
34
Those discussions about scripts and IPMI are most likely related to forum members with motherboards from Supermicro or so that have IPMI on board. Within IPMI you can set a couple of standard profiles for your fan control. If those are not sufficient you can enter and run a script in the console for fine-tuning your fan behavior. See this thread for more information. I could not find IPMI in the specs of your motherboard but it has the WfM 2.0 interface that was the predecessor of IPMI.

One note about your system. Your motherboard supports ECC ram but as far as I know your CPU not. To have the ECC functionality both the CPU and the motherboard must support it.
er. Supermicro isn't all that's available. Either way, you should find fan control in your BIOS. My tower was running warm under auto mode, and switched it to Maximum - giving me another 10C. Look up the documentation for your board and figure out the speed control. Airflow? Make sure your pushing lots of air cross those platter boxes...
I'm sitting at 30C in ambient temp... I keep my drives at 128 (Minimum power usage without Standby (no spindown).

I'd say if your running some hot temps, take a look at your cooling situation...
 

esamett

Patron
Joined
May 28, 2011
Messages
345
I did some BIOS poking and internet reading...
Supermicro X9DR3/i-F
SMC version 3.3, 7/12/18

BIOS:
Advanced ->-> CPU Power Management Configuration-> Energy Performance
changed Balanced Performance to Performance

The fans ran fast until the FreeNAS bootloader screen came up. Fans then slowed down again. I guess FreeNAS changes the fan setting(??) I'm waiting to see what temps do...

IPMI then??

I will read the reference material kindly suggested. All advice appreciated.
 

nanodec

Dabbler
Joined
Jan 14, 2015
Messages
34
I did some BIOS poking and internet reading...
Supermicro X9DR3/i-F
SMC version 3.3, 7/12/18

BIOS:
Advanced ->-> CPU Power Management Configuration-> Energy Performance
changed Balanced Performance to Performance

The fans ran fast until the FreeNAS bootloader screen came up. Fans then slowed down again. I guess FreeNAS changes the fan setting(??) I'm waiting to see what temps do...

IPMI then??

I will read the reference material kindly suggested. All advice appreciated.
no dude it's all BIOS.. fan speed (at least all that I've ever seen) can be overrode through the BIOS.. fans are cheap.. I run them till they die then replace, I use 140mm so I get lower rpm (quieter) but run them full speed. Get good fans with good bearings and let them roll.
 

jon_2112

Explorer
Joined
Feb 8, 2016
Messages
52
no dude it's all BIOS.. fan speed (at least all that I've ever seen) can be overrode through the BIOS.. fans are cheap.. I run them till they die then replace, I use 140mm so I get lower rpm (quieter) but run them full speed. Get good fans with good bearings and let them roll.

I don't think this is true. My Supermicro board doesn't have any fan settings in the BIOS. They're in the IPMI only, and can be edited from within FreeNAS using ipmitool.

Also, OP is using a Supermicro rackmount chassis that has some pretty custom cooling and fans.
 

nanodec

Dabbler
Joined
Jan 14, 2015
Messages
34
weird.
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,740
Instead of invoking the BIOS setup, you should connect to the IPMI UI with a web browser. At least that's how all my Supermicro servers work. Default user: ADMIN, default password: ADMIN

There's a "Fan Mode" section in the "Configuration" menu:

fan.png


HTH,
Patrick
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
no dude it's all BIOS.. fan speed (at least all that I've ever seen) can be overrode through the BIOS.. fans are cheap.. I run them till they die then replace, I use 140mm so I get lower rpm (quieter) but run them full speed. Get good fans with good bearings and let them roll.

Supermicro boards do not do much, if any, control of their fans through the BIOS.

If you use good fans, they should almost never die. There's a reason servers use industrial grade fans which can (and do) last ten to fifteen years without failing. Those consumer grade fans gamers love, like the Noctuas, won't last anywhere near as long.
 

nanodec

Dabbler
Joined
Jan 14, 2015
Messages
34
Supermicro boards do not do much, if any, control of their fans through the BIOS.

If you use good fans, they should almost never die. There's a reason servers use industrial grade fans which can (and do) last ten to fifteen years without failing. Those consumer grade fans gamers love, like the Noctuas, won't last anywhere near as long.
Agreed - I think in the 20 years working in IT I could count on one hand how many server fans I've seen die.
 

esamett

Patron
Joined
May 28, 2011
Messages
345
Progress, Partial Success:

I got IPMI IP address from bios and used browser to log in. Fan options are: Standard Speed, Full Speed, Optimal Speed and Heavy I/O. I checked fan speeds with Server Health ->Sensor Readings. At "Full Speed" fans 1, 2, 3, 5, 6 run at around 6-7000 RPM and sound like a shop vac. At the other three settings (standard, optimal and heavy i/o) fan speeds run around 3000 RPM without significant difference in sound or measurements between them.

I started an all disk scrub to stress test system and checked a random disk temp via FreeNAS with: smartctl -a /dev/da0 | grep -i temperature. Temp started at 25'C, went to 26' when I decreased fan speed from Max to lower setting. I went back to Max when temp hit 28'C and temp is trending back down again and seeming stable at 27'C.

On the plus side I seem to have resolved my disk overheating problem. On the other hand it looks like there are only two effective IPMI fan speed options, 3000 and 6000 RPM. I have seen some posts about "fan scripts" for speed control. Are there options for controlling fan speed between these two speeds? If so please direct me to a NOOB tutorial on using FreeNAS scripts in general and fan speed in particular. Is a fan script something that gets executed once or at every boot up?

Thanks in advance.
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924

esamett

Patron
Joined
May 28, 2011
Messages
345
I found this comment:
https://www.ixsystems.com/community...upermicro-boards-using-pid-logic.51054/page-6

Well, you're not going to be able to set the PCM (Pulse Code Modulation) rates/duty cycle on X9 motherboards; they don't support this functionality. I wanted to do so, which is why I verified w/ Supermicro.

All you can do is set the basic 'fan mode', as I described above, using the standard IPMI commands described and used by @Glorious1 and many others here on the forum.

This all takes place at the IPMI level, meaning that it won't help you much to delve into the gory technical details of your mobo's Winbond WPCM450 chip.

And these links:


will make myself a cup of tea...
 

G8One2

Patron
Joined
Jan 2, 2017
Messages
248
My SuperMicro BIOS most certainly does have fan speed control. Its under Advanced tab, Hardware heath Configuration/System Fan Monitor

FanSpeed.jpg
 

G8One2

Patron
Joined
Jan 2, 2017
Messages
248
When i purchased my server, a majority of the fan noise was from the redundant power supplies. I purchased quieter ones to bring the fan noise to a tolerable level.
 

esamett

Patron
Joined
May 28, 2011
Messages
345

G8One2

Patron
Joined
Jan 2, 2017
Messages
248
No i believe i have an x8, but i would think any of the other SuperMicro boards would be generally the same. Though i could be wrong about that, you'd have to reference your user manual or find it in your BIOS
 

esamett

Patron
Joined
May 28, 2011
Messages
345
just tried to change duty cycle on my x9 based upon STH: Supermicro X9/X10/X11 Fan Speed Control

I was able to check fan mode with: ipmitool raw 0x30 0x45 0x00
I was able to set full fan mode with: ipmitool raw 0x30 0x45 0x01 0x01

I got error message for trying to set fan duty cycle for peripheral zone: ipmitool raw 0x30 0x70 0x66 0x01 0x01 0x16
I got error message for trying to set fan duty cycle for system: ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x24

[root@freenas ~]# ipmitool raw 0x30 0x70 0x66 0x01 0x01 0x16
Unable to send RAW command (channel=0x0 netfn=0x30 lun=0x0 cmd=0x70 rsp=0xcc): I
nvalid data field in request


Is this because of the x9 BMC limitations?

If so what are my options for setting fan speed somewhere around 4500 - between 3 and 6K
--??
 

esamett

Patron
Joined
May 28, 2011
Messages
345
also this text seems confusing: "
If you have the fan mode set to "Full" before setting the duty cycle them the BMC doesn't seem to ever change
the fans. I suppose its possible that if you go above the high threshold values then it might alarm and reset
them - but in normal operation they seem to be stable.
If you have the fan mode set to "Standard", "Optimal" or "Heavy IO" then the BMC will continue to try and
manage the fans and will override your settings within a couple of minutes."

I read it as you can set "full" or BMC wont change fan speed, and you cant set the other options because BMC will override. Doesn't this mean that nothing works?? Then OP uses example starting with setting full fan speed

fyi, my IPMI is set to "full" fan speed
 
Top