Supermicro fan control, IPMIUTIL or FEEIPMI?

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
... The down side is that I can't monitor all three fans. But the alternative was to put two on the CPU PWM and only one on the chassis PWM.
Buy a 3-pin fan extension cable of adequate length. Remove the black and red wires (12v & Ground) and the male plug shell then insert the remaining yellow wire into the male connector on your fan splitter and connect the female end with the single yellow wire into an available fan header on the motherboard. Repeat as desired.

This will send the tachometer "tac" signal from a fan to the motherboard.
 

Z300M

Guru
Joined
Sep 9, 2011
Messages
882
@Pharfar:
Fan1-Fan4 and Fan A are for System/CPU Fan headers - correct. That does not mean that FanA is the best choice for your CPU fan.

MANY OF YOU ARE THINKING LIKE DESKTOP SYSTEM BUILDERS, NOT LIKE ENTERPRISE SERVER BUILDERS.

Desktop fans are designed to be quiet. Server class fans are designed to move air. These are not the same priorities. When you use quiet fans with a server motherboard, you need to think about your cooling. For most NAS applications the stock CPU cooler will be OK when plugged into any of the fan headers, but if you build a system that is going to run your CPU at full load then you need to make certain your fan is running from a fan header that monitors the CPU.

I will summarize for Supermicro boards:

1. There are two PWM zones on most SuperMicro boards: One PWM zone monitors system temp, and that zone controls FanA. The other PWM zone monitors CPU temp, and that zone controls Fan1-4. Fans1-4 are controlled as one group, although all fans report their RPM individually. If you want your CPU fan to be responsive to CPU temp, then it should be connected to one of the Fans1-4 but it doesn't matter which one (except for specific motherboards that designate Fan1.)

2. PWM can be used to control fan speed, but it does not control RPM directly. The exact methodology is outside the scope of this thread. Just keep in mind that 100% is full speed, and 50% is less - but 50% does not mean 50% of the RPM. Do not expect a fan with 2000 RPM at 100% to run at 1000 RPM when set to 50%. PWM does not work that way. The lower the PWM percentage, the less predictable a specific fan's actual performance will be. Below 30%, many fans will not run reliably which is why you never see PWM power settings below 30%.

3. SuperMicro's fan settings are designed for enterprise server fans, which run at very high RPM's compared to quiet desktop fans. Enterprise server fans are designed to move air, and noise is not a big concern. So, at the end of the day, do not expect the fan controls to behave the same way with our quiet low RPM desktop fans as they do with high RPM enterprise class server fans. This is one of the reasons our low RPM fans run at minimum RPM when set to Standard mode (50%) or Optimum mode (30%.) The PWM controls on Supermicro boards are not designed for low RPM fans.

A last thought:
It doesn't really matter which fan header you use for a CPU fan as long as you have enough cooling. However, if you want the CPU fan to be responsive to CPU temp, then it should be in FAN 1-4. Using a utility to alter the fan speed alerts by lowering speed thresholds will make the alarms go away, but won't alter the basic way the fans are controlled.

If you want fans to run at a specific RPM, you need to use 3 pin fans, or use some type of a manual controller.

Another last thought:
Check your disk temps. On my system, I just discovered one disk that is running at 46c, the others are OK. I'm going to revise my fan strategy.
Surely Supermicro is not thinking like a desktop builder (because it isn't one) when it suggests that the CPU fan should be connected to Header A: there must be some reason for the designer of a server motherboard to recommend connecting the CPU fan to Header A.

It' s true that some of the Supermicro fans (the small ones) run at very high speeds (even up to 22,500rpm), but their larger fans run at slower speeds; e.g., the max. for their 120mm fans is 4,000rpm. And some of those smaller high-speed fans are still not moving as much air as the larger, slower ones.

My FreeNAS system is built in a tower case with 3 x 120mm and 1 x 200mm case fans connected to Fan Headers 1-4 and currently running at 400-600rpm. The CPU fan connected to Header A is running at 1100rpm. The PCH temp. is 39C, the CPU temp. 31C, the DIMM temps. 21-22C. Fan mode is set to Optimal.

The drive cages have their own fans, speed unknown; the drive temps. range from 25-30C.
 
Last edited:

pjc

Contributor
Joined
Aug 26, 2014
Messages
187
This will send the tachometer "tac" signal from a fan to the motherboard.
That's what the splitter did that I referenced in my original post. The catch, as you're aware, is that you can only monitor the tach signal from a single fan, not all three.
 

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
Surely Supermicro is not thinking like a desktop builder (because it isn't one) when it suggests that the CPU fan should be connected to Header A: there must be some reason for the designer of a server motherboard to recommend connecting the CPU fan to Header A.
There is a lot of confusion about which headers for SuperMicro boards are for the CPU and which are for the chassis.

After reading 300 or more posts in the FAQ section at SuperMicro and 10 or 12 manuals for the current line of "Workstation" and low-end server boards it appears that FAN-A is the single sole connection for chassis fans and FAN-1 through FAN-4 can be used for the CPU but SuperMIcro expects it to be on FAN-1 as this appears to be the header they monitor by default to adjust the PWM signal though they must monitor the others as well as others would have noticed this.
 

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
That's what the splitter did that I referenced in my original post. The catch, as you're aware, is that you can only monitor the tach signal from a single fan, not all three.
The splitter sends the TAC signal from only one fan back to the motherboard (the one connected to the male splitter header with all with 4 pins).

What I am suggesting is that you can use one pin from a 3-pin fan extension cable to wire the tac leads from any of the other fans you have connected to spare headers on the motherboard and use these spare headers to monitor these fans.
 

pjc

Contributor
Joined
Aug 26, 2014
Messages
187
What I am suggesting is that you can use one pin from a 3-pin fan extension cable to wire the tac leads from any of the other fans you have connected to spare headers on the motherboard and use these spare headers to monitor these fans.
Ah, so control their speed using the chassis header, but use the spare CPU headers simply to monitor the tach? Hadn't thought of that...

And, yes, FAN A is chassis and FAN 1-4 are controlled based on CPU temperature. I've confirmed that with SuperMicro tech support.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
Ah, so control their speed using the chassis header, but use the spare CPU headers simply to monitor the tach? Hadn't thought of that...
This is where a mis-spent youth with a soldering iron comes in handy ;)
Here is a diagram.
SuperFans.jpg


For the extra tac leads you can just buy some of these for fans you wish to monitor and disconnect all but the yellow (3rd) wire from the female connector and connect the male end of the yellow (3rd) wire to the appropriate connector on the fan splitter.
 
Last edited:

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
... The "A" of FAN-A would stands for "Add-on card" who allegedly control the chassis fans.
and I can confirm that this is supposed to be controlled by the "System" sensor not the "Peripheral" sensor.

Good to know it works in "Optimal" mode on the X10SL7 as that is the same board I have.

I will be opening a ticket with SuperMicro to get the exact location for the sensors as I'm not particularly interested in spending an hour pouring over the board with a magnifying glass trying to read the 2-point type to find the surface-mount thermistors.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
The ideal solution would be a dedicated controller (some Arduino kind of thing) that receives the hard drive temperatures over USB/Serial, monitors the fans and sends the appropriate PWM signal to the fans.

I might try this as a summer project...
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
According to Eric (https://forums.freenas.org/index.ph...s-with-ipmi-using-ipmitool.23571/#post-178284) it's the PCH temp who controls FAN-A. Seems like it's "pick the one you prefer", SuperMicro doesn't do a good job at clearly describe some of the things about theirs MB... :rolleyes:

Arf, look at what I've designed the last week: http://img.ohm.free.fr/files/w_866ba8197536bb5c3099b821b1d6811c.png and http://img.ohm.free.fr/files/w_afad46779dbfec6b5069e8557bfd5496.png :D

The typon is for a stripboard (in blue) with the jump wires in red (the yellow is for special junctions who use the components leads or a blob of solder), I've not exactly followed the best practices for some of the things but I wanted to reduce the size of the board the maximum possible...

The concept is:
  • MB CPU FAN header PWM signal pass-through + PWM signal buffering for a second fan
  • MB CPU FAN header Tach signal emulation
  • Temp sensing for the HDD and PWM generation for 6 fans with adjustables offset and gain of the slope
  • Watchdog on the tach signals for the 8 fans with minimum speed threshold independently adjustable for each fan; if the watchdog is triggered by one or more fans:
    • Stale recovery + air flow compensation by overriding all the PWM signals to 100%
    • Then after 5 to 20 sec (adjustable timer), if the watchdog is still triggered, a beeper beep and a soft shutdown is initiated (via the power button header on the MB) <-- after rethinking I'll probably put the beeper directly on the output of the watchdog, so it'll beep as soon as the watchdog is triggered and not only after the delay

It uses a LM35 + LM324 for HDD temp sensing and PWM generation and CD4000 logic for the watchdog and timers.

I've actually simulated some of the parts of the circuit with success but I haven't built it yet, so if somebody wants I can post hi-res schematic and the BOM but it's still untested for now.

Why this DIY controller? because the fan modes of the X10 are just "quiet but not enough cooling (my HDDs are at 40-41 °C idle...)" or "plenty cooling but VERY loud (and I sleep at 2 meters of the server...)", there is no golden mean and moreover the HDD temp can't control the fans speed (and it's just the most important thing I think...)

Yeah, I can use an arduino or some other MCU but I love to use discrete design whenever possible and unlike a MCU it can't freeze or crash and fail to do it's job :D

PS: "you can just buy one of these for each fan" you seems to have forgotten the link :p
 
Last edited:

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
The ideal solution would be a dedicated controller (some Arduino kind of thing) that receives the hard drive temperatures over USB/Serial, monitors the fans and sends the appropriate PWM signal to the fans.

I might try this as a summer project...
I think the assumption we have to make is that whatever rotational speed used to move an appropriate volume of air to keep the disks well below 40 deg. C. will also be adequate to keep the CPU only moderately "warmish" which I think we can assume (trust by verify).

My ideal solution would be to be able to have the on-board thermal management solution run the case fans at as slow a speed as possible to:
  1. Keep the drives under 40 deg. C.
  2. Minimize fan noise to 20 - 25 dBA
I find it encouraging that I've seen some references in the more recent SuperMicro manuals to "Low Speed Fan Control". Now whether this is real or marketing speak is the question.

While I fully understand that these boards are, for the most part, engineered for 1/2/3/4U rack use SuperMicro does offer a range of boards targeted for Workstation Use as well as the more recent Gaming series so there are some outliers. My X9SCL that I've been running for several years as a workstation came in a SuperMicro Mid-Tower case so they do a market for quality boards in a desktop chassis.

Small NAS systems in mid to full tower cases that run quietly are, however, a completely different set of requirements (waiting for jgreco to jump in with "Desktop boards for Desktop systems")
 

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
Nice - but too much trouble for me these days. I actually got my soldering iron out last night for the first time in a year as rather than waiting for delivery on 3-wire fan extension cables I hacked it up using cables from dead fans and Molex adapters so I needed to solder some wires together and put some heat shrink tubing over them.
PS: "you can just buy one of these for each fan" you seems to have forgotten the link :p
Whoops - link added to expensive 3-wire fan extender.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
According to Eric (https://forums.freenas.org/index.ph...s-with-ipmi-using-ipmitool.23571/#post-178284) it's the PCH temp who controls FAN-A. Seems like it's "pick the one you prefer", SuperMicro doesn't do a good job at clearly describe some of the things about theirs MB... :rolleyes:

Arf, look at what I've designed the last week: http://img.ohm.free.fr/files/w_866ba8197536bb5c3099b821b1d6811c.png and http://img.ohm.free.fr/files/w_afad46779dbfec6b5069e8557bfd5496.png :D

The typon is for a stripboard (in blue) with the jump wires in red (the yellow is for special junctions who use the components leads or a blob of solder), I've not exactly followed the best practices for some of the things but I wanted to reduce the size of the board the maximum possible...

The concept is:
  • MB CPU FAN header PWM signal pass-through + PWM signal buffering for a second fan
  • MB CPU FAN header Tach signal emulation
  • Temp sensing for the HDD and PWM generation for 6 fans with adjustables offset and gain of the slope
  • Watchdog on the tach signals for the 8 fans with minimum speed threshold independently adjustable for each fan; if the watchdog is triggered by one or more fans:
    • Stale recovery + air flow compensation by overriding all the PWM signals to 100%
    • Then after 5 to 20 sec (adjustable timer), if the watchdog is still triggered, a beeper beep and a soft shutdown is initiated (via the power button header on the MB) <-- after rethinking I'll probably put the beeper directly on the output of the watchdog, so it'll beep as soon as the watchdog is triggered and not only after the delay

It uses a LM35 + LM324 for HDD temp sensing and PWM generation and CD4000 logic for the watchdog and timers.

I've actually simulated some of the parts of the circuit with success but I haven't built it yet, so if somebody wants I can post hi-res schematic and the BOM but it's still untested for now.

Why this DIY controller? because the fan modes of the X10 are just "quiet but not enough cooling (my HDDs are at 40-41 °C idle...)" or "plenty cooling but VERY loud (and I sleep at 2 meters of the server...)", there is no golden mean and moreover the HDD temp can't control the fans speed (and it's just the most important thing I think...)

Yeah, I can use an arduino or some other MCU but I love to use discrete design whenever possible and unlike a MCU it can't freeze or crash and fail to do it's job :D

PS: "you can just buy one of these for each fan" you seems to have forgotten the link :p

Phew... Remind to brush up on discrete electronics...

Anyways, I'm not worried about the failure mode if the whole thing crashes - the default failure mode would be no signal on the PWM line and no monitoring, which effectively leaves the fan at 100% and no feedback to the motherboard, which should trigger the fan failure alert.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
My ideal solution would be to be able to have the on-board thermal management solution run the case fans at as slow a speed as possible to:
  1. Keep the drives under 40 deg. C.
  2. Minimize fan noise to 20 - 25 dBA
Funnily enough I've designed this controller for the exact same requirements :D

Yeah, I can understand that's not for everyone. It's easier to use an arduino or even an already made rheobus ;)

Edit: "and no feedback to the motherboard, which should trigger the fan failure alert." nop, because the MB can't do the difference between 0 RPM and no fan connected. It's partly because of that that I bypass the built-in fan watchdog ;)
 
Last edited:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
Nice - but too much trouble for me these days. I actually got my soldering iron out last night for the first time in a year as rather than waiting for delivery on 3-wire fan extension cables I hacked it up using cables from dead fans and Molex adapters so I needed to solder some wires together and put some heat shrink tubing over them.
Whoops - link added to expensive 3-wire fan extender.

Phew, those things are crazy expensive, given that Noctua sells three nicely sleeved ones for 7 bucks...
 

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
Phew, those things are crazy expensive, given that Noctua sells three nicely sleeved ones for 7 bucks...
- Link adjusted for Noctuas - Nice find Eric

Funnily enough I've designed this controller for the exact same requirements :D

Yeah, I can understand that's not for everyone. It's easier to use an arduino or even an already made rheobus ;)

Edit: "and no feedback to the motherboard, which should trigger the fan failure alert." nop, because the MB can't do the difference between 0 RPM and no fan connected. It's partly because of that that I bypass the built-in fan watchdog ;)
My thoughts are:
  1. If we can get the fan low speed threshold on the built in controller down to 300rpm and;
  2. Use many (4) 140mm 4-wire (PWM) fans and;
  3. Use the System sensor to control the FAN-A and adjust the high and low setpoints
Then we should be able to have a system that starts out at 300RPM at idle and only starts to ramp up as the case temperature starts to rise. It will take some playing about to get an adequate threshold but I believe that even at 300RPM 140mm fans should be able move a large enough volume of air to keep the drives cool under minimal load (low and slow). The trick will be adapting to the curve of the built in controller so the drives stay cool under moderate and high loads with an adequate margin.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
1. Not a problem thanks to this thread: https://forums.freenas.org/index.ph...or-thresholds-with-ipmi-using-ipmitool.23571/
2. Not really a problem but not every case can use 140 mm fans
3. There's your problem! You can't change the setpoints/slope/whatever. You can only change the fan mode.

Just for info I've 3x 120 mm fans for 8 drives with almost no restrictions on the air path and at 600 RPM the drives are at roughly 40°C idle. I doubt that 140 mm fans at 300 RPM are enough (mainly with a rather restrictive air path like on the server cases) but I can be wrong.
 

GrumpyBear

Contributor
Joined
Jan 28, 2015
Messages
141
Yup, saw that thread and I'm itching to play with it.
1. Not a problem thanks to this thread: https://forums.freenas.org/index.ph...or-thresholds-with-ipmi-using-ipmitool.23571/2. Not really a problem but not every case can use 140 mm fans
No but I can ;)though I've installed 120mm ones just in case I have to go with the stock minimum speed.
1. Not a problem thanks to this thread: https://forums.freenas.org/index.ph...or-thresholds-with-ipmi-using-ipmitool.23571/3. There's your problem! You can't change the setpoints/slope/whatever. You can only change the fan mode.
understood. That's why I want to use multiple fans. Hopefully the curves on the FAN-A header will work.
1. Not a problem thanks to this thread: https://forums.freenas.org/index.ph...or-thresholds-with-ipmi-using-ipmitool.23571/Just for info I've 3x 120 mm fans for 8 drives with almost no restrictions on the air path and at 600 RPM the drives are at roughly 40°C idle. I doubt that 140 mm fans at 300 RPM are enough (mainly with a rather restrictive air path like on the server cases) but I can be wrong.
you've got the exact same hardware as me down to the drives but I'm using the fractal design Define R4 case so it likely has less airflow impedance than your rack-mount chassis so I might be able to get away with it. Once I dig out some more of my lighting kit I'll take some more photos showing the clearances around the drive bays.
 
Last edited:

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
It's not a classic rack-mount chassis, it's more like a big cubic desktop case with all the system in a rack who takes half the space of the case (the other half is used to trap the noise while letting the air flows). I'll try to find some pictures to explain a bit better.

Edit:

b_w_9974290734ced1e21043cd5f93e5e807.png


In red the drives, in blue the MB and in green the PSU. The front in on the left of the picture, the rear on the right. The air is taken on the top rear through an air filter, sucked by the fans on the top separation (not represented on the 3D view), blown at the front of the drives and above the MB, then it passes through the rack floor (by holes under the MB, not represented) and is exited at the bottom front. Please forgive the crudeness of the 3D modelling, I just did it to get the exact dimensions for each part. There is more details on the pictures below (but they are old and the case wasn't finished at that time).
w_991adb7a0b50b914ff839c4e44b2e8b8.jpeg

w_f4d620dac6fc10793b50deea0fd5ea72.jpeg

w_a74d9317995fa7896eaca7b5a0b9ca9d.jpeg

w_69531a157d7dacb7c0a80777f9749fb3.jpeg

w_5fe3e23936e66101b678ca695f078ec4.jpeg

w_547999520cbd2734a1f80b040a4e87a7.jpeg
 
Last edited:
Top