Disable IPMI kernel module in 9.3 [resolved]

[DaveF81]

Does anyone know how to disable the IPMI kernel module in 9.3? Tried setting tunable ipmi_load = NO, but that didn't work. For now, I've unloaded the module manually.

Why do I want to do this? Well, the IPMI is bugging out with the module loaded and spinning down all the case fans causing my drives to idle at a much higher temperature. Usually, they idle around 32C, but with this, I see them above 40C.

 

[Ericloewe]

Does anyone know how to disable the IPMI kernel module in 9.3? Tried setting tunable ipmi_load = NO, but that didn't work. For now, I've unloaded the module manually.

Why do I want to do this? Well, the IPMI is bugging out with the module loaded and spinning down all the case fans causing my drives to idle at a much higher temperature. Usually, they idle around 32C, but with this, I see them above 40C.

What's the hardware? That sounds like something that would've been caught long ago. My Supermicro X10 (AST2400 BMC) never gave me any problems with the IPMI module when I loaded it (it wasn't loaded by default in 9.2.1.x

 

[DaveF81]

This is the motherboard .. http://www.asrockrack.com/general/productdetail.asp?Model=E3C224D2I

Same thing happened in 9.2.1.x with the IPMI module loaded. At the time, I did upgrade the firmware for the IPMI back then, didn't resolve the issue.

Edit: Just verified my BIOS and IPMI firmware versions, both are at the latest.

 

[Ericloewe]

What are the thresholds like? If they're not set to your fans' specs, try this first:

https://forums.freenas.org/index.ph...or-thresholds-with-ipmi-using-ipmitool.23571/

I wrote it for Supermicro X10, but it should be very similar, if not exactly the same, for most IPMI implementations. Especially since ASRock also uses ASpeed BMCs.

Doesn't ASRock expose some fan control option in the IPMI web interface?

 

[DaveF81]

What are the thresholds like? If they're not set to your fans' specs, try this first:

https://forums.freenas.org/index.ph...or-thresholds-with-ipmi-using-ipmitool.23571/

I wrote it for Supermicro X10, but it should be very similar, if not exactly the same, for most IPMI implementations. Especially since ASRock also uses ASpeed BMCs.

Doesn't ASRock expose some fan control option in the IPMI web interface?

Thanks for this. Was able to set my lower and upper non-critical, however setting critical and non-recoverable, it fails:

[root@tardis] /mnt/volume1# ipmitool sensor thresh "FRNT_FAN1" lcr 500
Locating sensor record 'FRNT_FAN1'...
Setting sensor "FRNT_FAN1" Lower Critical threshold to 500.000
Error setting threshold: Command illegal for specified sensor or record type

Doesn't ASRock expose some fan control option in the IPMI web interface? .. Sadly not.

 

[Ericloewe]

https://forums.freenas.org/index.ph...or-thresholds-with-ipmi-using-ipmiutil.23331/

Try this version. It's a bit smarter (at least it feels smarter), but requires additional software. I assume ASRock uses fewer thresholds or maybe names them differently... *shrug*

 

[DaveF81]

I'll give that a try. Looking at the sensor list however, ASRock may have borked the sensor thresholds. Turns out most of the fans show 'na'.

Code:

CPU_FAN1         | 1700.000   | RPM        | ok    | na        | na        | 1500.000  | na        | na        | na      
REAR_FAN1        | 800.000    | RPM        | ok    | na        | na        | 700.000   | na        | na        | na      
FRNT_FAN1        | 900.000    | RPM        | ok    | na        | na        | 700.000   | na        | na        | na      

 

[Ericloewe]

Maybe ipmiutil has better luck talking to it.

 

[DaveF81]

Maybe ipmiutil has better luck talking to it.

Installed IPMITOOL from source on my Ubuntu workstation. At first glance it appears to have worked:

Code:

000d SDR Full 01 01 20 a 04 snum 11 REAR_FAN1        = 08 OK   800.00 RPM
    Setting SDR 000d sensor 11 to lo=07 hi=0a

But, when running 'ipmitool sensor list' on the FreeNAS box, I still see the same 'na' for all other thresholds except lnc.

Code:

REAR_FAN1        | 800.000    | RPM        | ok    | na        | na        | 700.000   | na        | na        | na

 

[WordlyBedouin]

I've got this same motherboard and pretty much the same issue. I've only got the FRNT_FAN1 installed and it basically gets turned off, so the entire case warms up (CPU, HDD, etc.) by about 10C. In BIOS I've configured the fan to be "Full On" instead of "Smart Fan". Its a Scythe 1200RPM 120mm fan which stays full on after a reboot, but sometime afterwards (I'm guessing FreeNas talks to the mobo) and slows it down/turns it off. So by the time I wake up in the morning the mini-ITX case is like a small oven cooking my parts.

At this point, I'd rather just turn off FreeNAS IPMI talking to the mobo entirely and just control the fan speeds directly in BIOS without it interferring. I've tried both of those utils/procedures and no luck. Any help on this would be appreciated.

 

[DaveF81]

Basically you need to unload the IPMI module at boot. To do this, go to Tasks > Init/Shutdown Scripts > Add Init/Shutdown Script:

• Type: Command
• Command: kldunload ipmi
• When: Post Init

Hit OK then reboot.

 

[WordlyBedouin]

I can't believe I completely missed the "Add Init/Shutdown Script" in the GUI! I've been mucking about with a "disable-ipmi.sh" script I threw together and been trying to get it to run under rc.d but no luck. I'll give this a try!! Many thanks!!!!

 

[j_r0dd]

Basically you need to unload the IPMI module at boot. To do this, go to Tasks > Init/Shutdown Scripts > Add Init/Shutdown Script:

• Type: Command
• Command: kldunload ipmi
• When: Post Init

Hit OK then reboot.

Thank you for this! I have the ASRock E3C224D4I-14S and I have woken in the morning a handful of times in the last month with emails from drives getting too hot. Last night I even had all my fans set to "Full On" in the BIOS. This morning I woke and my drives were pushing upper 40 degrees because there was a long test that was ran over night and the fans were running at their lowest speed. I have been in contact with ASRock because I was leaning towards a BIOS or BMC issue, but after I found this this may just be the culprit. I'm disabling all scheduled drive tasks until I have verified this holds up for some time. Has anybody created a bug report yet on this? I would say this is a serious bug. This could prematurely kill drives and if they get too hot the manufacturer could blame the temps and not honor any warranty. Thankfully a scrub was running last night or I would brought all my drives to a temp that I could never trust them again. Not good at all....

 

[WordlyBedouin]

Thank you for this! I have the ASRock E3C224D4I-14S and I have woken in the morning a handful of times in the last month with emails from drives getting too hot. Last night I even had all my fans set to "Full On" in the BIOS. This morning I woke and my drives were pushing upper 40 degrees because there was a long test that was ran over night and the fans were running at their lowest speed. I have been in contact with ASRock because I was leaning towards a BIOS or BMC issue, but after I found this this may just be the culprit. I'm disabling all scheduled drive tasks until I have verified this holds up for some time. Has anybody created a bug report yet on this? I would say this is a serious bug. This could prematurely kill drives and if they get too hot the manufacturer could blame the temps and not honor any warranty. Thankfully a scrub was running last night or I would brought all my drives to a temp that I could never trust them again. Not good at all....

Just an FYI, I make the changes that DavfF81 recommended, and my fans stayed on properly through the night...so disabling that module certainly helped...not ideal, but better than having my mini-ITX turn into a toaster!

 

[j_r0dd]

Just an FYI, I make the changes that DavfF81 recommended, and my fans stayed on properly through the night...so disabling that module certainly helped...not ideal, but better than having my mini-ITX turn into a toaster!

Even if this only affects ASRock boards, that we know of, it should be disabled by default.

 

[WordlyBedouin]

Even if this only affects ASRock boards, that we know of, it should be disabled by default.

What's the reason for having it disabled by default? Is it for security reasons or something else? This my first ever server-grade IPMI-enabled mobo and FreeNAS experience, so learning lots along the way...

 

[j_r0dd]

What's the reason for having it disabled by default? Is it for security reasons or something else? This my first ever server-grade IPMI-enabled mobo and FreeNAS experience, so learning lots along the way...

I'm saying the kernel module should not be automatically loaded by FreeNAS by default because of the issue at hand. Destroying my drives and/or data, possibly causing permanent damage to other peripherals from running at higher temps is a major concern. IPMI was enabled by default from ASRock on the board itself, but that is not the issue.

From a security standpoint I would always block internet access to the IPMI address. Even better would be having it on a subnet dedicated for management and blocking that subnet from any other subnet and the outside world.

 

[WordlyBedouin]

I'm saying the kernel module should not be automatically loaded by FreeNAS by default because of the issue at hand. Destroying my drives and/or data, possibly causing permanent damage to other peripherals from running at higher temps is a major concern. IPMI was enabled by default from ASRock on the board itself, but that is not the issue.

From a security standpoint I would always block internet access to the IPMI address. Even better would be having it on a subnet dedicated for management and blocking that subnet from any other subnet and the outside world.

Ah gotcha! Thanks for the clarification. I hadn't thought to isolate my IPMI address, so I appreciate the suggestion.

 

[fngregb]

Has the cause of this IPMI / FreeNAS bug been identified?

I have FreeNAS Mini XL running FreeNAS-9.10-STABLE-201604140622, with
AMI BIOS UEFI version C2750D4I L2.71A, and
IPMI Firmware Revision: 0.23.0 (Nov 19 2014 16:16:53 CST).

System shows similar fan control bug: something in the system causes fan to drop to idle speeds.
Setting BIOS fan controls to a specific level, full on, or changing setpoints has no apparent affect - the fan speeds drop to idle during the night (10-ish PM local time).

Unloading the FreeNAS IPMI module (kldunload ipmi) allows the normal fan control to operate as expected. But this doesn't really fix the problem.

Thanks, gregb

 

[j_r0dd]

Has the cause of this IPMI / FreeNAS bug been identified?

I have FreeNAS Mini XL running FreeNAS-9.10-STABLE-201604140622, with
AMI BIOS UEFI version C2750D4I L2.71A, and
IPMI Firmware Revision: 0.23.0 (Nov 19 2014 16:16:53 CST).

System shows similar fan control bug: something in the system causes fan to drop to idle speeds.
Setting BIOS fan controls to a specific level, full on, or changing setpoints has no apparent affect - the fan speeds drop to idle during the night (10-ish PM local time).

Unloading the FreeNAS IPMI module (kldunload ipmi) allows the normal fan control to operate as expected. But this doesn't really fix the problem.

Thanks, gregb

I consider this a pretty serious bug. Finally created a bug report:
https://bugs.freenas.org/issues/16028