Help with FAN control

[melloa]

Guys,

I decided to sell all my desktop gear and invest on supermicro servers. Today I'm running my main server on a X9DR3-LN4F+ with two E-2650 and my back-up on the same MoBo with two E-2630. They are great, I must say.

I liked them so much that I bought an anticipated Christmas gift for my son to replace his FreeNAS running on a desktop with a:

2U 8 Bay Supermicro X9SRW-F Server
1x E5-1620 Quad Core
32GB RAM (8x 4GB 8500R)

To run ESXi and FreeNAS as a VM. Works like a charm. The only problem is the noise. That server is loud and I only can deliver if I can quite it down, otherwise my daughter-in-law will divorce my son.

I did find several scripts on the resources, but not sure what/if I need to pass through anything else to make it work as I'm getting ipmi errors from them.

Using the nas_fan_control-master, I got:

Code:

Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory

Using the spinscripts_2018-01-01, I got:

Code:

dev/ipmidev/0: No such file or directory
Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory
Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory
Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory
^32  24.40sysctl: unknown oid 'dev.cpu.0.temperature'

Most likely I didn't configure or install some ipmi drive/software.

Not sure what I'm missing. Any help will be appreciated.

[]'s

 

[droeders]

It looks like you need to load the ipmi kernel module. This provides a driver interface (/dev/ipmi0) so that IPMI can be queried/commanded from your OS.

You can load it from the CLI using:

kldload ipmi


And verify it's loaded with:

kldstat


I would look in the "Tunables" section of the FreeNAS manual for how to load this driver on boot.

 

[melloa]

It looks like you need to load the ipmi kernel module. This provides a driver interface (/dev/ipmi0) so that IPMI can be queried/commanded from your OS.

You can load it from the CLI using:

kldload ipmi

I think is loaded:

Code:

root@mellobc_nas:/mnt/raid/scripts # kldload ipmi
kldload: can't load ipmi: module already loaded or in kernel

Code:

root@mellobc_nas:/mnt/raid/scripts # kldstat
Id Refs Address			Size	 Name
 1   82 0xffffffff80200000 20a0000  kernel
 2	1 0xffffffff82631000 ffe3c	ispfw.ko
 3	1 0xffffffff82731000 7f2a	 freenas_sysctl.ko
 4	1 0xffffffff82811000 84a6	 ipmi.ko
 5	1 0xffffffff8281a000 ef2	  smbus.ko
 6	1 0xffffffff8281b000 333885   vmm.ko
 7	1 0xffffffff82b4f000 3108	 nmdm.ko
 8	1 0xffffffff82b53000 101c6	geom_mirror.ko
 9	1 0xffffffff82b64000 46c3	 geom_stripe.ko
10	1 0xffffffff82b69000 fbad	 geom_raid3.ko
11	1 0xffffffff82b79000 16e56	geom_raid5.ko
12	1 0xffffffff82b90000 59c9	 geom_gate.ko
13	1 0xffffffff82b96000 4d68	 geom_multipath.ko
14	1 0xffffffff82b9b000 837	  dtraceall.ko
15	9 0xffffffff82b9c000 41e31	dtrace.ko
16	1 0xffffffff82bde000 48f4	 dtmalloc.ko
17	1 0xffffffff82be3000 5b4e	 dtnfscl.ko
18	1 0xffffffff82be9000 67f3	 fbt.ko
19	1 0xffffffff82bf0000 58e8a	fasttrap.ko
20	1 0xffffffff82c49000 1741	 sdt.ko
21	1 0xffffffff82c4b000 bf02	 systrace.ko
22	1 0xffffffff82c57000 c082	 systrace_freebsd32.ko
23	1 0xffffffff82c64000 5452	 profile.ko
24	1 0xffffffff82c6a000 1bbc9	hwpmc.ko
25	1 0xffffffff82c86000 d006	 t3_tom.ko
26	2 0xffffffff82c94000 4626	 toecore.ko
27	1 0xffffffff82c99000 15e3a	t4_tom.ko
28	1 0xffffffff82caf000 3b96	 vmmemctl.ko
29	1 0xffffffff82cb3000 3a16	 vmxnet.ko
30	1 0xffffffff82cb7000 463c	 vmblock.ko
31	1 0xffffffff82cbc000 294e	 uhid.ko

I would look in the "Tunables" section of the FreeNAS manual for how to load this driver on boot.

Don't remember have loaded that manually, so not sure it is needed to be added to the Tunables (?)

 

[MrToddsFriends]

There should not be a need to load the ipmi module manually or using a tunable. From a pretty much standard 11.1-U4 install (o.k., upgraded during the last years, originating back to 9.3):

Code:

~ # kldstat | grep ipmi
 4	1 0xffffffff82811000 84a6	 ipmi.ko

Is the boot medium in good condition (System->Boot->Scrub Boot, System->Update->Verify Install)?

Unfortunately I don't have a clue to answer the primordial question, how to quiet down a Supermicro X9SRW-F Server.

 

[droeders]

You're right - the driver is definitely loaded.

And you're certain it's this motherboard: http://www.supermicro.com/products/motherboard/xeon/c600/x9srw-f.cfm

Can you get to the IPMI web interface from the network?

Do either of these commands show anything about IPMI in them?

Code:

dmesg | grep -i ipmi
dmidecode | grep -i ipmi

 

[melloa]

Is the boot medium in good condition (System->Boot->Scrub Boot, System->Update->Verify Install)?

ESXi VM as per OP. And:

Code:

Boot Volume Condition: HEALTHY

Last Scrub Run on: Wed Apr 18 17:05:20 2018	

Size: 31.8 GiB
Used: 851.0 MiB (2%)

Unfortunately I don't have a clue to answer the primordial question, how to quiet down a Supermicro X9SRW-F Server.

I do ... there are two, at least, but I don't want to do either: (1) Power down and keep it that way or (2) Get a X9. (Although it seems odd, that was sold as a X8 and all my X9s are so quiet that I didn't even had to play with this for myself).

And you're certain it's this motherboard: http://www.supermicro.com/products/motherboard/xeon/c600/x9srw-f.cfm

That's what I see in ESXi:

Code:

X9SRW-F 4 CPUs x Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz 31.97 GB

That matches what I was invoiced:

Code:

2U-8B3-X9SRW-F-E5-1620-8x4GB-8500R-2xSATA-9211-8i-2PS-RK

Only if I de-rack and physically look the MoBo or reboot to look in the bios, I don't know how to confirm :(

Can you get to the IPMI web interface from the network?

dmesg | grep -i ipmi

Nada ...

Code:

root@mellobc_nas:/mnt/raid/scripts # dmesg | grep -i ipmi
root@mellobc_nas:/mnt/raid/scripts #

dmidecode | grep -i ipmi

Nada ...

Code:

root@mellobc_nas:/mnt/raid/scripts # dmidecode | grep -i ipmi
root@mellobc_nas:/mnt/raid/scripts #

 

[droeders]

I'm wondering if this is VM related? Can you install bare metal and run the same dmesg/dmidecode commands?

I have two (newer but different) Supermicro MBs running FreeBSD and get the following:

Code:

dmesg | grep -i ipmi
ipmi0: <IPMI System Interface> port 0xca2,0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
ipmi0: IPMI device rev. 1, firmware rev. 3.58, version 2.0
ipmi0: Number of channels 2
ipmi0: Attached watchdog

dmidecode | grep -i ipmi
IPMI Device Information

 

[melloa]

I'm wondering if this is VM related?

I know couple guys here running ESXi/FreeNAS VMs, but possible.

Can you install bare metal and run the same dmesg/dmidecode commands?

Can try a live iso via IPMI ... Let me ctrl-c my rsync :(

 

[melloa]

Can you install bare metal and run the same dmesg/dmidecode commands?

Let me try something first ... replace the fans ;)

Put some Cool Masters there and they are quiet as a mouse. Gonna run overnight at max load to see if will blow from heat :D

Also, it is a X9SRW-F.

 

[melloa]

For future reference:

Changed the fans to three Cool Masters I had on another SM chassis and temps at start-up were:

12 hours running:

Still gonna look on those scripts on my two servers. They are on FreeBSD VMs though ... but got the same issues, so not FreeNAS specific.

 

[droeders]

Out of curiosity - did you ever try accessing the IPMI from a bare metal FreeBSD or FreeNAS install?

Maybe a Linux ISO instead?

 

[melloa]

Out of curiosity - did you ever try accessing the IPMI from a bare metal FreeBSD or FreeNAS install?

Maybe a Linux ISO instead?

Not yet, but I'm planing to, as my FreeBSD VM on another server (X9DRi-LN4F+/2x intel Xeon E5-2650 V2) is also giving me the same problem.

just need to find five more hours on my day :(

 

[AVSION]

Hi Guys, i have exactly the same problem ipmitool is not working, get the same error as describe above and did all the steps as well but nothing!! i'm running ESXi 6.7 with FN VM 11.2 BETA3. i did find something, when running

Code:

root@freenas[/]# ipmitool
Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory

when going to the directory /dev/ and looking for ipmi device its not their missing, is that mean its not installed or their is no driver?

and can see it in the following locations

Code:

root@freenas[/]# find / -name "ipmitool"
/usr/local/share/ipmitool
/usr/local/bin/ipmitool
/mnt/Tank/.warden-template-standard-11.0-x64-20180612231357/usr/ports/sysutils/ipmitool
/mnt/Tank/.warden-template-standard-11.0-x64-20180313141143/usr/ports/sysutils/ipmitool
/mnt/Tank/.warden-template-standard-11.0-x64/usr/ports/sysutils/ipmitool

as well no problem accessing the IPMI web UI all normal can make any changes just ipmitool on freenas not available

please help

EDIT: Just installed FN 11.2 BETA3 bare metal to check the ipmitool and its working fine can accept ipmitool commands with no issues.

is there anyway to test where the issues is and how to fix it/workaround?

Thank you