Supermicro MB Questions IPMI - Power Consumption, Rules

[tmacka88]

Hi,

I recently went all out and upgraded my FreeNAS as below in my signature from the advice of lots on these forums.

Now that I have my wonderful new set up I have a few questions about it. Unfortunately there aren't any forums specifically for supermicro other than there support portal so figure some of you would now something about these systems.

Basically, since using the setup I have been using IPMI and now would like to do a few things but not certain how.

1. Power Consumption - This is a feature I have found within the IPMI settings "Server Health > Power Consumption" but nothing is being displayed (see img below). How can I get my power consumption to show up here? BTW I am using a Supermicro 933T server case that has 3 x power supplies.

2. Is there a way to change my temperature thresholds for e.g. CPU, System etc. Then I want to be able to set up rules. E.g threshold 1 email myself of the problem and threshold 2 shutdown the system.

Thanks in advanced. Any help would be great.

 

[cyberjock]

If you read the IPMI manual you'll find that the power supply stuff is available if you have a special compatible PSU(read: expensive).

As for temp threshold, you'll probably have to ask Supermicro if the hardware can do that.

 

[tmacka88]

ahh always more to spend on things haha.

thanks.

 

[Tomas Liumparas]

Regarding the temp, in order to control the fans, you need to use 4 pin fans.

 

[tmacka88]

Regarding the temp, in order to control the fans, you need to use 4 pin fans.

Hey,

Yeah I have converted all the original fans to PWM 4 pin fans. They do speed up and slow down. But was wondering more about notifications when a certain temp has been reached.


cheers

 

[Mguilicutty]

Hey,

I've got almost the same build as you.. I was wondering what you did to convert the fans in your 933. Did you retain the cages and wire around them or are there 4 pin cages available? I've given some thought to replacing the power supplies as I don't really need the massive redundancy and they seem to be crazy inefficient. My system is pulling quite a lot more power than I had calculated ahead of time and other than the fans screaming away at full blast all the time, I'm thinking that is why.

I think cyberjock is right regarding the IPMI stuff, I believe all of that is set in firmware. If you want specific notifications that should be achievable via software in the O/S. You might have to write a script to handle it. For instance...

sysctl -a |egrep -E "cpu\.[0-9]+\.temp"

will return all cpu temps. You could do a cron to check periodically and take what ever actions you desire.

 

[jgreco]

You can probably chew up 100 watts on those Supermicro fans alone if they're screaming away. Fix your chassis or configuration to make them blow as directed by environmental conditions.

 

[Mguilicutty]

I haven't checked the spec's on the fans but there are 6 of them and they are pretty powerful. The cpu heatsink is fanless and I have shrouding to direct the airflow right through it. I'm in the exact same boat as tmacka88 with the 3 pin fans on a 4 pin mobo so I was wondering what he did. My plan was just to wire new 4 pin fans into the cages and skip the hot swap connections all together but if 4 pin cages are available I'd rather go in that direction.

 

[cyberjock]

You can probably chew up 100 watts on those Supermicro fans alone if they're screaming away. Fix your chassis or configuration to make them blow as directed by environmental conditions.

They absolutely can. My 4-fan setup can see my computer's wattage DOUBLE just by running the fans at 100%! That's over 150watts in fans alone!

 

[tmacka88]

Hey,

I've got almost the same build as you.. I was wondering what you did to convert the fans in your 933. Did you retain the cages and wire around them or are there 4 pin cages available? I've given some thought to replacing the power supplies as I don't really need the massive redundancy and they seem to be crazy inefficient.

Hey,

Ok so the current fans in the 933T case are:

4 x 92mm x 92 x 38 = 3.1m3/min or 110 CFM, 12v 1.1A , static press 150 Pa, 50 dBA
2 x 80mm x 80 x 32 = 1.52 m3/min or 54 CFM, static press 91 Pa, 45 dBA, 12v 0.55A

I then replace them with:
4 X Nidec Ultraflow T92E12BMA7-PWM - $21.99 each
http://www.newegg.com/Product/Product.aspx?Item=N82E16835706042

2 X R4-BM8S-30PK-R0 Cooler Master Blade Master 80 - $9.99 each
http://www.newegg.com/Product/Product.aspx?Item=N82E16835103071

So the original fans had:
92mm - 110 CFM, 1.1A/13.2w, 50 dBA
80mm - 54 CFM, 0.55A/6.6w, 45 dBA

and the new ones had:
92mm - 83 CFM, 0.39A/4.7w, 43.5 dBA
80mm - 40.8 CFM, 0.26A/3.1w, 28 dBA

So power wise it's gone from 66 watts down to a max of 25 watts which is has reduced it by more than 62%. Because the fans aren't spin anywhere near full speed it would have reduced a fair bit more I would say. Guestimate around 10-15 watts maybe even less...

Now noise. Well I can finally say that I can it soooooo much better. You can still hear them but they are not bad at all (not full power, this is when they are set for "optimal" in the supermicro IPMI fan settings or even standard is the same). Can tell you the actual dBA but lets just say you can think in the room. The loudest thing now it the 3 x PSU fans, if these could be quieter it would be great. Soon it will all go into a RU cabinet so I won't hear anything from those fans (would with the old ones).

Performance. Well it's too early to say as I put them in a couple of days ago and now I am away for 4 months so can't monitor them really. But I check the IPMI temps after and my CPU temp was 49 degrees Celsius (120 degrees fahrenheit for my US friends still living in the OLD imperial system lol) when my room temp was about 34 degrees Celsius (93 degrees fahrenheit) and the fans were not even spinning much.

Also I replace the standard CPU fan with a fan less heatsink (can find model of heatsink if interested for X10SLM-F MB) and the hotswap cables with supermicro 4 pin version.

6 Supermicro CBL-0088L 4 pin PWM cables - $4.99 each
http://www.superbiiz.com/detail.php?name=CA-CBL088L

However, it wasn't until i received all my new fans I realised that the female headers were different to the supermicro fan heaters so therefore they would not fit in to the fan cage. I am working on finding some headers that are the same so I can cut the old ones off and replace with the required ones so I can keep them as hot swappable. So in the mean time I just did a quick botch job to get all the new fans working until I can get around to fixing it. I did this by swapping them all out in to the cages and then just threaded the wires through the bottom and connected it to the hot swappable male header cable which was then connected to the MB (the new fan cables where not long enough to reach to the MB by them selves). Also I got a PWM Y slitter cable so that I can use all 6 fans on my MB (mine only has 5 headers on the MB). Slightly off topic, I noticed the PWM Y splitter cable had 1x4 pin and 1x3 pin. I was concerned that the 3 pin would not be adjusted as needed, but from a quick test it LOOKS like it still speeds up and slows down with the others (correct me if this is wrong).

O and 2 more thing. The new fan female headers are not the same as the supermicro ones as statedso to get the to connect in to the hotswap cable header I got a Stanley knife and cut the safety edge off the restricting one so it could then connect. Don't worry if you just connect this cable straight to the MB you won't have this issue. With the new 2 x 80 mm fans, it leaves a 7mm gap from the end of the case as they are not as deep as the original ones so I just used some masking tape over the top to keep the flow going rearward (bottom was to hard to get to). If anyone can find some 80x80x7mm spacers please let me know.

If anyone has any suggestions on how to get the female headers required for the hotswap cage please let me know.

Hope this helps you all.

 

[tmacka88]

slightly off topic but with the scope of the subject problem.

I just updated my IPMI to the lastest firmware (unticked the save setting thing as per the manual) and then uploaded my saved setting to it it (as it defaults after the upgrade) and now I can not log in to it at all. not even ADMIN/ADMIN work as I got rid of that in my setting but my pw/user doesnt work at all. Any ideas?

thanks

 

[jgreco]

Run the IPMICFG utility from the local keyboard/VGA. There might be a BIOS option to reset it as well.

 

[tmacka88]

Run the IPMICFG utility from the local keyboard/VGA. There might be a BIOS option to reset it as well.

sounding like a noob. how do i run IPMICFG?

it is installed right? if i turn on the system and unplug my freenas boot hdd do i just run the commands there?

thanks

 

[AiRLAC]

I guess that is a CLI command to be ran either from a DOS prompt or a Windows command line. Get Rufus and create a DOS bootdisk on a USB flashdrive+copy IPMICFG to it.
ftp://ftp.supermicro.com/CDR-X9_1.2...CFG/Supermicro_Utility_User_Guide_IPMICFG.pdf

 

[Mguilicutty]

Thanks for the response tmack, a lot of good information there. As to the Celsius debate, actually for some things it does seem C is the norm, like CPU temps for example. Room or outdoor temp I'd still recite in Fahrenheit. The 3pin header on your Y cable omits the RPM return signal from the fan, as the mobo wouldn't be able to deal with two rpm signals on the same pin, or maybe it would read it as the sum of the two? In any case, I believe that signal is only there to give information to humans. If the fan connected to the 3pin header dies you will not receive notification. The PWM signal that controls the fan speed is probably based on CPU temp or case temp in some systems.

The loudest thing now is the 3 x PSU fans, if these could be quieter it would be great.

Surprised to hear that, mine don't make hardly any noise at all and they are pushing some air. Maybe it's just all the other noise in the room.

If anyone has any suggestions on how to get the female headers required for the hotswap cage please let me know.

This... I've been putting off additional work I need to do to this server for couple of months now, as life has been crazy in other areas and a broken heel has discouraged me from travelling to the basement too often. Thank God for IPMI. I've got two more arrays worth of disks sitting here waiting to go in, in addition to the fan issue and a few others to deal with. Perhaps this weekend I'll find some time to head down there and make some progress. If I do I'll post some pics and maybe somebody else can see what we're talking about and has the magic solution. I fear there is not a good one.

 

[tmacka88]

The 3pin header on your Y cable omits the RPM return signal from the fan, as the mobo wouldn't be able to deal with two rpm signals on the same pin

ah ok that makes sense then.

Surprised to hear that, mine don't make hardly any noise at all and they are pushing some air. Maybe it's just all the other noise in the room.

Yeah you can not hear them with the other fans on, but after the upgrade they are the loudest of them all. They aren't too bad but still wouldn't mind them quieter if possible.

Yeah i am away from home on a course for another 4 months but when I get back I will be trying to fix the fans to make them hot swappable. Not really needed but why not hey.