SuperChassis 847E16: MB/CPU/HDs recommendations

Status
Not open for further replies.

Tenek

Explorer
Joined
Apr 14, 2014
Messages
97
This thing is LOUD. I'm wondering if 847E16 chassis generally loud?
It is 75dB right now. It is temporary in my office during setup time and I'm slowly getting crazy from that sound.
It will live in the basement, but right under the bedroom and it will be loud enough to hear it upstairs.

So, is it normal noise and I have to think about sounds insulation? I hope can replace some fans and make it quieter.
I think if I could reduce it to 60dB my wife would be happy...
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
You should not replace the fans.

The fans are loud because a large amount of energy is required to create the needed static air pressure in front of the fan bulkhead. This is what causes air to flow in through the front, and through the very tiny airspaces around all of the drives. There needs to be a substantial differential in order to ensure your drives are cooled correctly. If you replace the fans, it's trivial to make it quieter, but if you want to go that route, just remove the fans entirely and let the drives do a proper job of cooking. You'll get to that nice hot burn-y drive smell more quickly.

The 847 is a particularly bad chassis for this because of the placement of an extra set of drives in the rear. Look at the wattage on the power supply. You're intending to dissipate potentially a lot of watts in a small space. If you got the faster 7200RPM drives, they are even more finicky about temps, because they generate more heat more quickly.

What you *could* do is to do a little creative noise reduction. Presumably you're building this into a rack? Go down to your local home improvement store and pick up some UltraSonic Acoustic Panels. They also sell it in insulation-like batts which may be more convenient for the purpose ... if you have an open 4 post rack, get some plywood sheeting, cut it to fit the sides, line one side with the acoustic material, and then use some spacers to mount it off the sides of your rack. This doesn't do much to reduce noise by itself. But then you can take ... ah hell I'll draw you a pic.

ghetto-acoustic.png

Here ya go. Then you take some more ply, make some L-shaped door-y type things, but the trick is, you need to have some real space for airflow to flow in - absolutely a minimum of a foot, ideally more. You can make L-shaped panels like I've drawn, or you can attach the L-shaped panels with hinges to the sides (this means better accessibility to the rack), or you can even just extend the plywood two feet out and install a hinge at the corner of the "L".

This sort of setup is pretty good for a basement where things are cool and there's a large reservoir of air. You can also install a top to the rack, or otherwise modify the design as needed, as long as you do not substantially impede the airflow.
 

Tenek

Explorer
Joined
Apr 14, 2014
Messages
97
jgreco, point taken, no "plays" with fans.

Thank you for taking time to make a pic! It helps a lot. I think it is a great idea.
Maybe I will build fully custom rack. Sounds like a fun project. If necessary, there could be added quiet fans (or blower) to help move air.
Together with system temperature monitoring it should work out just fine.

PS: I was just surprised how loud is it. Just a warning for a new users like me.
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
I have a 846e16 and mine is in the crawl space and when fans are on full it is still too loud.
 

Tenek

Explorer
Joined
Apr 14, 2014
Messages
97
After additional thoughts, I think I will be building a separate server closet (9x6ft) with AC unit inside like this: Tripp Lite SRCOOL12K Portable Cooling
Please let me know if this should work.

My basement is open and unfinished. I use it as a workshop and it can get dusty. So, having a separate closet for servers should be a good idea. I would like to keep it not more than 9x6 ft.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
After additional thoughts, I think I will be building a separate server closet (9x6ft) with AC unit inside like this: Tripp Lite SRCOOL12K Portable Cooling
Please let me know if this should work.

My basement is open and unfinished. I use it as a workshop and it can get dusty. So, having a separate closet for servers should be a good idea. I would like to keep it not more than 9x6 ft.

The Tripp Lite unit would have to be outside the room. An air conditioner is actually a heater that generates some cold air as a byproduct. The Tripp Lite sucks up room air and dispenses warm air out the backside, while blowing cool air out the pipe. Overall it heats. Having it outside the room just creates other issues, such as air pressure.

You can find dual hose standalone air conditioners. These are typically meant to be piped to a window, and draw in outside air through one pipe, expelling warm air through the other. The problem is that most of these aren't really sized well for the sort of project you're suggesting, plus you have an issue if the thing dies. You suddenly have a nice hot smoke shack.

I'll tell you that the least expensive thing to do, if you've already committed to enclosing a closet-sized space, is just to go get some window air conditioner units. Get two, sized a little larger than what you need. Build an appropriate hole in your server room wall. Run one unit by default and set the temperature on the other one for somewhat higher, it's basically just a backup in case the first one toasts. Run them off separate circuits. Back before the Internet was a big thing, I'll tell you that that's how our machine room was cooled. We had some cheap office space and we converted one room to be the "server room", with a window unit blasted through one wall.

The downside is that you'll need to figure out how to collect and drain off the condensate. Your local hardware store will sell what's called a condensate pump (look in HVAC aisle). Works great. Also, a whole setup with two window units and a condensate pump probably costs about what a single standalone unit costs.

Sizing of HVAC isn't terribly hard. Once you hit equilibrium, you primarily have sensible load (not latent/humidity). BTU = watts * 3.41; 1 KWH consumption ends up being ~= 3410 BTUH ~= 1/4 ton cooling. So if you aren't sucking down watts like there's no tomorrow, it is likely that your usage will fit within the 5000-12000 BTU range that most window units fall into.
 

Tenek

Explorer
Joined
Apr 14, 2014
Messages
97
jgreco, thank you for sharing your experience!
I have to think about it. Window air conditioner will emit a pretty good noise too. But it is a great approach.

Also, do you think if I will use a few of these guys couple at bottom on one wall and couple on top (exhaust) on oposite. It will create a good enough air circulation?
I could use flexible ducting if necessary to direct air more precisely. I also can power 50% of fans from off separate circuit.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
Ah, yes, the whole problem basically revolves around what your basement's actually like. If it's normally rather cool and not too humid, you might well get away with some duct fans. Another option is that you could do a dual-hose portable air conditioner inside the closet (keeps things quiet-ish) and then install some fans of some sort that could kick in based on too-high a temperature (say at 75 degrees), something like a pair of Broan 511's. One pulling air from the top of the closet, one bringing air in on the bottom. I've used these as well and they're pretty hardy, though they aren't really intended for 100% duty cycle.

Lots to think about.
 

tvsjr

Guru
Joined
Aug 29, 2015
Messages
959
I don't know about the 12K standalone unit, but TrippLite also makes a rack-mount unit which I've researched pretty thoroughly. It has two solutions to these problems. First, there is a hot air exhaust kit that would let you duct the hot air into a plenum space or an attic. Second, it can vaporize the condensate into the hot air... however, you need to make sure that wherever you send that exhaust air is ready to deal with warm, moist air. Otherwise, you're going to have mold issues.

The problem I ran into is the small air volume. Even a 9x6 closet is only 432 cubic feet - this AC unit will be turning that air over about once a minute. You stand a serious chance of freezing the unit up (followed by thermal runaway).

Personally, I have a smaller closet (3x4) which just fits a rack. I'm installing a Panasonic WhisperQuiet exhaust fan at the top and rear of the closet to pull air out, and then a louvered door to let air in. So far, things seem to work... however, I'm not up to full load yet, so I don't know if it will keep up. I do have thermal monitoring built into the UPS (both inside, and with an external sensor) so I can carefully monitor temperature. Once we get fully moved into the new house, I'll start doing more testing. At full load, I'll be dissipating about 1.5KW in the closet... via some calculations, I need about 90CFM of air movement, and the vent will supply 110. It's also rated for 100% duty cycle, so I'm hopeful.
 

Tenek

Explorer
Joined
Apr 14, 2014
Messages
97
I ran memset86 for 28 hours (3 iterations, 1 pass) is it good enough?
 

Attachments

  • IMG_20160117_105554751.jpg
    IMG_20160117_105554751.jpg
    291.9 KB · Views: 371

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
28 *days* is good enough. One pass, not good enough. This is your opportunity to find problems before they become Real Problems.

It's partially a function of how trusty you want or need your gear to be. On one hand, there's an argument to be made that with ECC also protecting you, the likelihood that you're going to run into a catastrophic event of some sort is very low.

Here, where we build gear for long service life in a distant data center, and for customers, the idea is that a machine has a five to ten year service life. That's as many as 120 months. If you test it for ~~1% of its lifetime and see no failures, that is probably a good indication that there's no glaringly obvious issues in the silicon. You can test for shorter, but you're running a higher risk of not finding an (admittedly unlikely) issue.
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
'll be dissipating about 1.5KW in the closet... via some calculations, I need about 90CFM of air movement, and the vent will supply 110. It's also rated for 100% duty cycle, so I'm hopeful.
If it is not too much to ask, would you care to elaborate some on these calculations?

cheers

edit:
As per usual, some googling resulted in a good find. I thought this maybe is of interest to others in this thread. This pdf from Sunon:
http://www.sunon.com/uFiles/file/03_products/07-Technology/004.pdf
If I got this right, assuming no real 'flow obstruction';

P = heat dissapated
Tc = temerature rise in Celcius.
Required CFM = (1.75*P)/Tc

Beyond some reasoning this is probably a good starting point, for calculating 'clothsets':
Ie, for 500W worth of server, not more than 5C above ambient; would require a flow of 175CFM.

In a "freenas+utility" server setup at a total of 250w, would translate to about 90CFM. Which seems kind of reasonable?
 
Last edited:

Tenek

Explorer
Joined
Apr 14, 2014
Messages
97
You can test for shorter, but you're running a higher risk of not finding an (admittedly unlikely) issue.
Good deal :).

BTW, before I build a separate ESXI server. Can I use mirrored SSD drives as storage for a few systems running on VistualBox template. Will it make a difference in performance?
 
Last edited:

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
Good deal :).

BTW, before I build a separate ESXI server. Can I use mirrored SSD drives as storage for a few systems running on VistualBox template. Will it make a difference in performance?

I would imagine so, but since I don't actually run jails or VirtualBox, that's merely an educated guess.
 

tvsjr

Guru
Joined
Aug 29, 2015
Messages
959
If it is not too much to ask, would you care to elaborate some on these calculations?

/ Dice

Sure. Two equations you need:
First - 1W = 3.412BTU/hr.
Second - CFM = BTU/hr. / (Tout-Tin * 1.085) (temps in Fahrenheit)

So, a 500 watt server is 1,706BTU/hr
CFM = 1706 / (85-75 * 1.085) = 157.24CFM for a 10-degree rise
CFM = 1706 / (100-75 * 1.085) = 62.89CFM for a 25-degree rise

The important thing to remember is this basically models things assuming perfect airflow. Temperature rise can be unlimited (well, until the structure catches fire, but I digress) - ASSUMING all of the devices are ingesting the intake air and the airflow isn't being "short-circuited", allowing warm air to make it back to the intake. Think of it like so:

Intake air --> Heat generation --> Exhaust fan --> Discharge

Things go to crap when this happens:

Intake air --> Heat generation --> Exhaust fan --> Most of the discharge air
|<- Hot air <-|

The secret is making sure that your airflow isn't short-circuited. In my case, I have a fairly small, 3x4x8 closet. The servers exhaust to the rear, and the exhaust fan is located toward the back of the closet, so air should travel up the back wall and out. Now, this house is still getting remodeled, so I've only tried this out with a limited heat load. I'm hoping, in the next month or two, to have everything up and running so I know for sure.

I'm also not accounting for static pressure here. Most of these exhaust fans have a huge loss of flow when they are presented with any substantial flow restriction. You need to make sure plenty of intake air is available, with minimal restriction.

Finally, this doesn't account for heat loss through any other method. You will lose some heat (not much, most likely) through the walls, etc.

If my plan doesn't work out, the next step will be building a "frame" on the front of the rack (only 25U tall) that seals to the front of the rack and to the louvered closet door (when closed), ensuring that the servers all ingest air through the door, and no hot air gets reingested.

If you're working with a small space, you also need to ensure your exhaust fans will run on alternative power (UPS, generator, whatever) for at least as long as it takes the servers to shut down. If not, things are going to get *hot*, very quickly. If you can seal the intakes to prevent hot air from getting reingested, it won't be as bad... but then you have to deal with heat soak, etc. Suffice it to say, you better keep your exhaust fans running until everything's off.

If there are any civil engineer types in here (I'm but a lowly computer engineer), they're probably cringing at my math (I have taken a few shortcuts and overlooked some more esoteric concerns), but this is at least a decent guideline.
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
Thanks for the elaboration tvsjr :)
Some helpful points in terms of thinking of 'trapping hot air'. From my 12 years of running small home fileserver setups in clothsets, never with any heat evacuation of the clothset the temperatures have always benefited from having the exhaust of the case directed towards the opening of the clothset.
When installing my new freenas box, I will however, ventilate the clothset (hence this interest in air flow of larger spaces).

I re-ran the numbers I found from the Sunon paper, compared to the ones in your example.
Either Sunon guys did a good job of simplifying the formula, or the Farenheit scale messes calculations up if not paying attention.

For the 'Celcius users' I can verify the tvsjr calculations add upp perfectly well with the Sunon paper.
I ran the Farenheit to Celcius conversion manually and then transferred the numbers to the Sunon formula:
C = (5(F-32)/9)
F = (9C/5)+32

85F-75F = 29.44C- 23.88C = 5.56C
100F-75F = 37.77C - 23.88C = 13.89C

CFM = 1.75*500W / 5.56C = 157.37 CFM
CFM = 1.75*500W / 13.89C = 62.89 CFM

Cheers /
 
Last edited:

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Either Sunon guys did a good job of simplifying the formula, or the Farenheit scale messes calculations up if not paying attention.

That's what happens when you don't use real units in formulas... :D
 

tvsjr

Guru
Joined
Aug 29, 2015
Messages
959
ome helpful points in terms of thinking of 'trapping hot air'. From my 12 years of running small home fileserver setups in clothsets, never with any heat evacuation of the clothset the temperatures have always benefited from having the exhaust of the case directed towards the opening of the clothset.

Very true... unfortunately, in my case, not an option. The closet is just wide enough for the servers to slide out from their rails through the doorway, and just deep enough for the rack and servers (they actually end up with their noses in the hallway if fully extended). Thus, my hope is for the back wall to act like a "chimney" of sorts, pulling the hot air up the wall and out the exhaust fan. I'm hoping 110CFM is sufficient, but I do have ceiling space available for a second unit if required, which would get me to 220CFM.

It's hard work needing lots of horsepower, in a small space, with a very high WAF (that's Wife Acceptance Factor - the most critical part of this build!)
 

tvsjr

Guru
Joined
Aug 29, 2015
Messages
959
That's what happens when you don't use real units in formulas... :D

Pfft... their formulas are still using CFM, not m^3/min...

My car gets 40 rods to the hogshead, and that's the way I likes it! :D
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
their formulas are still using CFM, not m^3/min...
In the paper there's a m^3/min version too...
That part is on me :P
I have a feeling what 30 - 70 - 150 CFM is like, but I've no conception of 0.43 m^3/min or 1.44m^3/min translates to :P
 
Status
Not open for further replies.
Top