BUILD Proposed FreeNAS build

[Ericloewe]

Your quote was my initial response to IPMI. After using it I wouldn't give it up for the world. It's just so incredibly useful.

Most useful thing I've seen in quite a while. I'm just saying that the whole implementation (and, to be clear, a lot of it is preexisting on modern x64 machines, especially the opaquely-named Management Engine) has a number of shady aspects.

While not related to FreeNAS, I've built some VMware hosts before using Supermicro mini-ITX boards. The same issue cropped up... the fans cycling from slow to full speed. There is apparently a hard-coded minimum RPM expected for each fan input... dropping below this speed causes the MB to think the fan has died, so it spins every fan up to full speed - including the one that actually isn't dying. The MB then sees that all are running wide-open, so the failed state clears and the fans slow back down... below that minimum threshold. The process repeats ad nauseum.

Your options are either a fan fast enough to make the board happy, or an external fan controller.

Wrong. As explained in the guides I wrote (you can find them from the X10 FAQ linked in my sig - they apply generally to IPMI, though), the thresholds are indeed user-definable.

If you need the cooling power of Noctua Industrial fans (awesome as they are, i'm a huge fan of Noctua!) you're doing something VERY wrong.

They're actually relatively quiet during normal usage, so they make quite a viable office environment fan.

 

[Something]

Most useful thing I've seen in quite a while. I'm just saying that the whole implementation (and, to be clear, a lot of it is preexisting on modern x64 machines, especially the opaquely-named Management Engine) has a number of shady aspects.

Very much agreed.

They're actually relatively quiet during normal usage, so they make quite a viable office environment fan.

Issue is they're very costly and there are cheaper, quieter options. I'd only recommend the industrial fans if you actually NEED that level of performance.

 

[MindBender]

Could you post what Supermicro said, exactly? Not doubting you, but I'd like to see what they said, exactly, to try to figure out where to go from there.

Shall I PM you the email thread? Perhaps it's not the best thing to post it here...

 

[MindBender]

There is apparently a hard-coded minimum RPM expected for each fan input... dropping below this speed causes the MB to think the fan has died, so it spins every fan up to full speed - including the one that actually isn't dying.

Fortunately with SuperMicro the limits are not hard-coded. Eric's FAQ describes how you can set them to - for your fans - more realistic values.

Edit: D*mn; Eric beat me to it.

 

[MindBender]

And I have to agree with you guys; I was impressed with IPMI. I do doubt if it's safe to expose it to the outside world though.

 

[Something]

I definitely wouldn't expose my IPMI without some form of secure VPN.

Would be one HUGE security risk.

 

[Ericloewe]

Shall I PM you the email thread? Perhaps it's not the best thing to post it here...

Sure

 

[MindBender]

They're actually relatively quiet during normal usage, so they make quite a viable office environment fan.

Are the fans controlled proportionally, a little faster when it's a little warmer, or are they running at either low speed or full speed?

 

[Ericloewe]

Are the fans controlled proportionally, a little faster when it's a little warmer, or are they running at either low speed or full speed?

The control law is more sophisticated than "high or low", but I can't tell you for sure what exactly the control law is.

 

[MindBender]

The control law is more sophisticated than "high or low", but I can't tell you for sure what exactly the control law is.

I am asking because I have both Noctua NF-F12 industrialPPC-2000PWM and Noctua NF-F12 industrialPPC-3000PWM on trial. I have bought one of each and I'm trying to decide which one to go with (and order a second one of). As long as they're running at low speed, none of both are audible. But if they run at full speed, the 3000 is a lot louder than the 2000. I can live with that level of sound for emergency cooling, and it's reassuring to have that amount of cooling power just it case, but if I have to listen to that every time I stress my NAS a bit, it will get annoying real quickly.

Do your 3000s go full power often?

 

[Something]

If you need Noctua Industrial fans running at 3000RPM something has gone catastrophically wrong and more cooling power is definitely NOT the solution.

What CPU and cooler?

 

[MindBender]

If you need Noctua Industrial fans running at 3000RPM something has gone catastrophically wrong and more cooling power is definitely NOT the solution.

What CPU and cooler?

Xeon D1540, soldered on and with a stock cooler. I will have 8 disks in the cabinet though...

 

[Something]

Stock cooler with Noctua industrials.

Please stop hurting me so :'(

No but seriously, if you run into a catastrophic cooling scenario, Noctua industrials are not going to be the problem solver.

You should NEVER be in a scenario where you need those case fans at maximum or the 3000RPMs above 2000. Even in IntelBurnTest.

 

[Ericloewe]

Rackmounts routinely require 80mm 10k-ish RPM fans. Ear protection is highly recommended.

 

[Something]

We are talking about a small home FreeNAS setup though :p

 

[CheckYourSix]

Do NOT hook those Noctua fans to a standard fan controller! If you don't understand how PWM motors work, see here: http://www.noctua.at/main.php?show=faqs&step=2&products_id=84&lng=en#7. They get plugged into your motherboard, which will control them. Just change a setting on your motherboard to have them spin a bit faster so it doesn't register them as being stalled. I have the 3,000 RPM version of both the 120mm and 140mm fans in my desktop. They are excellent. I just have the settings tweaked for them to idle at 50% and they are barely audible from 3 feet away. 40% will make them inaudible completely. 68 degrees ambient, with a stack of 8x7,200 RPM HGST 2TB drives on top of each other in the drive cages, and my warmest drive among them is 28C. The coolest drive is 24C. I also have them mounted with rubber mounts, so when they get turned up to higher RPMs there is no vibration noise, just noise from the air movement.

 

[Something]

Do NOT hook those Noctua fans to a standard fan controller! If you don't understand how PWM motors work, see here: http://www.noctua.at/main.php?show=faqs&step=2&products_id=84&lng=en#7. They get plugged into your motherboard, which will control them. Just change a setting on your motherboard to have them spin a bit faster so it doesn't register them as being stalled. I have the 3,000 RPM version of both the 120mm and 140mm fans in my desktop. They are excellent. I just have the settings tweaked for them to idle at 50% and they are barely audible from 3 feet away. 40% will make them inaudible completely. 68 degrees ambient, with a stack of 8x7,200 RPM HGST 2TB drives on top of each other in the drive cages, and my warmest drive among them is 28C. The coolest drive is 24C. I also have them mounted with rubber mounts, so when they get turned up to higher RPMs there is no vibration noise, just noise from the air movement.

I may be misreading but that link doesn't say don't use a PWM fan controller.

But yes definitely plug PWMs into the mobo.

I'd say your drives are almost too cold at 24c. As a physical medium isn't it best they run at 27-33cish?

 

[CheckYourSix]

I may be misreading but that link doesn't say don't use a PWM fan controller.

I said don't use a standard fan controller. They work by varying the voltage to control the speed. The motors on these fans can't handle that. Only a couple true PWM fan controllers exist and they are very expensive. There are PWM splitters, like I have, which have a primary header that connects to the motherboard and a molex power connector. They have 8 PWM ports on them, allowing all of the fans to turn at the same speed and be controlled by the motherboard. See here: http://www.amazon.com/dp/B00IF6R4C8/?tag=ozlp-20

I'd say your drives are almost too cold at 24c. As a physical medium isn't it best they run at 27-33cish?

I'm not sure. It's not that I'm trying to run them that cold, they just are that cold. They warm up just a little bit if I pull the fans back to 25%. Ambient temperature matters a lot. If the ambient was 72 F, they would warm up to about 30 C, but I like my house cold at 68 F.

 

[Something]

I said don't use a standard fan controller. They work by varying the voltage to control the speed. The motors on these fans can't handle that. Only a couple true PWM fan controllers exist and they are very expensive. There are PWM splitters, like I have, which have a primary header that connects to the motherboard and a molex power connector. They have 8 PWM ports on them, allowing all of the fans to turn at the same speed and be controlled by the motherboard. See here: http://www.amazon.com/dp/B00IF6R4C8/?tag=ozlp-20

Oh hey I just got one of those!

I'm not sure. It's not that I'm trying to run them that cold, they just are that cold. They warm up just a little bit if I pull the fans back to 25%. Ambient temperature matters a lot. If the ambient was 72 F, they would warm up to about 30 C, but I like my house cold at 68 F.

Mmmm

Well as far as I know hard drives are one of the few pieces of tech that DO NOT like to be colder or warmer and you should be within a certain range.

 

[Bidule0hm]

From the google paper the range is 30 to 40 °C, if you go lower or higher the failure rate rise. It's far worse to go higher than lower.

Another important thing (not in the paper but it's an educated guess) is that precise mechanical systems don't like wide and/or fast temperature changes, so the more stable the temp, the better ;)