Proper use of static pressure fans?

[EvanVanVan]

Hi,

This is not really a truenas question but maybe somebody here can help me all the same.

I built a new TrueNAS Scale server with a Fractal Define 7 XL case. It has 8 hard drives mounted in the front of it. When building it and calculating positive pressure in the case, I learned (for the first time in 20 years of building PCs) about the existence static pressure fans. It sounded like that's what I was looking for, fans that exceled at cooling with a lot of obstructions (the dust filter on one side and 3.5" drives on the other).

But with three Artic P14 fans pulling air in the front of the case, TrueNAS keeps giving me high temp HDD alarms at 40C. In addition to the 3 fans in front, I have two fans exhausting, one on top, one out the back. There are all the same model (static pressure optimized) which may be a mistake?

Half the time the server already sounds like a jet engine lol, but it's just not cooling the hard drives very well.

I also don't really understand what limits I should set for a fan curve. Any advice?

Any other suggestions? Should I just replace them with fans that prioritize air flow over static pressure?

Thanks

 

[dak180]

I built a new TrueNAS sever with a Fractal Define 7 XL case. It has 8 hard drives mounted in the front of it.

I assume that is the TrueNAS Scale build in your sig.

Half the time the server already sounds like a jet engine lol, but it's just not cooling the hard drives very well.

You may want to check out my build and my fan control script in my sig (not sure how well that will work on scale, you will at least need to add bc which it uses for the math), I would also recommend, if you have not already, taking the door for the front panel off. With my script and the door off my drives maintain 30°C without objectionable noise up to 70°F ambient and with notable noise up to 85°F ambient (I have no air conditioning).

If you go this route please contribute your board's IPMI commands or validate that the defaults work for your board (you may need to contact ASRock Rack for the relevant commands).

I have two fans exhausting, one on top, one out the back. There are all the same model (static pressure optimized) which may be a mistake?

I would recommend more airflow optimized or balanced fans for exhaust. Let me know if you want a more detailed layout of my fan setup.

 

[Redcoat]

Any other suggestions?

Take the door off per @dak180 suggestion for sure.

Have you blocked off any open drive bays (and other openings) in order to force the air through the gaps above and below the drives?

 

[chuck32]

What is your ambient temperature?

Definetily look into the suggestions already made to you, but I think there is some issue at hand when you say the fans already sound like a jet engine. Did you mount them in the wrong direction? Maybe you can show a picture of your setup.

In addition to the 3 fans in front, I have two fans exhausting, one on top, one out the back. There are all the same model (static pressure optimized) which may be a mistake?

I don't use static pressure for exhaust fans, but I doubt it will make that much of a difference. What do your other thermals say?

My Define R5

I use two thermaltake toughfan 14, they have 1.5 x the static pressure of the artic P14, but I only run two of them and you run three. Other than the CPU cooler I only have one exhaust fan at the back.

With an ambient temperature of 21.6 °C I'm sitting at 29 °C to 33 °C temperature for the HDDs (in the dashboard the max temp is shown as 40 °C, not sure if this is an alltime high). I run the fans at 500 rpm, which is not audible at all (the space where the server is located is also a bedroom at the moment).

 

[EvanVanVan]

I assume that is the TrueNAS Scale build in your sig.


You may want to check out my build and my fan control script in my sig (not sure how well that will work on scale, you will at least need to add bc which it uses for the math), I would also recommend, if you have not already, taking the door for the front panel off. With my script and the door off my drives maintain 30°C without objectionable noise up to 70°F ambient and with notable noise up to 85°F ambient (I have no air conditioning).

If you go this route please contribute your board's IPMI commands or validate that the defaults work for your board (you may need to contact ASRock Rack for the relevant commands).


I would recommend more airflow optimized or balanced fans for exhaust. Let me know if you want a more detailed layout of my fan setup.

Thank you, I'll take a look at your script. Yes, it is on the Scale build. I'll probably swap and/or reconfigure the fans.

Take the door off per @dak180 suggestion for sure.

Have you blocked off any open drive bays (and other openings) in order to force the air through the gaps above and below the drives?

Thanks, I'll take the door off. To the @chuck32's point below, the third fan in the front isn't in front of any drives right now (future expansion). I just added it to make positive pressure simpler. Maybe it's doing more harm than good.

What is your ambient temperature?

Definetily look into the suggestions already made to you, but I think there is some issue at hand when you say the fans already sound like a jet engine. Did you mount them in the wrong direction? Maybe you can show a picture of your setup.


I don't use static pressure for exhaust fans, but I doubt it will make that much of a difference. What do your other thermals say?

My Define R5
View attachment 75621

I use two thermaltake toughfan 14, they have 1.5 x the static pressure of the artic P14, but I only run two of them and you run three. Other than the CPU cooler I only have one exhaust fan at the back.

With an ambient temperature of 21.6 °C I'm sitting at 29 °C to 33 °C temperature for the HDDs (in the dashboard the max temp is shown as 40 °C, not sure if this is an alltime high). I run the fans at 500 rpm, which is not audible at all (the space where the server is located is also a bedroom at the moment).

Thank you, as I mentioned above, the third fan isn't in front of any drives so maybe the lack of obstructions is actually hurting the performance of the other ones. I like the idea of blocking off the open spaces for now to force the air where its needed. They are in the right direction lol.

I actually haven't set any fan limits apparently. I originally built this machine with a Supermicro board before I learning it wasn't compatible with the iGPU so I swapped it for a the ASRock Rack board. On the Supermicro board, the fans were ramping up and down constantly so that's when I had modified the limits.

For the this board it's just the default:

Ambient tempertature is 70-74F.

Code:

root@truenas[~]# ipmitool sdr list | grep Temp
MB Temp          | 29 degrees C      | ok
Card Side Temp   | 31 degrees C      | ok
CPU Temp         | 32 degrees C      | ok
PCH Temp         | 46 degrees C      | ok

Code:

root@truenas[~]# sudo smartctl -a /dev/sd* | grep "Temperature"
194 Temperature_Celsius     0x0002   052   052   000    Old_age   Always       -       32 (Min/Max 24/53)
194 Temperature_Celsius     0x0002   051   051   000    Old_age   Always       -       32 (Min/Max 24/54)
194 Temperature_Celsius     0x0002   052   052   000    Old_age   Always       -       32 (Min/Max 24/51)
194 Temperature_Celsius     0x0022   074   062   000    Old_age   Always       -       26 (Min/Max 0/38) #boot-pool (SSD)
194 Temperature_Celsius     0x0002   054   054   000    Old_age   Always       -       31 (Min/Max 24/51)
194 Temperature_Celsius     0x0002   051   051   000    Old_age   Always       -       32 (Min/Max 24/53)
194 Temperature_Celsius     0x0002   051   051   000    Old_age   Always       -       32 (Min/Max 24/53)
194 Temperature_Celsius     0x0002   052   052   000    Old_age   Always       -       32 (Min/Max 24/54)
194 Temperature_Celsius     0x0002   052   052   000    Old_age   Always       -       32 (Min/Max 24/53)
194 Temperature_Celsius     0x0002   054   054   000    Old_age   Always       -       31 (Min/Max 24/47)

Right now temps are ok. I know those 50+ worst values are no bueno...that was me trying to fix the Supermicro ramp up/ramp down fan bug, and mistakenly setting a bad fan profile in the IPMI...fortuntely it wasn't an issue for very long...still beating myself up over it though :/

Here are some pictures:

I could have started the at bottom, but this was the first consecutive 8 bays and this case can hold another 8 drives in the future when my hoarding of 4K remuxes fills them. lol

 

[NugentS]

I use an R5 for a 7*HDD Pool
I initially used Fractal GP-14 Fans - which are very quiet. 3 at the front and one at the rear. I had to turn the machine off due to HDD temps.

I substituted 4*Corsair ML140 fans on a fan controller so I could adjust the temperature and noise. This works very well. I have the fans turned down to a satisfactory level that still provides more than enough airflow.

 

[chuck32]

On the Supermicro board, the fans were ramping up and down constantly so that's when I had modified the limits.

Yes, this happens if the fan speed is below the lower limit, the mainboard will then ramp up to 100 %. This happens with low noise fans / the default limits are aimed at high rpm fans. I had the same with my CPU cooler, changing the lower limit is enough, you do not necessarily ramp up the speed. have a look at this resource.

Code:

root@truenas[~]# sudo smartctl -a /dev/sd* | grep "Temperature"
194 Temperature_Celsius     0x0002   052   052   000    Old_age   Always       -       32 (Min/Max 24/53)
194 Temperature_Celsius     0x0002   051   051   000    Old_age   Always       -       32 (Min/Max 24/54)
194 Temperature_Celsius     0x0002   052   052   000    Old_age   Always       -       32 (Min/Max 24/51)
194 Temperature_Celsius     0x0022   074   062   000    Old_age   Always       -       26 (Min/Max 0/38) #boot-pool (SSD)
194 Temperature_Celsius     0x0002   054   054   000    Old_age   Always       -       31 (Min/Max 24/51)
194 Temperature_Celsius     0x0002   051   051   000    Old_age   Always       -       32 (Min/Max 24/53)
194 Temperature_Celsius     0x0002   051   051   000    Old_age   Always       -       32 (Min/Max 24/53)
194 Temperature_Celsius     0x0002   052   052   000    Old_age   Always       -       32 (Min/Max 24/54)
194 Temperature_Celsius     0x0002   052   052   000    Old_age   Always       -       32 (Min/Max 24/53)
194 Temperature_Celsius     0x0002   054   054   000    Old_age   Always       -       31 (Min/Max 24/47)

Looks good, what's the problem then? Are the current settings too loud? I do not see any action necessary based on these temps. Or are you still getting > 40 °C alarms?

fortuntely it wasn't an issue for very long...still beating myself up over it though :/

I doubt the HDDs will die if they've seen a spike to 50 °C one time in their life..

Maybe it's doing more harm than good.

I do not think it has any negative influence.

They are in the right direction lol.

One can never know ;) Mistakes happen.

When I inspect your image further I'd say the HDDs seem to be closer together in your case than in the R5. Try the fan script and if all fails maybe you need to upgrade your fans.

 

[EvanVanVan]

Looks good, what's the problem then? Are the current settings too loud? I do not see any action necessary based on these temps. Or are you still getting > 40 °C alarms?

This morning (when it snowed for the first time this winter last night lol), the temps look good... But I've only really had this server up and running for 2 to 3 weeks now and have gotten 40°C high temp alarms five or six times (mid day)..

I don't think they go much past 40C but it would nice to keep them down in the low 30s.

Thanks for all the advice!

 

[Redcoat]

I do not think it has any negative influence.

Right - it has the +ve influence that it is providing air to raise the overall static pressure in front of all the drives. Blocking the open spaces will improve the situation further. I have "dummy drives" made from stacked corrugated cardboard in my open slots.

 

[dak180]

Thank you, I'll take a look at your script. Yes, it is on the Scale build. I'll probably swap and/or reconfigure the fans.

Just to give you an idea of how things work with my script (the server is mostly idle right now):

Code:

CPU_FAN1    auto    500 rpm
FRNT_FAN1    29%    600 rpm
FRNT_FAN2    20%    700 rpm
FRNT_FAN3    66%    1600 rpm
REAR_FAN1    29%    400 rpm

To give some context: FRNT_FAN1 is the 3 Noctua -NF-A14 2000 ip67 140mm fans connected through the case's fan hub in the front of the case, FRNT_FAN2 is the Noctua NF-A4x10 5.2 CFM 40mm fan that I used to replace (with a reversed direction) the builtin fan of the ICY DOCK ToughArmor MB996SP-6SB that holds my 2.5" sata SSDs, FRNT_FAN3 is the Noctua - NF-A9x14 29.7 CFM 92mm fans for the HBA and expander, and REAR_FAN1 is the CRYORIG - XF140 76.0 CFM 140mm fan as a rear exhaust.

Code:

CPU1 Temp        | 29 degrees C      | ok
Card Side Temp   | 25 degrees C      | ok
MB Temp          | 29 degrees C      | ok
TR1 Temp         | 34 degrees C      | ok

Code:

Current SSD setpoint temp:    35°C
Current HD setpoint temp:    30°C

mpr0 Temp:    45°C
ses0 Temp:    37°C
 
ada2 Temp:    32°C
da4 Temp:    31°C
da5 Temp:    29°C
da6 Temp:    30°C
da7 Temp:    31°C
da8 Temp:    30°C
da9 Temp:    29°C
da10 Temp:    27°C
ada0 Temp:    32°C
ada1 Temp:    31°C
da0 Temp:    33°C
da1 Temp:    33°C
da2 Temp:    28°C
da3 Temp:    28°C

Average HD Temp: 29.875°C
Average SSD Temp: 30.833°C
HBA Temp Average: 41.000°C

ada2 and da4 - da10 are the 3.5" drives and the rest are the 2.5" SSDs.

 

[dak180]

Just to give you an idea of how things work with my script (the server is mostly idle right now)

And for some numbers with a high IO load:

Code:

CPU1 Temp        | 49 degrees C      | ok
Card Side Temp   | 25 degrees C      | ok
MB Temp          | 29 degrees C      | ok
TR1 Temp         | 34 degrees C      | ok

Code:

CPU_FAN1    auto    500 rpm
FRNT_FAN1    51%    1000 rpm
FRNT_FAN2    100%    4500 rpm
FRNT_FAN3    84%    1800 rpm
REAR_FAN1    51%    800 rpm

Code:

Current SSD setpoint temp:    35°C
Current HD setpoint temp:    30°C

mpr0 Temp:    46°C
ses0 Temp:    36°C
 
ada2 Temp:    32°C
da4 Temp:    30°C
da5 Temp:    30°C
da6 Temp:    29°C
da7 Temp:    32°C
da8 Temp:    30°C
da9 Temp:    30°C
da10 Temp:    28°C
ada0 Temp:    31°C
ada1 Temp:    31°C
da0 Temp:    34°C
da1 Temp:    35°C
da2 Temp:    41°C
da3 Temp:    41°C

Average HD Temp: 30.125°C
Average SSD Temp: 42.600°C
HBA Temp Average: 41.000°C