New FreeNAS Mini XL+ heat problem - do this right away

[MikeO]

Hi all,

Just got my new FreeNAS Mini XL+ today and loaded it with 8 6TB drives, mostly WD Red Pro.

While idling, the disk temperatures shot through the roof - several were in the 60 - 70C range.

Logging into the IPMI, it displays the speeds of the four fans. The default fan mode is "optimal", which gave a rear fan speed of 3100 rpm. Changing the mode to "HeavyIO speed" bumped that to 6000+ rpm and my drives started cooling down. There's also a full speed mode.

Make sure you change from optimal to HeavyIO or full - it's a bit noisier, but you won't melt your disks.

Best regards,
Mike

 

[MikeO]

Oops! Case fan went from 500 to 1200, the 6000 rpm fan is the cpu fan. Fan1 and Fan2 are the lower front case fans, Fan3 is the cpu fan, and FanA is the case fan.

 

[Basil Hendroff]

Platters are spinning faster at 7200 RPM on the Pro so they will run hotter and probably noisier. Ideally, for longevity, you want that to be below 43°C. I'm curious to know by how much you manage to get the temp down to and if you're able to keep the ambient noise down as well?

 

[MikeO]

With the HeavyIO mode, rear fan went from 500 to 1100 and the hottest drives dropped about 15C, although one dropped 30C (from 70).. Still warmer than I want, so just boosted it to full and got a rear fan of 1400 rpm. Cpu fan went from low 6000s to 7000, so that's most of the noise - high pitched.

Will let it run a few hours at the full mode and let you know. Willing to trade noise for disk life!

 

[Basil Hendroff]

Cpu fan went from low 6000s to 7000, so that's most of the noise - high pitched.

What's your CPU temperature?

 

[MikeO]

CPU temp stayed between 25 and 30C

 

[MikeO]

Found a better way to reduce internal temps.

To summarize:

• With the default "optimal" fan mode, temps on 8x 6 TB WD Pros were in the 60 to 70C range
• Changing fan mode via IPMI:
  • Heavy IO mode helped, but cpu fan rpm went from 3000 to 6000 and case fan from 500 to 1200 or so. Temps dropped about 15C
  • Full fan mode: cpu fan 7000, case fan 1400. cpu fan noise noticeable, disk temps much better.

Final solution was to get a computer fan extender cable (https://smile.amazon.com/gp/product/B01FFFHDKO/), then cut the PWM wire on the extender cable and insert it between the case fan plug and the Fan-A connector on the motherboard.

This allows the case fan to run at full speed while retaining rpm monitoring. I then reset the fan mode to "optimal" and the cpu fan was once again inaudible.

 

[Basil Hendroff]

I'm surprised you had to 'hack' the new Mini XL to get it to work the way you want it to. Do you think that's because it designed with (cooler) 5400 RPM disks in mind?

 

[MikeO]

I think it's a couple of issues. The fan modes were probably set by Supermicro when they did the firmware, and they couldn't predict the heat loads on the case. The Mini XL+, the way I've populated it, has 8 fairly warm disks in a small case with just one case fan.

I think the case fan selection was done by the case manufacturer, and probably assumed smaller disks. I think one of the reviews mentioned that iXsystems added the two small fans that cool the motherboard.

I had a similar problem with my previous DIY FreeNAS systems - the last one was a Norco 4220 case with 20 disks. It got toasty, even after I replaced all of the fans with higher output ones.

It seems that each generation of disks uses a little more power as the number of platters and rotation speed goes up. In the years after a case is designed, the heat it needs to dissipate goes up, invalidating the original heat assumptions (assuming that an engineer actually tried to calculate them).

Overall, I'm very happy with my new Mini-XL+. I bought it as diskless / 32 GB. Waiting for extra memory to arrive - the DIMMs that I pre-ordered were close but not compatible.

I'll write a new owner report in a few weeks.

 

[SoonerLater]

OP wrote
>>> Logging into the IPMI, it displays the speeds of the four fans.

Is this information which can only be seen via IPMI? I ask, because I sure don't see it on the normal web GUI.

Is IPMI only accessible via connecting an ethernet cable to the IPMI ethernet port? I can connect a second ethernet cable from there to the same network switch that the ordinary IX1 (or IX0) ethernet port used to connect my Mini-XL+ to the network. However, I don't have a practical way to connect the IPMI port straight to another computer.

 

[Redcoat]

However, I don't have a practical way to connect the IPMI port straight to another computer.

Why don’t you use the computer from which you are accessing the TrueNAS GUI to access IPMI?

Connect a cable from the IPMI port to your router. look for a new IP to be handed out by your router's DHCP - that's the IPMI address to use in your browser to access IPMI. Also suggest you set the IPMI mode in the BIOS - set to Dedicated instead of Failover.

It's not clear in your motherboard manual if IPMI is available shared on ETH0. If you are presently connected ETH0 to switch and your router shows an additional IP assignment that's "unfamiliar", try accessing that IP with your browser.