New TrueNAS Mini XL+, drive temperatures REALLY HOT

[rlpowell]

(Similar posts: https://www.truenas.com/community/threads/drive-temperatures-in-mini-xl.92451/ and https://www.truenas.com/community/threads/new-freenas-mini-xl-heat-problem-do-this-right-away.84713/ )

I just bought a brand new Mini XL+ (Amazon says: TrueNAS Mini XL+ Compact ZFS Storage Server with 8 + 1 Drive Bays, 32GB RAM, Eight Core CPU, Dual 10 Gigabit Network (Diskless)). I put in 2 https://www.newegg.com/seagate-ironwolf-st10000vn0008-10tb/p/1JW-001N-00023 and 2 https://www.newegg.com/red-plus-wd101efbx-10tb/p/1B4-005X-001D5 in the top 4 bays. It is louder than my desktop-style servers, so the fans are definitely doing *something*.

I install Scale right away because I'm used to linux and docker. I have done almost nothing else. No-one is using the system at all; the mean for CPU idle is 96%

The disks are burning hot to the touch. Like, the interface says they're at 55-66oC but there's no obvious hot air blowing out of the thing and the sides feel cool, so I popped a drive out and it almost burned my hand!

Ambient temperature is entirely comfortable; my "server room" is adjacent to my bedroom and there's no door between them. I don't actually know the exact temp in there but if it was higher than 27oC I'd be very surprised.

This is not what I expect from fresh hardware! What is going on? I could do the IPMI thing from one of the posts I linked, but, *really*? I shouldn't have to do that on a fully-supported piece of hardware.

 

[rlpowell]

Just in case it's useful to anybody else, here's a totally different suggestion about this issue: https://www.amazon.com/iXsystems-TrueNAS-Mini-XL-Diskless/product-reviews/B08FCVFXLC

 

[rlpowell]

Soooooo, in case anyone else hits this, support won't discuss this issue with me because I'm not using quite the right drives. The drives are in the recommended *product lines*, but not actually the correct recommended models per the list at https://www.truenas.com/truenas-mini/

While I understand their position, I'm pretty unhappy; it seems to me that if buying a very slightly different model of drive makes your system unusable, your system's thermal behaviour is way too close to the line.

Mostly just sharing for the next person coming along doing this search.

 

[jgreco]

The drives are in the recommended *product lines*,

You probably need to explain that in order for this to be a meaningful message.

My recollection is, for example, that WD has Red Plus models that are both 5400RPM and 7200RPM.

https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-red-plus-hdd/product-brief-western-digital-wd-red-plus-hdd.pdf

The 7200RPM drives are expected to be significantly warmer than their 5400RPM counterparts, despite being the same WD Red "product line". The WD80EFBX, for example, is an 8.8 watt/5.3 watt unit whereas the WD60EFXX is a 4.7 watt/3.1 watt unit, a substantial difference. One of the problems in these little chassis is that it is difficult to force airflow around the drives. In a rackmount chassis, high static pressure differentials (and noisy powerful fans) do that trick, but in a smaller case, it is often exceedingly difficult to get the airflow to move around (and particularly in between) the drives. Failure for cooling here results in hot spots developing.

So if you've piled up a bunch of high wattage drives in your chassis, you may have sabotaged its ability to keep those disks cool. There are numerous reasons manufacturers provide lists of recommended drives.

You might also want to check airflow using a sheet of paper to see if air ingress/egress seems reasonable, or alternatively the "candle smoke" test.

 

[rlpowell]

Yep, it does indeed turn out that I bought drives with average operating power around 8W. Weirdly, *all* the WD Red Plus drives that 10TB or above are 7200 RPM, but the 10TB ones on the data sheet you linked have *much* higher wattages than the 12TB and 14TB ones; I have no idea why that is.

Thank you for the info, that is the sort of info I was looking for.

I feel like "you'd better pick these drives exactly or your system can easily end up unusable" could have been a *lot* more obvious during the purchasing process, and I've let them know that, but it is definitely the case that I made a mistake here.

At this point my goal is more about making sure future searchers get this info, instead of just being super confused the way I was.

The toilet paper test showed pretty weak suck on the bottom front, and ... OK I guess blow on the middle back.

 

[Ericloewe]

Weirdly, *all* the WD Red Plus drives that 10TB or above are 7200 RPM, but the 10TB ones on the data sheet you linked have *much* higher wattages than the 12TB and 14TB ones; I have no idea why that is.

Helium vs. air is typically the reason.

 

[rlpowell]

Helium vs. air is typically the reason.

LOL for a second I thought you were trolling me or subtly insulting me or something? (Because "hot air" or helium making people make funny voices or something?)

But no you were being accurate and literal about an actual thing I'd never heard of. :D That's super cool (heh), thanks.

 

[jgreco]

LOL for a second I thought you were trolling me or subtly insulting me or something? (Because "hot air" or helium making people make funny voices or something?)

<in squeaky voice>Yeah, around here, we might do that.</>

There has been a catastrophic shift towards random crap showing up labeled as a HDD in recent years. It used to be that you could identify critical characteristics of a drive family by the line name; back in 1995, for example, a Seagate Hawk was a 5400RPM drive, a Barracuda was 7200RPM, and faster drives were Cheetah etc.

Unfortunately, marketing folks noted that you could fool some of the people some of the time by marketing a Hawk-class drive as a Barracuda, and today if you buy a Barracuda it is likely to be a 5900RPM affair.

I don't blame you for being confused by the product lines being conflated in this manner; as someone who's done this professionally for years, I also resent needing to go to spec sheets to pull out critical bits of information. I'm just saying that you have every right to be a bit miffed, and also you shouldn't feel too bad about having been fooled in this manner.

Shame on Seagate and WD.

 

[rlpowell]

I hadn't even thought about it in that way, as being more about deception on the manufacturer's end. Fair point.

 

[Davvo]

If you can do, spinning down your drives could be your only option now.

 

[joeschmuck]

Did I miss something? You stated that you installed the four drives in the top four drive bays and it was hot! Did you ever consider doing every other drive bay for now? This will place a large gap between the heat producing drives and allow them to cool down. If you haven't done this already, do it now.

If you can do, spinning down your drives could be your only option now.

this is not a recommended option due to it tends to shorten the life of a typical hard drive, but it is an option.

You may not desire to do something drastic as this but if you do plan to keep the system then you should "upgrade" the air flow across the drives and out of the case. I wouldn't want you to void any iX warranty so case modification is entirely up to you. You might be able to install a high flow fan in the rear of the case to help pull air out the back. If there are any air inlets besides the ones in front of the hard drive bays, place tape over them to block the air flow, this will allow more airflow to come into the case from the hard drive vents. If one of the inlet vents is aimed at your motherboard then evaluate it and you may need to leave some of them open, at least partially.

 

[jgreco]

shorten the life of a typical hard drive, but it is an option.

Also can blow your power supply all to hell and back when it isn't properly sized to handle concurrent spinups.

 

[Davvo]

this is not a recommended option due to it tends to shorten the life of a typical hard drive, but it is an option.

If his temps are as high as he says it would probably lengthen their lifetime.

Also can blow your power supply all to hell and back when it isn't properly sized to handle concurrent spinups.

I forgot about this.

 

[rlpowell]

I did, in fact, stagger the drives until I got the correct model, but I would like the option of using all 8 bays in the future.

I now have the top four bays installed with WD120EFBX, and after being in there all night with no significant load they look like this:

• sda: 52 °C
• sdb: 57 °C
• sdc: 55 °C
• sdd: 49 °C

That seems like a lot for drives that aren't really doing much of anything?, I dunno, what do y'all think?

The whole point for me of buying a complete system instead of building it myself was to *not* have to do things like add fans until the temps are good; if this system cannot in fact provide reasonable temps with the drives it is qualified on I'm going to talk to them about sending it back.

 

[jgreco]

The whole point for me of buying a complete system instead of building it myself was to *not* have to do things like add fans until the temps are good; if this system cannot in fact provide reasonable temps with the drives it is qualified on I'm going to talk to them about sending it back.

As mentioned in #3 above, it appears the drives you're using are not actually the correct kind of drives. The nature of the beast is that not every device is suitable for every job.

When I went down to Texas some years back to collect some servers being donated to one of my clients, I carefully checked the load rating of the pickup truck I was using, and made sure we didn't overload it even though there was definitely still some space available in the truck by the time I declared it "full" (note: about 46 servers). Even at that it was a little bit of a scary thing, but it was nice to know I was within the manufacturer's specifications and they felt that it would be capable of the job. Checking to make sure I'm within specs, though, was my job.

 

[rlpowell]

As mentioned in #3 above, it appears the drives you're using are not actually the correct kind of drives. The nature of the beast is that not every device is suitable for every job.

Yes, we definitely already had that conversation and I heard what y'all said. :)

Apparently I was insufficiently clear: I now have 4 WD120EFBX in the top 4 bays. These *ARE* supported drives.

The idle temps just above were idle temps *with the supported drives*.

Here's the temps after about 90 minutes continuously running md5sum on a bunch of movies and stuff:

sda: 52 °C
sdb: 58 °C
sdc: 56 °C
sdd: 51 °C

So, OK I guess?

 

[jgreco]

Apparently I was insufficiently clear

Sorry, missed that. It's difficult since I often participate in many threads simultaneously.

The idle temps just above were idle temps *with the supported drives*.

I'd say that sucks, but it may instead be that it doesn't blow sufficiently. (har, har).

Honestly, I look for drive temps to be around 30'C and I'm uncomfortable if they are significantly off that. There's some significant difference of opinion about this in the industry; Google and Backblaze both feel that heat is not a significant factor, and indeed I have seen drives run hellishly hot without apparent issue.

Hard Drive Temperature—Does It Matter?

How much does operating temperature affect the failure rates of disk drives? Not much. The unlimited online backup service provided by Backblaze requires

www.backblaze.com

On the other hand, especially back in the '90's, I saw lots of Quantum Fireball Plus and Seagate Cheetah class drives cook themselves to death.

My own perspective from the POV of a small specialist shop that does lots of server work is that I don't have the luxury of letting things run hot and praying for the best, so I spend a lot of time working on cooling. I run servers as far as 2500 miles away and it is *expensive* to fix things that break.

I can also say that it is REALLY difficult to get good cooling out of a chassis like the one used for the Mini's. In the rackmount cases, you have high static pressure differential being used (high pressure fans) to force air through the millimeter-sized gaps that surround drives. Most of the "desktop" style NAS cases are not designed to force air in between the drives, which is where heat pockets build up. Unfortunately, short of custom-building a NAS-optimized chassis, iXsystems has to pick from the options that are available off the shelf from China Genericom Shenzhen Backalleyco. If they were selling the Mini at volumes rivalling Synology or QNAP, then they'd probably have a nice custom chassis with optimized airflow.

I think I'd be unhappy-ish at those temperatures, and if you now have the recommended drives, it may be worth seeing what iXsystems support has to say.

 

[joeschmuck]

I did, in fact, stagger the drives until I got the correct model, but I would like the option of using all 8 bays in the future.

I now have the top four bays installed with WD120EFBX, and after being in there all night with no significant load they look like this:

• sda: 52 °C
• sdb: 57 °C
• sdc: 55 °C
• sdd: 49 °C

That seems like a lot for drives that aren't really doing much of anything?, I dunno, what do y'all think?

What I think is that you should stagger the drives again so they run cooler. I suspect that when you hit 66C, your warranty will be void for the respective drive so in my opinion you are playing with fire (no pun intended). Not sure if you knew the drive retains it's minimum and maximum temperatures.

When do you plan to add another 4 drives? If it's next year then you have time to figure it out. Or maybe iX Systems will now help you with the cooling issue. It could be a faulty fan.

I would be upset too if I had this kind of problem.

Good luck!

 

[rlpowell]

Just to close things out, after extensive back-and-forth with support, the resolution is "yes, it's 25oC above ambient, but it's still under the manufacturer's temp cap, so we're going to call it good".

Which, I mean, I'm not ecstatic that I bought a system that has drives idling at 52oC, but the cap is 65oC so it's ... fine.

I've seen other threads around here with actual ixSystems engineers being like "yeah we work with the manufacturers, we promise as long as you stay under their rated temps it's going to be OK", so, sure, OK, I'm rolling with it.

They pointed out the IPMI mechanisms for changing the fan speed, from the same thread I linked to at the beginning, which I probably won't try unless things get worse.

 

[jgreco]

Just to close things out, after extensive back-and-forth with support, the resolution is "yes, it's 25oC above ambient, but it's still under the manufacturer's temp cap, so we're going to call it good".

Yuck. Sorry to hear it.