Power Management of Disks

MalVeauX

Contributor
Joined
Aug 6, 2020
Messages
110
Hey all,

Do you all keep all your disks in a constant state of spinning, or do you let them spin down after a period of time? I'm looking at HDD standby and Advanced Power Management of discs in general. I would think for a frequently accessed machine, always up would be ideal. But if something is not used at night, at all, do you let it spin down? There are other options, like minimal use but without spindown, etc. I'm curious if anyone is using any of this for a file server with application purposes such as just having available access to media files to play on client machines.

Very best,
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
Spin spin spin!

Seriously, having disks spin down, park their heads, and then spin up again, accelerates wear on the spindle bearings, and risks head crashes. You want to have that steady-state lubrication film in the bearing spindles, and the air film keeping the heads flying.
 

MalVeauX

Contributor
Joined
Aug 6, 2020
Messages
110
Got it; so let them fly.

I was asking based on power consumption point of view. I noticed when I let things spin down, of course, power consumption goes down. With a few discs spinning, power is up. I'm idling currently with 3 discs all spinning at 51 watts. The lowest I was able to get it, idle, with things parked, was around 42 watts. Not a big difference. But I'm sure that matters with large pools with many discs involved.

It just had me thinking about large periods of down time, such as when the entire family is sleeping, and it's not accessed easily for 6~8 hours at a time.

But if its more wear & tear on the discs to spin down, the power consumption difference with them all constantly spinning is negligible really for me, with a 9~10 watt difference overall currently (which may go up as I add more discs, but just seeing what my footprint will be).

Very best,
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
APM requires hardware support in the drive. You can see if your drives support APM via camcontrol identify <device> | grep "advanced power management". Most drives nowadays no longer support APM.
 

MalVeauX

Contributor
Joined
Aug 6, 2020
Messages
110
APM requires hardware support in the drive. You can see if your drives support APM via camcontrol identify <device> | grep "advanced power management". Most drives nowadays no longer support APM.

Understood, thanks, that solves everything!

Very best,
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
In all cases, do some maths here :
9 watts saved ; over 1 000 hours, that will be 9K Watt hour. Considering where you are, that can be even less than 1 dollar. So to payback a 150$ drive, you need 150 000 hours. That is over 17 years.

And that is at 24 hours per day. If you are in saving mode 1 third of the time, you are up to 51 years.

Power saving like this is just plainly insignificant....
 

MalVeauX

Contributor
Joined
Aug 6, 2020
Messages
110
Agreed, good point! Spinning they shall!

Very best,
 

Volte

Dabbler
Joined
Feb 11, 2016
Messages
19
Sorry to zombie an old thread (although I think if I'm really honest with myself, I'm not—I feel like that's the whole point of forums, to help organize information).

So, given all that was discussed above, are we proponents for or against using APM in TrueNAS? Seems to me like level 254 would be ideal? At first I was thinking 128, but that leaves me thinking that the disk will only ever reach minimum power usage, even when it "needs" more power (being used). Curiously, 254 is the only one that mentions nothing by way of spindown/standby. I assume that it is intended to be obvious that there is no standby.

What's the default behavior for disks when this is Disabled (appears to be the default setting).

Screen Shot 2021-07-09 at 08.08.04.png
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,110
What's the default behavior for disks when this is Disabled (appears to be the default setting).
Disabled is APM 255 - not all drives fully respond to this, but it would be equivalent to APM 254 - full performance, full power, no spindown.

Generally speaking, "no spindown" is the preferred option for drive longevity.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,358
The most important thing is to not let the drives get into a situation where they are spinning down, just to spin up again a few moments/minutes later. This is what causes horrendous load cycle counts.

Personally, there's always something hitting the drives... thus they never spin down for long enough to warrant the wear. So I disable the spin down.

It might not help that the fan script is pinging the drive for its temp every 3 minutes.

If you had a backup chassis it may make sense to powerdown the chassis when its not backing up... and then perhaps use some sortof timed or WOL thing before the backup is due to run.
 

scandrew

Cadet
Joined
Sep 22, 2021
Messages
5
My drives (IronWolf 16TB) keep parking the heads even in the APM 254 (Max power). I recently moved System Dataset from them to the SSD pool, and now they keep parking the head every few minutes. Is there a way to stop them from doing so?
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
You must provide us with the complete details about your hardware. One possibility is that TrueNAS does not see or handle the drives itself. That would explain why it can not prevent the disks to spin down. But that would also means your data are at high risk because TrueNAS and ZFS must have that direct disk access. IronWolf or not, if they are hidden behind a Raid Controller, that entire pool will have to be destroyed and re-created with proper resources.

But before going any further, give us here the complete details about your hardware...
 

hliang

Cadet
Joined
Sep 16, 2023
Messages
2
Here is my experiment result:

Level 1, minimum power usage with standby (spindown). Consume average 33watts when there is no load. I can hear the spin up.

Level 64, intermediate power usage with standby. Consume average 35w. I am not sure if it spin down or not.

Level 128, minimum power usage without standby 44W.

So if allow standby, then I save 9w. In Toronto, 1watt*year (7x24) cost $1.6CAD. 9watts*year saving is $14.4CAD.

I am curios the standby vs spindown. Can someone explain the difference. My current setting is level 64, HDD standby 180min. Seems good balance to me.
 
Joined
Oct 22, 2019
Messages
3,579
Level 64, intermediate power usage with standby. Consume average 35w. I am not sure if it spin down or not.
It does. Anything 127 or lower allows spin-down. The difference is the amount of idle time that will trigger a spin-down.

So if allow standby, then I save 9w. In Toronto, 1watt*year (7x24) cost $1.6CAD. 9watts*year saving is $14.4CAD.
Is it worth possibly saving $14 per year, at the cost of extra wear on your HDDs by increasing their spin-down/spin-up cycles?

Now consider the cost of replacing a new HDD, which is more likely to happen sooner (and more often) by stressing out the mechanical components of an HDD due to an increased number of spin-downs and spin-ups.

(Just noticed this thread started 3 years ago...)
 
Last edited:

hliang

Cadet
Joined
Sep 16, 2023
Messages
2
It does. Anything 127 or lower allows spin-down. The difference is the amount of idle time that will trigger a spin-down.
Thanks for the information. That is what I am looking for. Anyway switch to always on.
Now consider the cost of replacing a new HDD, which is more likely to happen sooner (and more often) by stressing out the mechanical components of an HDD due to an increased number of spin-downs and spin-ups.
It is not the absolute cost bother me. It is comparing to OneDrive. I have subscription that gives me 6TB for yearly price $124. My NAS server cost me $200 + electricity + old hardware and give me 8TB.
 

DanRelfe

Cadet
Joined
Nov 29, 2023
Messages
1
APM requires hardware support in the drive. You can see if your drives support APM via camcontrol identify <device> | grep "advanced power management". Most drives nowadays no longer support APM.
Sorry to bring back this old thread, but Google brought me here.

I've been using Freenas since, well, I can't remember, must be since around 2005 ish, but recently installed a new system using Truenas Core. I was poking around power management settings with the view to reduce power usage (even if it's just by a bit..) and came across the above command.

I am using 2x Netapp disk shelves, a DS2246 and a D4246, connected via a PM8003 (Sierra) card and IOM6 SAS controllers. The DS4246 houses 24 SATA drives (it has SAS-->SATA headers) but the DS2246 houses 24 SAS drives.

The above command camcontrol identify <device> works on my DS4246 SATA drives but returns the following when I try it on my DS2246 SAS drives, so I tried camcontrol inquiry <device>

Any idea if I can see if the SAS drives support power management ? The SATA drives appear to support APM so I have set them all to 128.

Thanks.
 

Attachments

  • Capture.PNG
    Capture.PNG
    5.7 KB · Views: 70

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
connected via a PM8003 (Sierra) card and IOM6 SAS controllers.
I'm amazed you've gotten this far with that thing, most products from that end of the SAS world crash and burn spectacularly before you even get a semi-functional system. You should really look into replacing that with an LSI HBA, unless this is just a test system.
Any idea if I can see if the SAS drives support power management ?
Well, as they implement the SPC-4 command set, they are expected to handle SCSI power commands. Whether they honor them is an entirely different matter, however...
 

Slava L

Cadet
Joined
Dec 2, 2023
Messages
1
It does. Anything 127 or lower allows spin-down. The difference is the amount of idle time that will trigger a spin-down.


Is it worth possibly saving $14 per year, at the cost of extra wear on your HDDs by increasing their spin-down/spin-up cycles?

Now consider the cost of replacing a new HDD, which is more likely to happen sooner (and more often) by stressing out the mechanical components of an HDD due to an increased number of spin-downs and spin-ups.

(Just noticed this thread started 3 years ago...)
Greetings,
it was a long-term dilemma for me as well...
I have started to keep a record of my HDDs' start-stop counts with only one aim to see whether it is worth it to keep the HDDs in sleep mode.
I came to the following results* :
Power, W
Spin Down all 4 drives - 26W - Idle mode
Spinning Up all 4 drives - 44W
Spin Up (2 new WD Red) - 34W
Spin Up (2 old WD drives)- 38.5W

** Start-up of the system (a peak power) - 62W

I don't use the NAS daily, only once or twice a week or so, for the backup or access archives, plus PLEX (movies). The TrueNas is set to do scrubbing and SMART twice per week for all drives.
On average, I have got 2-3 times wake-ups (start) per disk per day, monitored over a few months. That leads me to the following (*- my own conclusion):
  1. I would expect drives to have a lifetime between 16-17 years, in the worst case - 8 years only, based on start/stop cycles (lifetime).
  2. The new drives have a higher number of Unload/Load cycles compared to the older ones, also energy efficiency. New disks roughly consume ~4 W when spinning; old ones are ~6.5W
  3. In case of early failure of HDD (8 years) - the overall savings over the period is worth of 5 new drives*** (***- cost I paid for 4Tb ~65Eruo/disk). In the case of an averaged lifetime, the savings are worth 11 drives. (0.26 €/kWh)
  4. I prefer to have NAS at low power consumption, quiet, and seating at low temperature (~30-32 deg) with only one casing fan running at lowest setting, with passive cooling of CPU. The NAS was not build for highest performance as it runs on ASUS AT3IONT-I DELUXE with external power supply, and built for compactness and quietness. It has 4x HDDs and 1x SSD (32 Gb) for TrueNAS OS.
1x WD7500AAD
2x WD40EFZX1x WD10EZEX
Load Cycles600000
Load/Unload Cycle
300,000
Load/Unload Cycle
MinMaxSpecSpec
Start/Stop Cycles (life-time)10000200002000010000
Start/Stop Cycles (per year)118610161186608
Years (life-time)8201716

Start/Stop cycles were assumed to be ~30 times lower than Load/Unload cycles declared in datasheets.


*A few thoughts. Yes, there is thermal and mechanical stress introduced to the electrical motor during start. But it should be manageable.
*Bearings should be absolutely fine, as 2–3 starts per day provide sufficient “preservation” to avoid any issues with bearings and rotors (pitting, bowing). From my personal experience with big rotating equipment (motors, pumps, compressors) with a weight of several tons, we need to do a once / twice a month partial rotation if machines are in long-term preserved mode. In case of idle, operation, or short preservation, the machine's start frequency is weekly. The small bearings and weight of rotors in HDDs are far from the industrial weight, and there by great amount of strength margin from the materials used.
*A thermal stress. I found it as a negligible risk. All my HDDs normally stay within the 32-45 deg C range, while the operating range (per design) is up to 65 deg C. When all running, the running temperature of HDDs is close to 42-45 Deg C. When all are idling the temperature drops to 32-34 degrees C, as the overall NAS temperature. So the thermal cycle of 10–15 degrees has negligible impact on the integrity, as of my thoughts.
 

Sawtaytoes

Patron
Joined
Jul 9, 2022
Messages
221
@Slava L, thanks for such great information! I was wondering about the same thing (https://www.truenas.com/community/threads/how-to-make-500w-hdds-spin-down-or-draw-less-power.114791/).

My situation is extreme compared to everyone else here since I'm at 500W idle with the "Disabled" or 255 value, and that's just my HDDs in a JBOD chassis; no CPU or anything else. I would assume any power management value would have a significant impact on my power usage.

I went with 60 HDDs, all HGST Helium. I figured that with Helium, the spin up and spin down would be less of an issue, but even trying that 128 value (no spindown, but less power) could be useful. Like @Slava L, I also really only use my NAS for Plex, but this HDD zpool is actually a backup. I never read data off it unless I need to recreate a zpool (which I had to do recently). 500W is too much power for the little value I get out of it.

That camcontrol command isn't available in TrueNAS SCALE. Is there something else I can use to set the Advanced Power control on all 60 drives in one go? I wanna try 128 and see how much power it uses.

EDIT: I ran hdparm -B /dev/sdX on every drive and got this:
Code:
/dev/sdX:
 setting Advanced Power Management level to 0x80 (128)
SG_IO: bad/missing sense data, sb[]:  72 05 20 00 00 00 00 10 02 06 00 00 c0 00 00 00 03 02 00 21 80 02 f8 21 00 00 00 00 00 00 00 00
 APM_level      = not supported

Is it that the drives don't support it or that the LSI SAS controller doesn't support it? Or is there something I'm doing wrong?
 
Last edited:
Top