HDD Spindown Timer 2.2.0

[GreaseMonkey88]

I checked back, and I might have overlooked something or used the wrong command to list... The script was still there, so false alarm from my side

The script is working so far on two machines just fine!

 

[NAK]

Does anyone know a way to use this script as post init script on 11.3(U1)? There is only an option to let it get killed after x seconds.

 

[ngandrass]

Does anyone know a way to use this script as post init script on 11.3(U1)? There is only an option to let it get killed after x seconds.

Ah I see, the task timeout was introduced by the 11.3 release. Interestingly the script is never killed by FreeNAS for me, therefore I never noticed the new timeout feature. However, that might be due to the upgrade path I took. For this reason I just performed a quick test on FreeNAS-11.3:

1. Delete old task that was present before the upgrade to 11.3
2. Re-create new task leaving the default timeout of 10 seconds
3. Check if the script is killed by FreeNAS

Interestingly the script still executes just fine without getting killed even though the middleware log indicates the registered timeout:

Code:

[2020/03/02 11:11:54] (DEBUG) middlewared.execute_init_tasks():173 - COMMAND /bin/bash -c 'SCRIPT="/mnt/ssd/spindown_timer.sh"; ARGS="-t 3600 -p 600 -i ada6 -i da0"; CHECK=60; while true; do if [ -f "${SCRIPT}" ]; then ${SCRIPT} ${ARGS}; break; else sleep ${CHECK}; fi; done' timed out

I launched it as a Post Init Command, described in the corresponding README section, leaving the default 10 second timeout. See: https://github.com/ngandrass/freenas-spindown-timer#automatic-start-at-boot



Can you please confirm if this works for you? I'd than update the documentation accordingly. Thanks!

 

[NAK]

I can confirm that it is working. Should have tried it before :)
I hope they will fix the timeout but provide an option to disable it.

 

[ngandrass]

I can confirm that it is working. Should have tried it before :)
I hope they will fix the timeout but provide an option to disable it.

Perfect, thanks for confirming this. I definitely keep an eye on the behavior of that option though.

 

[ngandrass]

ngandrass updated HDD Spindown Timer with a new update entry:

Deployment instructions updated with FreeNAS-11.3 changes

The deployment instructions, provided in the README.md, now incorporate information on the new task/command timeout feature introduced by the FreeNAS-11.3 update.

See: https://github.com/ngandrass/freenas-spindown-timer/commit/013d7f1b20721cb98709363d23911dc60ff6c63a

Read the rest of this update entry...

 

[MikeyG]

@ngandrass I'm having an issue where half my drives keep waking up. I have 8 drives in a RAIDZ2 vdev, with 4 on a 9300 HBA, and 4 attached via SATA cables to the motherboard. Every few minutes the da drives wake up and need to be spun down again, while the ada drives stay asleep. I'm not sure how this is possible, since any write to the pool should be waking up all drives. I doubt this is an issue with your script, since it keeps successfully spinning everything down again, but figured I'd ask. This behavior seems to have only shown up since upgrading to 11.3.

Code:

root@backupnas:/mnt/nvme/Scripts # ./spindown_timer.sh -t 120 -p 60 -v -i ada4 -i da4 -i da5
Detected drive ada1 as ATA device
Detected drive ada0 as ATA device
Detected drive ada3 as ATA device
Detected drive ada2 as ATA device
Detected drive da0 as ATA device
Detected drive da1 as ATA device
Detected drive da2 as ATA device
Detected drive da3 as ATA device
Monitoring drives with a timeout of 120 seconds: ada1 ada0 ada3 ada2 da0 da1 da2 da3
I/O check sample period: 60 sec
2020-04-10 13:07:24 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:08:24 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:09:25 Spun down idle drive: ada1
2020-04-10 13:09:25 Spun down idle drive: ada0
2020-04-10 13:09:26 Spun down idle drive: ada3
2020-04-10 13:09:26 Spun down idle drive: ada2
2020-04-10 13:09:26 Spun down idle drive: da0
2020-04-10 13:09:27 Spun down idle drive: da1
2020-04-10 13:09:27 Spun down idle drive: da2
2020-04-10 13:09:28 Spun down idle drive: da3
2020-04-10 13:09:28 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:10:28 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:11:28 Drive is already spun down: ada1
2020-04-10 13:11:28 Drive is already spun down: ada0
2020-04-10 13:11:28 Drive is already spun down: ada3
2020-04-10 13:11:28 Drive is already spun down: ada2
2020-04-10 13:11:28 Drive is already spun down: da0
2020-04-10 13:11:28 Drive is already spun down: da1
2020-04-10 13:11:28 Drive is already spun down: da2
2020-04-10 13:11:28 Drive is already spun down: da3
2020-04-10 13:11:28 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:12:28 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:13:28 Drive is already spun down: ada1
2020-04-10 13:13:28 Drive is already spun down: ada0
2020-04-10 13:13:28 Drive is already spun down: ada3
2020-04-10 13:13:28 Drive is already spun down: ada2
2020-04-10 13:13:28 Drive is already spun down: da0
2020-04-10 13:13:28 Drive is already spun down: da1
2020-04-10 13:13:28 Drive is already spun down: da2
2020-04-10 13:13:28 Drive is already spun down: da3
2020-04-10 13:13:28 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:14:28 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:15:28 Drive is already spun down: ada1
2020-04-10 13:15:28 Drive is already spun down: ada0
2020-04-10 13:15:28 Drive is already spun down: ada3
2020-04-10 13:15:28 Drive is already spun down: ada2
2020-04-10 13:15:29 Spun down idle drive: da0
2020-04-10 13:15:29 Spun down idle drive: da1
2020-04-10 13:15:30 Spun down idle drive: da2
2020-04-10 13:15:30 Spun down idle drive: da3
2020-04-10 13:15:30 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:16:30 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:17:30 Drive is already spun down: ada1
2020-04-10 13:17:30 Drive is already spun down: ada0
2020-04-10 13:17:30 Drive is already spun down: ada3
2020-04-10 13:17:30 Drive is already spun down: ada2
2020-04-10 13:17:30 Drive is already spun down: da0
2020-04-10 13:17:30 Drive is already spun down: da1
2020-04-10 13:17:30 Drive is already spun down: da2
2020-04-10 13:17:30 Drive is already spun down: da3
2020-04-10 13:17:30 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:18:30 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:19:30 Drive is already spun down: ada1
2020-04-10 13:19:30 Drive is already spun down: ada0
2020-04-10 13:19:30 Drive is already spun down: ada3
2020-04-10 13:19:30 Drive is already spun down: ada2
2020-04-10 13:19:31 Spun down idle drive: da0
2020-04-10 13:19:31 Spun down idle drive: da1
2020-04-10 13:19:32 Spun down idle drive: da2
2020-04-10 13:19:32 Spun down idle drive: da3
2020-04-10 13:19:32 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:20:32 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:21:32 Drive is already spun down: ada1
2020-04-10 13:21:32 Drive is already spun down: ada0
2020-04-10 13:21:32 Drive is already spun down: ada3
2020-04-10 13:21:32 Drive is already spun down: ada2
2020-04-10 13:21:32 Drive is already spun down: da0
2020-04-10 13:21:32 Drive is already spun down: da1
2020-04-10 13:21:32 Drive is already spun down: da2
2020-04-10 13:21:32 Drive is already spun down: da3
2020-04-10 13:21:32 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:22:32 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:23:32 Drive is already spun down: ada1
2020-04-10 13:23:32 Drive is already spun down: ada0
2020-04-10 13:23:32 Drive is already spun down: ada3
2020-04-10 13:23:32 Drive is already spun down: ada2
2020-04-10 13:23:32 Drive is already spun down: da0
2020-04-10 13:23:32 Drive is already spun down: da1
2020-04-10 13:23:32 Drive is already spun down: da2
2020-04-10 13:23:32 Drive is already spun down: da3
2020-04-10 13:23:32 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120
2020-04-10 13:24:32 Drive timeouts: [ada1]=60 [ada0]=60 [ada3]=60 [ada2]=60 [da0]=60 [da1]=60 [da2]=60 [da3]=60
2020-04-10 13:25:32 Drive is already spun down: ada1
2020-04-10 13:25:32 Drive is already spun down: ada0
2020-04-10 13:25:32 Drive is already spun down: ada3
2020-04-10 13:25:32 Drive is already spun down: ada2
2020-04-10 13:25:33 Spun down idle drive: da0
2020-04-10 13:25:33 Spun down idle drive: da1
2020-04-10 13:25:34 Spun down idle drive: da2
2020-04-10 13:25:34 Spun down idle drive: da3
2020-04-10 13:25:34 Drive timeouts: [ada1]=120 [ada0]=120 [ada3]=120 [ada2]=120 [da0]=120 [da1]=120 [da2]=120 [da3]=120

 

[ngandrass]

Every few minutes the da drives wake up and need to be spun down again, while the ada drives stay asleep.

Might it be possible that your system dataset is located on the problematic drives? If not, it might be an error in the script during timeout updates.

I'm not sure how this is possible, since any write to the pool should be waking up all drives.

I actually noticed that reads to small files can partially wake up pool disks. Would fit my guess regarding the system dataset.

 

[MikeyG]

Thanks. Yes, I double checked system dataset. It's on a separate SSD drive.

The system does have SMB shares, but there are no automated tasks or users that access them. Most files I store are fairly large too, so should wake up all disks. I'd assume even if small files woke up only some of the drives, it would be more random, and not just the same 4 da drive every time (while leaving ada drives alone).

It serves as a backup server only - no jails, plugins, or anything that should activate disks outside of the daily backup.

 

[MikeyG]

Verified this is happening on another system (dataset also on ssd drives). That one has a 3 drive mirror pool, whose only purpose is as a replication target that is active once per day. Those drives also are spun up every couple of minutes and then spun down again by the script. I would guess the script is doing what it's supposed to, and seeing that there is no activity on the drives and correctly spinning them down. I wonder if FreeNAS is doing something with the drives that doesn't involve writing any data to them that wakes them up. I know there were issues with LSI 3008 based controller and some changes were made in 11.3-U2 to compensate. Both systems that I have use those controllers.

Do you have a system running U2 that you can use to confirm it still works as expected?

 

[MikeyG]

I did a little more testing, and it seems that disabling SMART on one of the drives allows it to stay spun down as normal, whereas the others woke up again after a few minutes. There may have been a change in 11.3-U2 that is accessing drive's SMART data for reporting - at least for drives on an HBA.

 

[ngandrass]

ngandrass updated HDD Spindown Timer with a new update entry:

Successfully tested on FreeNAS-11.3-U3.1

The script was successfully tested on the latest FreeNAS-11.3-U3.1 release :)

Read the rest of this update entry...

 

[MikeyG]

@ngandrass Thanks for verifying it works for you. Are you using an HBA in your testing? On U3.1 and drives still wake up unless I disable SMART on them. None of the drives I'm testing on have system dataset on them or any other activity. Using camcontrol standby dax for testing.

I have a 8 drive RAIDZ2 vdev, and if I disable SMART on 4 of the drives, exactly those 4 stay spun down indefinitely. The others wake up after 2-3 minutes.

 

[MikeyG]

I think I figured it out. Switching HDD Standby to 5 minutes seems to disable temperature monitoring (whether active or in standby), and now allows drives to stay asleep. This might be a result of https://jira.ixsystems.com/browse/NAS-105656 which has temperature monitoring falling back to smartctl if SCSI temp monitoring fails. There may be some error in how it's coded or working, since my spinning drives do return temperature data without smartctl, (I believe the primary method is sg_logs), and it may be failing over to smartctl anyway.

In any case, I'm content to forgo temperature monitoring on the drives in question.

 

[tophee]

Hi,
I've just tried to get this working on my FreeNAS system but I'm running into an issue.

I've followed the how-to through but failing at the Deploying the Script. I've copied the script to over to my FreeNAS box, issued the chmod +x shutdown_script command as per the instructions. Then tried to run the script from the cmd line, and I get this:

spindown_timer.sh: declare: not found
spindown_timer.sh: 47: Syntax error: "(" unexpected

I'm afraid I don't know enough to try and address this.

Thanks

Oh, on FreeNAS-11.3-U3.1

 

[ngandrass]

I think I figured it out. Switching HDD Standby to 5 minutes seems to disable temperature monitoring (whether active or in standby), and now allows drives to stay asleep.

Sorry, I somehow didn't receive a notification about your reply to this thread... It's good to hear that you were able to kinda solve the spindown issues :)

Just for reference, here are my disk and S.M.A.R.T configs:

 

[ngandrass]

I've just tried to get this working on my FreeNAS system but I'm running into an issue.

Looks like you are trying to execute the bash script in an sh environment. Please try the following:

Code:

chmod +x spindown_timer.sh
./spindown_timer.sh

Background: The script requires built in commands of the bash shell, which is automatically selected by the shebang line (#!/usr/bin/env bash). When you start the script through sh spindown_timer.sh it is executed in sh instead of bash. I hope this fixes your issue. If not, don't hesitate to ask for additional help!

 

[tophee]

Hi ngandrass,

Thanks for the info. Using the alternative command you provided I completed the setup and this seemed to go well, but I do not think it is running correctly on my system.

Two of my HDD do not appear to support advanced power states. I've run all the various commands and I am not getting the expected results. This may be related to BIOS settings when I first built this system years ago. I think I did not set the drives to be ACHI years ago when I first built this system (same drives different motherboard etc.) I'm beginning to think I'll go back up everything and start again. Thanks for your help and I may be back after I've backed up and rebuilt my pools.

Many thanks

 

[joeschmuck]

Just a quick little warning for folks who think sleeping their drives has no risks... (Just an opinion and you know what they say about opinions ;))

First of all, I use to sleep my drives a very long time ago and if you have the proper setup then sleeping the drives could be good for you. Different people have different reasons for sleeping drives and I'm not here to talk you out of sleeping your drives, I'm just here to provide you with some technical data and you can make your own decision.

Here it the warning... Pay attention to your Load_Cycle_Count and Start_Stop_Count on your hard drive. Hard drive hardware failures are typically related to the spindle motor current surge when spinning up a drive or head failure due to excessive head loading. All I'm saying is look at your S.M.A.R.T. data for your drives periodically and check the values. My personal opinion (just my opinion) is if the head load count is more than 12 in 24 hours then that is too high, and if the motor start count increases more than 5 in 24 hours, that too is too high. Honestly I prefer my drives to run continuously as typically these drives would remain in service longer.

So if you look at these values and you feel they are climbing faster than you would like, examine the settings you have set in the script because it appears that the script is well written and does what it claims, but you have to be smart enough to know how to setup the script to not cause more damage than the benefits gained.

 

[ngandrass]

ngandrass updated HDD Spindown Timer with a new update entry:

Successfully tested on FreeNAS-11.3-U3.2

The script was successfully tested on the latest FreeNAS-11.3-U3.2 release :)

Read the rest of this update entry...