prevent frequent reads from waking up HDDs

[derWalter]

Hey guys, I cant find out what is keeping my disks awake ... exact every 5min I have a read of 335,77 and every 30 min a read of 461,68

(even with SMART interval 1440 seconds, or disabled)




Goal is to have them sleep as long as possible, as they seldom get used, sometimes not even for weeks.
The frequent used data is stored on SSD mirror, those HDDs are only archive.
As its a home NAS which idles around 5-7Watts, two running HDDs push it to 17-20Watts,
so IU prefer to keep it as low as possible. The whole box is also passive cooled and
housed in my living room, so noise is also a reason to keep those HDDs sleeping.

 

[e7balt]

I'm also seeing a similar pattern but have not been able to determine the cause either.

 

[sretalla]

Where is your system dataset?

 

[LarsR]

Scale places the System Dataset on the main Pool, you can move it to your boot drive (not recommended).

 

[derWalter]

Hey guys, sorry for replying so late, but I didnt get any notifications...

My main pool should be ob my SSD mirror, i ve also got constant activity there, so I think that's not it (I also run my apps from there and the ssd mirror was built first)

Does anyone of the devs/admins/genius-geeks have any hint for us small SCALE users?

 

[mroptman]

@sretalla is correct. You should verify where your system dataset is and move it to a non-HDD pool (if you intend to spin down your HDD pool).

In /ui/storage -> Top right gear -> System Dataset Pool

Select a non HDD pool and then verify your HDDs spindown properly.

 

[ornias]

Yeah "should be" isn't something we can be sure of tbh.
"Assumptions are the mother of all f'ups", basically

 

[derWalter]

I just checked and can confirm that I remembered it correctly, system dataset pool was and is on LIVE (ssd mirror) not on ARCHIVE (hdd mirror), hdds dont spin down (or constantly spin up) due to these strange random reads

 

[derWalter]

so, I ran "dstat -tdD /dev/sdc --top-io" and can confirm that it is at least "asyncio_loop", which triggers a read of 4096B exactly every 5 minutes.


how can this be stopped without breaking anything?
would be nice to see some kind of energy saving features in 21.06,
that would end all that hacking permanently

 

[ornias]

how can this be stopped without breaking anything?
would be nice to see some kind of energy saving features in 21.06,
that would end all that hacking permanently

21.06 is in codefreeze, if they are not in (which they are not afaik) they are not going to be there ;-)

Did you make a Jira ticket for your issues anyway? Because that is how the devs work: JIRA -> Code, not Forum -> Code (usually)

 

[derWalter]

well ... u got me there ... off to jira I go again (DANG is jira slow...)

 

[derWalter]

till now there was no reaction to my jira ticket, is there a way to get it some attention?
as jira was so slow, I've managed to create two redundant tickets, sry :S

https://jira.ixsystems.com/browse/NAS-110854?workflowName=Suggestion+Workflow&stepId=7

https://jira.ixsystems.com/browse/NAS-110855?workflowName=Suggestion+Workflow&stepId=7

u can take the better one and close/delete the other one...


I guess for a dev its a real easy and quick task (as on debian everything is ready) and
most likely the administration of the work will take longer then the work itself :P

 

[Pas05]

Hi, so i noticed this problem already months, ago, i solved it by adding -n standby,10 into the S.M.A.R.T. extra options field on the disk settings page. This will make the smart demon skip 10 checks when the HDD is in standby.
I am also using a version of the freenas-spindown-timer script that i adapted for truenas scale since it can happen that the drives still won't spin down. With this solution my drives are always spun down and are only spinning when a smart test, scrub, or normal disk activity is going on.

 

[derWalter]

Hey @Pas05 thanks for that info, actually I dont have any smart test scheduled, but would you mind to share your adapted version of the script pls?

 

[Pas05]

I don't mind sharing it, but i would keep in mind that the reads every 5 minutes are most likely related to smart activity (polling temperature would be my guess) and setting the -n standby, 10 prevents that from happening and waking up the drives as far as i know.
here is the script i use. Please note that i am not a professional programmer and therefore there is most likely a more elegant way of doing things since i only cared about getting it to work and nothing else xD
I am corrently using this script in a cronjob that runs every 30 minutes
/mnt/POOLNAME/path/to/Idle_c_Debian.sh -m -p 20 -t 30 -i sda -i sdb -i sdc -i sdd
i removed the main loop since the cronjob is doing the looping for me.

 

[derWalter]

Hi, so i noticed this problem already months, ago, i solved it by adding -n standby,10 into the S.M.A.R.T. extra options field on the disk settings page. This will make the smart demon skip 10 checks when the HDD is in standby.
I am also using a version of the freenas-spindown-timer script that i adapted for truenas scale since it can happen that the drives still won't spin down. With this solution my drives are always spun down and are only spinning when a smart test, scrub, or normal disk activity is going on.

I don't know if I just managed to NOT SEE THE EXTRA OPTIONS including power options, or I just forgott about it, or they where just recently added with 21.06 *facepalm* but okay, I am a newbie with limited time resources here :P But thanks for pointing that out, I activated "standby: 10; force HDD standby: true; advanced power management: level 1; S.M.A.R.T. extra options: -n standby,288 (as SMARTs polling frequency is five minutes, this lets it check once a day, at least I hope it works that way)"

I don't mind sharing it, but i would keep in mind that the reads every 5 minutes are most likely related to smart activity (polling temperature would be my guess) and setting the -n standby, 10 prevents that from happening and waking up the drives as far as i know.
here is the script i use. Please note that i am not a professional programmer and therefore there is most likely a more elegant way of doing things since i only cared about getting it to work and nothing else xD
I am corrently using this script in a cronjob that runs every 30 minutes
/mnt/POOLNAME/path/to/Idle_c_Debian.sh -m -p 20 -t 30 -i sda -i sdb -i sdc -i sdd
i removed the main loop since the cronjob is doing the looping for me.

thanks a lot for that post as well, but it doesn't seem to be working for me at all, so I have to investigate further... I tried disabling the SMART service, but I still get my "asyncio_loop" read every 5 minutes...

cant use it like that... :/

 

[derWalter]

I tried resetting the installation and disabling everything I could find to disable, just leaving the disk sleep parameter and the script with crone job intact, still, a read every five minutes, dang it :/

ps.: it seems that one can post bug reports from inside truenas in jira, but not in jira itself... will try that next.

 

[Pas05]

What does the output of sudo systemctl status smartmontools say?

 

[derWalter]

Thanks for showing interest @Pas05

output:

smartmontools.service - Self Monitoring and Reporting Technology (SMART) Daemon
Loaded: loaded (/lib/systemd/system/smartmontools.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:smartd(8)
man:smartd.conf(5)


I also just disabled SMART for the disks themself, their settings now read:

Disk Type:
HDD
Description:

Model:
TOSHIBA_MG07ACA14TE
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
5
Adv. Power Management:
1
Enable S.M.A.R.T.:
false
S.M.A.R.T. extra options:
-n standby,288

 

[derWalter]

I guess I've found the culprit here...

As I said I disabled SMART directly in the extra Disk options (Storage -> Disks -> sdd ->...) and suddenly it worked.
To find out if thats it, I reenabled SMART on only ONE of the two disks, resulting in this:

nas02# hdparm -C /dev/sdc && hdparm -C /dev/sdd
/dev/sdc:
drive state is: idle
/dev/sdd:
drive state is: idle


after a few minutes:


nas02# hdparm -C /dev/sdc && hdparm -C /dev/sdd
/dev/sdc:
drive state is: idle
/dev/sdd:
drive state is: standby


And in the io graph under Reporting I see reads on the one Disk with SMART enabled and a NULL line on the Disk with SMART disabled.


After disabling SMART for both disks:

nas02# hdparm -C /dev/sdc && hdparm -C /dev/sdd
/dev/sdc:
drive state is: standby
/dev/sdd:
drive state is: standby


For now thats working for me, but it would be nice for the future, to have SMART running while keeping the disks in standby/sleep/spindown.

They stay spun down, even with freenas-spindown-timer.sh cronjob disabled.
But as I remember, the reason for the script is to have the disks spin down, while retaining SMART functionality, soooo the script is actually not working for me.


BUT the drive associations are changing between boot, the HDDs may be like sdc sdd, but after the next reboot, they may be sdc sde.... so I cant start the script automatically with the right drive associations, that may contribute to the problem. Does it also work with UID?