SOLVED TrueNAS wakes up pool HDDs every day

[user353795]

Hello everyone,

I have setup my NAS to put pool disks to standby after 30 minutes without usage and it works. They will stay in standby unless there is an access to them. And I don't plan to access them more than once or twice a week.
However there is a problem that TrueNAS wakes them up every day at the exact time when it is doing some planned procedures (check for updates/backup/clean up - not sure). I can see that there is a few kB of write to those disks.

I have check that system dataset is on boot-pool, there are no installed jails nor plugins (there were but I have deleted them), there are no scrub tasks nor smart tests nor cron jobs that could wake the disks. I have tried turning off SMART service, but the issue persists. There are no hardware raid controllers, the disks are connected directly to the motherboard. I'm not sure if I missed something else but I don't believe so.

In TrueNAS Task Manager I can see that there are these methods running at the time pool disks wake up (see image below). And I believe that one of them is causing the issue.

However I wasn't able to find these methods/scripts in the system. I don't even see them as root in system's cron job.

Could anyone help me figure out what is waking up the disks?
Where I can find those methods from task manager, what exactly are they doing and why (if) are they writing to pool disks?
Any help would be appreciated.

Version of TrueNAS: TrueNAS-12.0-U5.1

PS: I have read a lot of different comments on these forums that putting disks to standby is bad, so please no need to bring that up again. However if you have a link to some professional tests that were performed on disks to figure out how much waking up disks every hour/day/week/month shortens their lifespan, I would love to read that.

 

[NugentS]

I think rather than point out that sleeping disks is bad - I would like to congratulate you on succeeding where lots of others have failed to get TN disks to go to sleep for more than a few minutes.

 

[user353795]

That's kind of sad, nevertheless thank you. It doesn't look like there is much hope.
Anyway I went through the source code and few ideas occured to me that I could still try. If I figure out something I will write it here.

 

[user353795]

Looks like I have managed to solve it. The disks have been asleep for 2 days now.
The issue is in automatic system backups and having pool encrypted with a key. I have changed the encryption to use passphrase and the disks don't wake up anymore.
I'm still going to monitor it closely, but I believe this question can be already mark as solved (I don't have permission for that).

 

[NugentS]

@user353795 do you mind documenting what you have done overall to achieve this?
What you have done to make the disks sleep - its the subject of several questions here
And how you fixed the wake up issue?

 

[user353795]

Not a problem, though I might have just been lucky. I haven't done anything special that wasn't written on these forums several times already.

Anyway here are the things that I have played with to get disks to sleep:

• Version of TrueNAS [me: TrueNAS-12.0-U5.1]
  • I have to mention this because it may be the reason why it works for me, however I cannot confirm this. Most of the complains I have seen are at some (older) versions of FreeNAS and not TrueNAS.
• Move System Dataset to different pool [me: boot-pool]
• Setup `HDD Standby` timer [me: 30 min] and check `Force HDD Standby` for all pool disks
  • I haven't set anything in disks' SMART options because it didn't look like it was doing anything at all.
• Setup SMART service to run with `Power Mode` set to `Standby`
  • If `Power Mode` is set to `Never` the service will never let disks sleep.
  • This however brings a problem that if there are any SMART tests configured, the service will never wake disks up to run those tests. Workaround for this is to have a script run as a cronjob before each test to wake disks up. If disks are awake the test will run normally, however if disks are asleep, then the test won't run and will get discarded.
• Restart SMART service
  • Changing any value in SMART service looks like does not to have any effect unless the service is turned off and on again (at least for me).
• Use `-n standby` in scripts when using `smartctl`
  • If you are using any scripts that are requesting data from disks using `smartctl` and you don't want to wake them up, you have to use option `-n standby`.
• Pool/dataset encryption [me: encrypted with passphrase]
  • TrueNAS peforms backup every day and if pool or dataset is encrypted with a key, TrueNAS will wake disks up when it is performing backup because it is backing up the encryption key as well. So either changing encryption to use passphrase or (not tried) not using encryption at all fixes this issue.
• Logging users in through web GUI or any other service
  • It may be good idea to point out that whenever user logs in through web GUI or SSH/SMB/etc (if any data need to be read from a pool), TrueNAS wakes disks up. So when testing if disks are asleep it is better not to have any users logged in anywhere.
• Restart NAS
  • If nothing works it may not be a bad idea to just restart the NAS. I did that few times because few things got stuck here and there when I was setting it up.

That's all I can think of.

Also here are few things about my setup that could have helped as well:

• No installed jail or plugins
• Only these services are running: SMART, SSH, SMB, UPS

And now I can only wish all of you good luck. :)

 

[NugentS]

@user353795 Thank you for that

 

[Evertb1]

However if you have a link to some professional tests that were performed on disks to figure out how much waking up disks every hour/day/week/month shortens their lifespan, I would love to read that.

I would to. The truth is that a lot of the information on this subject is anecdotal. I haven't read a conclusieve study yet. At this point I think that you should do what you think is best for you. I own two servers. My main file server is running 24/7. My lab server is shut down when I am not using it. Some of the drives in that lab server are older then 10 years and still going strong. When drives in that lab server start producing SMART errors they are out but it doesn't happen that often. However I realise that I use way to few drives to be representative in any way.