prevent frequent reads from waking up HDDs

[Pas05]

Hi, so i am glad you figured it out.

i am using the script because setting the APM is not reliable enough and the drive doesn't go to standby by itself for 90% of the time, even tho its supposed to do it. The script itself sets the disks to standby when there hasen't been activity during a set amount of time.
My settings are the following:

HDD Standby: 10
Force HDD Standby: is disabled (its disabled because the freenas-spindown timer is basically doing the same stuff and it also accounts for ongoing smart tests and doesnt interrupt smart tests by putting the drive in standby
Advanced Power Management : Level 127 - Maximum power usage with Standby

Enable S.M.A.R.T: enabled
S.M.A.R.T. extra options: -n standby,10

The smart service itself is also enabled and i put the powermode to Never

i dont fully understand all this smart magic but as far as i know the drive itself still saves all the smart data and by using -n standby,10 smart still polls the SMART data every 10th run of smartd (smartd seems to run always every 30 minutes for me, doesnt matter what i set in the smartd service settings so i am not sure what the Check Interval option does behind the scenes). I am also still running smart tests and scrubs regulary. I personally think that this is a good tradeoff and for me the noise reduction is more important than checking that smart data constantly.

i think my settings should work fine for you aswell.

But to be honest i dont really get why you needed to disable the SMART fest or all your drives indidivually even tho you fully disabled the service, this might be some sort of bug

 

[derWalter]

i think my settings should work fine for you aswell.

But to be honest i dont really get why you needed to disable the SMART fest or all your drives indidivually even tho you fully disabled the service, this might be some sort of bug

while reading your post, I also got the impression of doing a little bit more tuning to find the breaking point for me.

thanks for pointing out the fact with the smart service, after playing around a little bit more, I will post a bug report for it.

 

[derWalter]

Update: I ran into a problem where I had a constant max readout by k3s-server on my archive pool.
somehow it seemed to be related to this bug: https://www.truenas.com/community/t...d-connected-to-the-network.94352/#post-653172

since it got fixed, I dont get the 200mbs reads by k3s-server anymore.

 

[joeschmuck]

@derWalter I'm just asking a crazy question here but in an ideal world for you, how long would you drives sleep and then how often would the drives wake up to do some work? In other words, how many times per day would the drives spin up and be active for the purpose of the NAS you created? Also, is the NAS just for storage only, no jails or other features?

I'm asking because I see that you are trying to sleep 14TB drives which are fairly heavy drives (9 platters), meaning the platters would require a lot of power to change it from sitting still into a rotating mass. History has proven that these frequent current surges can cause premature failure in the drive electronics.

Maybe you are desiring to spin down the drives because they generate too much noise at night (a frequent issue of course) and i fully understand it which is why I placed my NAS into a place where noise will not bother me or the family, but not everyone can do that.

Anyway, just consider how many times you are spinning up those heavy rotating masses over time and the cost of those drives. They will generally last significantly longer if you just leave them spinning 100% of the time.

But I do understand that sometimes a silent system in the middle of the night is desirable so good luck in finding a solution.

 

[derWalter]

@joeschmuck This is a real Archive, so they sleep for days, weeks and months on some times. I have a 1tb ssd mirror which handles my documents and everything else I would use more regularly. My whole system uses like 3-7watts, so I can have it running 24/7, 10-15watts constantly for nothing AND the noise in my office would be just a waste for nothing :)

 

[karolyi]

I think your best bet is what I did: I installed TrueNAS on an SSD with a partition that is kept for frequent writes (jails and stuff that accesses the FS), and the HDDs are really only used for storing files that are accessed rarely. They wake up now once daily, at midnight when cronjobs do some maintenance and wake them up, or when I explicitly access files on them that aren't already in the ZFS cache, or need writes. atime is turned off on them so file listing or getting contents of a file that's already in the memory doesn't wake them either. Considering that my HDDs have a load/unload cycle of 600.000, I think I'm good for a long time. :)

Unless I'll need huge storage in one of my jails, I'll keep using the SSD for stuff that accesses disks more often, and the HDDs will stay for storing long term data.

 

[ChrisRJ]

Here in Europe the electricity bill is the primary reason that people want to save as much energy possible. Running a NAS 24/7 that consumes about 100W will cost you almost 300 Euros per year.

 

[joeschmuck]

@joeschmuck This is a real Archive, so they sleep for days, weeks and months on some times. I have a 1tb ssd mirror which handles my documents and everything else I would use more regularly. My whole system uses like 3-7watts, so I can have it running 24/7, 10-15watts constantly for nothing AND the noise in my office would be just a waste for nothing :)

3-7 Watts? Sounds a bit low but that is not the question. The point you are making is understood, you rarely use the spinning rust drives and it's only for archived data and could be down for weeks at a time.

My best advice is to use TrueNAS Core vice TrueNAS SCALE (beta) and set your system up with that. For that matter, even FreeNAS 11.3 would work great (many folks still run this version, it's stable), then you should have no issues sleeping those drives.

 

[derWalter]

3-7 Watts? Sounds a bit low but that is not the question. The point you are making is understood, you rarely use the spinning rust drives and it's only for archived data and could be down for weeks at a time.

My best advice is to use TrueNAS Core vice TrueNAS SCALE (beta) and set your system up with that. For that matter, even FreeNAS 11.3 would work great (many folks still run this version, it's stable), then you should have no issues sleeping those drives.

I built it to be energy efficient and I am quite happy that I ve achieved it. I may be swapping my pentium gold for an i3 somewhen, but for now its enough.

thx for the advice, even though you are right, I came to TrueNas because of SCALE and sticking with it since its first release.

 

[joeschmuck]

I built it to be energy efficient and I am quite happy that I ve achieved it.

Okay, if SCALE is where you desire to be then you may need to live with some of the beta issues but that is a good thing in my mind, you are helping the developers diagnose early issues. When I was involved with FreeNAS 8.0 through 9.3 I compiled a lot of code, I actually built my new computer so I could crunch all those numbers and troubleshoot, make a code change, rebuild, test, and repeat. I had the time back then to do it, I don't these days, too much work (that pays the bills) is going on. So it's admirable that you are doing beta testing.

This is getting a little off topic now but could you post your system specs? (Nice to add to your profile) I'm 100% certain others would like to build a low power system and having the parts listed will help quite a bit, include the motherboard, RAM, power supply, number and size of fans, add-on cards, and all the drives, even case. I too like using SSD's, they make no noise and use very little energy, my VMware server has a few for my VM's. Also, how did you measure the current draw? I'm sure you do have some energy efficiencies going on but if you are measuring the current draw incorrectly, we can help you measure it properly, and unfortunately maybe make you a little sad at the same time. Off the cuff I'm thinking you may have measured Amps vice Watts and maybe I'm wrong in my assumption, but Amps multiplied by Voltage = Watts. So this means I'd also need to know your voltage where you measured. Think of it as a learning experience for me, you, both? And please do not take offense, that is not my purpose. I'm huge at trying to learn more and teach as well.

 

[derWalter]

this post is totally offtopic!

Okay, if SCALE is where you desire to be then you may need to live with some of the beta issues but that is a good thing in my mind, you are helping the developers diagnose early issues. When I was involved with FreeNAS 8.0 through 9.3 I compiled a lot of code, I actually built my new computer so I could crunch all those numbers and troubleshoot, make a code change, rebuild, test, and repeat. I had the time back then to do it, I don't these days, too much work (that pays the bills) is going on. So it's admirable that you are doing beta testing.

This is getting a little off topic now but could you post your system specs? (Nice to add to your profile) I'm 100% certain others would like to build a low power system and having the parts listed will help quite a bit, include the motherboard, RAM, power supply, number and size of fans, add-on cards, and all the drives, even case. I too like using SSD's, they make no noise and use very little energy, my VMware server has a few for my VM's. Also, how did you measure the current draw? I'm sure you do have some energy efficiencies going on but if you are measuring the current draw incorrectly, we can help you measure it properly, and unfortunately maybe make you a little sad at the same time. Off the cuff I'm thinking you may have measured Amps vice Watts and maybe I'm wrong in my assumption, but Amps multiplied by Voltage = Watts. So this means I'd also need to know your voltage where you measured. Think of it as a learning experience for me, you, both? And please do not take offense, that is not my purpose. I'm huge at trying to learn more and teach as well.

yeah, I already spent some hours with teamviewer and IX devs ;)

I put this into my profile now:

Disks:

2* Toshiba Enterprise Capacity MG07ACA 14TB, 512e, SATA 6Gb/s (MG07ACA14TE) ​

2* Patriot Burst 960GB, SATA (PBU960GS25SSDR) ​

1* Patriot Burst 480GB, SATA (PBU480GS25SSDR) ​

CPU:

Intel Pentium Gold G5420, 2C/4T, 3.80GHz, boxed (BX80684G5420) ​

​

CPU Cooler:

ARCTIC Alpine 12 Passive 8​

RAM:

Samsung DIMM 16GB, DDR4-2666, CL19-19-19, ECC (M391A2K43BB1-CTD) ​

Mainboard:

Fujitsu D3644-B (S26361-F5110-V162) ​

PSU:

picuPSU120 28 + Meanwell GST90 28​

Idle wattage:

with HDDS powered, but all three SDDs ON: 6-7 watts​

with all disks (including HDDs) ON: 15-20 watts​

​

max powerdraw <90watts​

​

​

​

​

​

I used this german website for getting data for my build:​

NAS Advanced 3.0 - NAS mit Intel C246 Serverchipsatz und 6x SATA

Unsere neue NAS Advanced 3.0 Zusammenstellung löst das eigentlich noch aktuelle NAS-Advanced 2.0 ab, welches mit einer AMD Ryzen 3 2200G APU ausgestattet ist..

www.elefacts.de

​

Most important is the mainboard, use only one RAM slot, use SDDs which draw very little power (look up power draw benchmark websites for that) and use a dc dc (like PICO PSU) power supply.​

​

I didn't even touch the ram and cpu voltage, I could definitely undervolt the system and save some watts there, but I am happy with it.​

I guess I'll swap the CPU for an i3 some time, to have more power reserve, but I dont need it atm and I guess I would have to install a cpu​

cooler fan. I also used a VERY!! small case for that build, its around the size of an XBOX ;)​

case: https://www.inter-tech.de/en/products/case/desktops/it-607-opsu​

​

and front panel not connected, as it also draws unused power, if I need to connect an usb stick, I just use the back ports of the case​

 

[Glowtape]

Hmm, so that's what it is. It's noticeable on my power meter by a few watts.

What's more interesting is that since rebuilding my NAS, I've been watching my load cycle count in SMART, and if I divide it by the power on hours, I get a value of 11.4. If you account for a bit disk usage (so far it idles a lot), you get to 12, which is every five minutes. It seems if this is SMART related, it seems to cause head (un-)parking.

 

[derWalter]

Hmm, so that's what it is. It's noticeable on my power meter by a few watts.

What's more interesting is that since rebuilding my NAS, I've been watching my load cycle count in SMART, and if I divide it by the power on hours, I get a value of 11.4. If you account for a bit disk usage (so far it idles a lot), you get to 12, which is every five minutes. It seems if this is SMART related, it seems to cause head (un-)parking.

sadly it looks exactly like this. the only "solution" I have found so far is: disabling smart on the disks I want to shut down. which is a very unpleasant solution.

 

[joeschmuck]

You can keep SMART enabled and drop the head loading count to zero, well you "should" be able to, hopefully. If you have definitely moved your your dataset to a SSD, and you are not running any memory crazy software that would force the hard drive cache into use (your cache is automatically created unless you override it during creation, or anything else which would poll the pool (like an outside data request), then the drives can sleep. The extra SMART parameter to use should be "-n standby", not "-n standby,10", I mean, why would you want the drives to see if there is a test is pending every 10 minutes? You could change it to 30 minutes if you like but if your goal is to sleep the drives. What you do not want is for the drives to spin down and to spin up frequently or you are asking for failure. Check the spinup cycle count, if it's increasing then you have a big problem. If you can't figure it out then you are better off disabling the sleeping and leave your drives spinning all the time.

Also, you should not need to manually set this up, you should be able to set this up in the Services GUI for SMART, select Standby and Interval check of 0. Save and see if it works. Your drives should remain powered off until the system needs data from the drives for some reason. If the drives do not stay down, disconnect the Ethernet cable for 24 hours, verify the drives do not spin up again, if they do not then you can state the issue is from outside source requesting some sort of status of the NAS. if the drives keep cycling then you need to figure out what other operations are happening with the NAS. Properly configured SMART "should" work fine but not all drives adhere to the SMART standard so well.

As for power consumption, you should use a meter to verify it but I would state that your system does not draw much power compared to a higher power motherboard. It's nice to have a passive cooler, good work on your project.

I hope you enjoy your system.

 

[Glowtape]

I have two Ironwolf 10TB from Seagate in the NAS, and the mere temperature read-out causes the parked heads to unload, remain on the platter for 2-3 minutes (or whatever the timeout is) and then park again until the next temperature readout.

Before I disabled SMART on the two drives: https://i.imgur.com/xdKnLwp.png
After I disabled SMART on the two drives: https://i.imgur.com/Y7n5xTL.png

When I cause the disks to intermittently come out of idle due to IO activity: https://i.imgur.com/GK5yFGh.png

 

[Pas05]

i figured the script i posted here months ago didnt do what it was supposed to do and put drives in standby when they were in access, i think i fixed that issue now and it should only set the drive in standby mode when its idle and no smart test is running.
i use it via a chron task (system settings > advanced) to run the command every 30 minutes
command: /mnt/SSDPool/tank/Idle_c_Debian.sh -m -p 60 -i sda -i sdb -i sdc -i sdd
This will check for io for 60 seconds on the disks sda, sdb, sdc and sdd and put the drives in standby if there is also no smart test going on.
I hope this is helpful for some people :)

 

[ngandrass]

Hey folks,

I finally found time to update the spindown-timer script. Check out the latest release v2.0.1 - I successfully tested it today on TrueNAS SCALE :)

See: https://github.com/ngandrass/truenas-spindown-timer/releases/tag/v2.0.1

 

[derWalter]

uuuuuuuh :D

nice thanks, will be nice to have SMART back in business :P

 

[Hafnernuss]

For what it's worth, I experimented a lot with this stuff in the last weeks and wanted to share a few insights...
First, the reasoning.. currently, the kWh here (Austria) costs ~55ct, meaning that my NAS with all hdds running (8x12TB w/ i5-4k) consumes around 90W, or in other words, a little over 2.1kWh per day, which would run me up to almost 400€ per year.

My drives are Seagate Exos, and apparently they come with Seagate PowerChoice, which is of course configurable. But here's a pitfall: Exos drives have their STANDBY timers disabled per default. But it can be enabled, with hdparm if you are using SATA plates, or sdparm if you are using SAS plates like me. This also means, that I unfortunately cannot use the script, but it's not necessary anyway, because the drives handle that themselves.
If someones interested, I can post the exact commands here that I used, but it's not that hard do figure it out by simply listing all available pages and change some bits here and there (This might help as well).

Specifically for the Exos drives, you can find information here: SAS models and SATA models.
Looking at some tables, you can see what the individual power states do. I did not know that even IDLE_B unloads the heads, and this power state is entered after a few seconds of inactivity.

This reduced the power consumption (with all drives spun down) to around ~50W, so almost half of what it consumed before.

Since apps cannot be run on spun down drives (duh!), I threw in an old 1TB SSD as pure application storage (no redundancy, if the SSD fails and I lose my PiHole config... so what?), I only make a weekly snapshot of this drive onto the main HDD pool.

And for "frequently used files" I bought myself 3x2TB SSDs in Raid Z1 (~110€ each), which should amortize themself by the savings in less than two years. Those SSDs will be snapshotted onto the main pool weekly as well.


Pitfalls that I encountered:

• System dataset not selected for "logs" etc.
• Apps/k8s pool on the HDDs (constant mini writes!)

 

[joeschmuck]

This is from the user manual in one of your links, it states that the manufacture default is enabled for the SATA drives but the text in the SAS drive document does not state the same thing so I'm thinking that maybe you have SAS drives? But you would know better than I would since I do not have any of those drives. I like the Standby_Y setting, it saves some power and the drive stays spinning, but that is just me.

I have always liked the idea of using SSD's for your normal data usage but unfortunately I doubt this would work for most users here. You pay a lot for electricity so I can see how this would be a huge benefit to you. If i could replace all my hard drives with some long lasting SSD's and afford it, I would. Dang, now I'm going to go look at SSD's. My HDD's are 5 years of age, 2 years beyond the warranty. I need to start entertaining the thought of replacing my hard drives.