"Force HDD Standby" has no effect on smartctl.conf

[blemasle]

Hi,

On a fresh install of TrueNAS-12.0-U2.1, I cannot get my pool drives to stay down when not in use, and I think the culprit resides in smartd service that performs temperature read every 5 minutes. Let me explain :)

Despite using "Force HDD Standby" set on every one of my pool drives, /usr/local/etc/smartd.conf still use -n never.

/dev/ada1 -a -d removable -n never -W 0,0,0 -m root -M exec /usr/local/libexec/smart_alert.py\
-s S/../../(6)/(00)\

/dev/ada2 -a -d removable -n never -W 0,0,0 -m root -M exec /usr/local/libexec/smart_alert.py\
-s S/../../(6)/(00)\

...

The code that generates the smartd.conf file still seems to use the old global smartd value. That value cannot be configured in the UI anymore and has been moved in each drive configuration if I understand it right.

According to the smartctl(8) man page :

-n POWERMODE
[ATA only] Specifies if smartctl should exit before performing any checks when the device is in a low-power mode. It may be used to prevent a disk from being spun-up by smartctl. [...]

never - check the device always, but print the power mode if '-i' is specified.

sleep - check the device unless it is in SLEEP mode.

standby - check the device unless it is in SLEEP or STANDBY mode. In these modes most disks are not spinning, so if you want to prevent a disk from spinning up, this is probably what you want.

Doing some manual tests, I can indeed perform a S.M.A.R.T. read without spinning up the drive if I use smartcl -a -d removable -n standby while using smartcl -a -d removable -n never does spin up the drives.

On my system, the spun down drives wake up on a 5 minutes mark (0h05, 0h10, 0h15 etc), which match the S.M.A.R.T. temperature read.

Questions :

1. Is this intended or is this a bug ?
2. How can I let my drive spun down even while doing S.M.A.R.T. temperature reads using TrueNAS ? Before you ask : Yes, my system dataset is not on that pool, and I'm not using them when they spin up again :)

Thanks

 

[Alex728]

OK. by the way, if you set the additional parameter -n standby in the disk settings, then the disks will not spin up from the planned smart test (30 minutes by default). in /usr/local/etc/smartd.conf will appear the -n standby parameter (/ dev / ada1 -a -d removable -n never -W 0,0,0 -m root -M exec / usr / local / libexec / smart_alert .py -n standby) but has no effect on unwinding every 5 minutes. I reported for statistics. sorry for wasting time. I do not understand English.

 

[blemasle]

When I do so, it does add -n standby to /usr/local/etc/smartd.conf indeed. But it does so after the one TrueNAS mistakenly put as -n never which I believe cancels the effect.

No need to be sorry. You haven't wasted any time to anybody :)

 

[Alex728]

I believe that these settings only apply to the smart autotest. and the temperature review ignores this. perhaps there is a config for the temperature.

 

[blemasle]

You're probably right. I didn't realize that. Nonetheless, as a user, I expect this checkbox to act in every situation, whether it'll be the regular smart test check or a temperature reading. In any case, I certainly do not expect that a temperature check will spin up my drive every time it spins downs. This is absurd...

 

[blemasle]

Actually, it seems that the value set on the global page is still used during individual drive temperature reading

See here for individual disk temperature read.
The powermode itself seems to comes from the global smart config, see distktemp.py (not a per drive config).

 

[blemasle]

For whoever might be interested, and if you face the same issue : SMART reads will wake up your drive every 5 minutes so the "Force HDD Standby" is pretty useless on its own. For HDD standby To work correctly, it must be set along with the SMART powermode set to STANDBY.

Sadly, the UI for editing that setting has been removed. If like me, you need this for your drive to stay asleep, use the script below. It uses the TrueNAS API to edit the incriminated setting without the need of the UI.

Modify parameters to fits your needs / current configuration, but leave powermode on STANDBY.

Code:

function uuidv4() {
  return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
    var r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
    return v.toString(16);
  });
}

var ws = new WebSocket('ws://truenas.local/websocket');
ws.send({
    "msg": "connect",
    "version" : "1",
    "support" : [1]
});

ws.send(JSON.stringify({
      "id": uuidv4(),
      "msg": "method",
      "method": "auth.login",
      "params": ["root", prompt("Password ?")]
}));

ws.send(JSON.stringify({"id": uuidv4(),"msg":"method","method":"smart.update","params":[{"interval":30, "powermode": "STANDBY", "difference":0,"informational":0,"critical":0}]}));

PS : a prompt will appear asking you for your password. This is required to use the API.

 

[blemasle]

After several weeks, I can now report that the fix works as expected and my drives are entering sleep as they should.
This is clearly a bug in the TrueNAS smart configuration that I reported here. Hopefully it'll be fixed soon. In the meantime, you can use the above workaround.

 

[Hello_World]

The bad thing is that now if "HDD Standby" is enabled in TrueNAS, the temperature monitoring of the hard disk will be automatically turned off and the temperature of the hard disk will not be recorded in the system, unless the data is directly read from the command line using "smartctl".

 

[Hello_World]

https://github.com/truenas/document.../SCALE/SCALEUIReference/Storage/DisksSCALE.md “Temperature monitoring is disabled for standby disks. " The current solution seems to be very bad...

 

[chruk]

I have an interesting observation.

My two WD Red drives will accept SMART read requests whilst in standby, they do "not" spinup. This is logical to me, as the info is read from the chip on the drive, not from the platters.
However my two Seagate Ironwolf drives are not able to serve SMART read requests without spinning up, this seems a flawed design.

So when I originally enabled this on sleeping disks, I seen my two WD drives drop in temperature and that data could still be collected whilst the two Seagate drives span back up again.

For this reason I think there needs to be a per disk configuration variable here for the temperature data collection, and also would like the frequency to be customizable as well.

Finally of interest, SMART read requests will also take Seagate drives out of Idle_b mode, I have nearly 80k load cycle count on drives that are barely 9 months old. By default they will go in rest state after 2 mins, but then 3 mins later they moved again on a temperature reading. This happens on out of the box TrueNAS and drive configuration. I recently adjusted the behaviour to prevent this using the seagate 'SeaChest_PowerControl_x86_64-portbld-freebsd12.0' tool. More info on this thread.

Exos x16 - High head cycle count due to toggle between EPC idle_a and idle_b power states

I'm currently trying to build a new Storage Server based on TrueNas and having some trouble because the "load_cycle_count" of the drives gets increased every 3 minutes, which would lead to a load_cycle_count of 480 per day or 175200 per year. I don't think this is intended behaviour. But first...

www.truenas.com

I did google to see if the spin up on SMART can be changed as the seachest tools allow a lot of tinkering but all I could find was people asking how to stop the disks spinning down rather than how to stop them spinning up on SMART read requests. For this reason I probably wont buy Seagate again.

 

[Protopia]

I agree that temperature monitoring of disks are less relevant when they are in standby, however that doesn't mean that temperature readings are not relevant when they are not in standby.

But (obviously) you do not want temperature readings when the drives are spinning to prevent it going into standby. I am unclear whether standby is a hardware initiated event or a Truenas-software initiated event, but if (as I think) it is the latter then it should be possible to avoid including these I/Os in the counts that decide whether or not to spindown.

Excessive temperatures are important for two reasons:

1. A major cause of drive failure is due to overheating;
2. Drive overheating can be indicative of issues with the platter bearings which can also lead to drive failures.

So tracking temperatures is pretty darned important, and to avoid doing this for drives that are capable of standby even when the drives are spinning is IMO a very foolhardy approach.

If disks that are spundown will stay spundown if you check the temperature: then IMO it still makes sense to record the temperature even when they are spun down. Although the motor is powered off and consequently not adding heat to the drive temperature, there still may be an ambient temperature issue, and recording the temperature after it has spun down to know that it is cooling off and how fast is potentially useful information.

Obviously, if disks that are spundown won't stay spundown if you check the temperature: then you shouldn't check the temperature when they are actually spundown.

But when disks are spinning, then you should IMO definitely record the temperatures.

 

[Stolar]

Is there solution for that?
I have the same problem. When "HDD Standby" is enabled, temperature reading is not working (see screenshot). This is OK, but when disks are spinning again, temperature reads are still not working anyway.
Funny thing is that if I check "Force HDD Standby" option, disks will automatically spin-up around 1-5min after they spin-down. But temperature reading works. This makes no sence for me... Shouldn't this work other way around?