ATA Idle & WD RED Drives

[clownphish]

Hi all,

I'm having some issues with drive spin-down that I'm trying to troubleshoot. First off, does anyone know if the WD RED drives work with the ataidle spin down timer (or has had it working)?

I have the "HDD Standby" set to 30 for all 4 of my drives, however the WD Red drives never seem to spin down on their own (according to the value of smart attribute 4 - Start_Stop_Count as reported by smartctl -a /dev/adaX). If I issue the immediate command (-s) they report that they are spun down.

All other drives in my system (one WD Black drive, one Seagate Barracuda and a 2.5" WD Scorpio Blue) spin down automatically after the timer expires. The Seagate is the only one with APM enabled, FWIW. I've read here about head parking on these drives using a DOS utility but that is different to it spinning down (and not recommended for NASs apparently)

I've read a lot of similar threads about this issue but can't see anyone with exactly the same issue, as my settings appear to be working for the other drives. Does anyone have any idea why these drives don't spin down like my others?

My other related question is, why do some drives report that they are asleep and others active? If I attempt to access data on the NAS all drives in the pool spin up (ada0-3) - see below. Is it possible for any data to be read from the drive while some disks are spinning or is the data always evenly distributed across the drives?

My RAID setup is a RAIDZ1 comprising of 4 2TB disks (ada0-3) in one pool. All have "-m root" as an extra SMART option, a SMART check interval of 120, Power mode 'Standby - Check the device unless it is in SLEEP or STANDBY mode', Offline Immediate Test every 2 hours, Conveyance Self-Test every 4 hours and the other two at much longer intervals. Motherboard is an ASUS E45M1-I DELUXE with 8GB of memory. FreeNAS version 8.3.0

Some debug output below. All scripts have come from other posts on the forum but pretty much are just checking whether the 'spinning value' is FF or 00. I don't have a Kill-a-watt or something similar to measure power usage

[root@axiom] ~# ./lstemp.sh

System Temperatures - Mon Mar 11 21:45:11 EDT 2013
FreeNAS-8.3.0-RELEASE-p1-x64 (r12825)

System Load: 0.34, 0.12, 0.08

CPU Temperature:

Drive Activity Status
ada0:FF ada1:00 ada2:FF ada3:FF ada4:FF

HDD Temperature:
ada0 40 WD-WMAY02684900 WDC
ada1 N/A - Drive in standby mode
ada2 36 WD-WMC300105434 WDC
ada3 36 WD-WMC300446835 WDC
ada4 31 WD-WXL1E32FNWMY WDC

[root@axiom] ~# ./spintest.sh
time: 21:45:05
ada0: SPINNING
ada1: IDLE
ada2: SPINNING
ada3: SPINNING
ada4: SPINNING

After accessing a network share:

[root@axiom] ~# ./lstemp.sh

System Temperatures - Mon Mar 11 22:07:29 EDT 2013
FreeNAS-8.3.0-RELEASE-p1-x64 (r12825)

System Load: 0.02, 0.10, 0.09

CPU Temperature:

Drive Activity Status
ada0:FF ada1:FF ada2:FF ada3:FF ada4:FF

HDD Temperature:
ada0 38 WD-WMAY02684900 WDC
ada1 29 W1E0F8E9 ST2000DM001-9YN164
ada2 34 WD-WMC300105434 WDC
ada3 34 WD-WMC300446835 WDC
ada4 30 WD-WXL1E32FNWMY WDC

Spin-up counter. ada2 and ada3 are the WD Red drives.

[root@axiom] ~# smartctl -a /dev/ada0 | grep "Start_Stop_Count"
4 Start_Stop_Count 0x0032 098 098 000 Old_age Always - 2828
[root@axiom] ~# smartctl -a /dev/ada1 | grep "Start_Stop_Count"
^[[A^[[D 4 Start_Stop_Count 0x0032 098 098 020 Old_age Always - 2860
[root@axiom] ~# smartctl -a /dev/ada2 | grep "Start_Stop_Count"
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 15
[root@axiom] ~# smartctl -a /dev/ada3 | grep "Start_Stop_Count"
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 16
[root@axiom] ~# smartctl -a /dev/ada4 | grep "Start_Stop_Count"
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 301
[root@axiom] ~#

Thanks for your time.

 

[cyberjock]

All drives in the zpool must be spinning because checksums are ALWAYS used when any data is read, and that is striped in all setups except single-disk single vdev. That is how ZFS works.

As for the WD Red I wouldn't be surprised if they couldn't sleep. As has been noticed by many people in the forums, putting drives to sleep and waking them up seems to put wear and tear on them. Nobody really understands why, but people who have had drives spinning 24x7 for years notice better hard drive life than drive that are cycled. Since those drives are targeted at small business NASes having them sleep excessively(some people do 10 minutes or less) can wear the drives down quickly. Assuming that sleeping and waking(or power cycling) your hard drives does cause excessive wear and tear if I were WD I might disable them just to minimize the number of RMAs WD would get.

 

[joeschmuck]

As my friend stated, wear and tear on starting these drives up (not just the WD Red, all drives) can lead to an early demise. The WD Red series has a 3 year warranty and they were designed to run 24/7. Unfortunately I shut down my NAS last night because I needed a drive power adapter to fix my fathers corrupt hard drive he shipped me so I was not able to test out the sleep capability. I'm pretty sure I posted if they slept or not after I got them, should be here somewhere in the forums including what SMART reported for them because I know one feature doesn't exist, I think ACC. That is not to say APM works with FreeBSD or not, I just don't recall.

So do another search, pay attention to specifically WD Red posting which are evaluating them, not just complaining of a problem. That type of posting will likely give you better results in this case.

Happy hunting.

 

[Alchete]

I have WD Red HDs - 3TB. Mine spin down with the following settings:

Advanced Power Management: FALSE
HDD Standby: 20 minutes

Plus ... you need to reboot the server when changing these settings.

Good luck.

 

[SkyMonkey]

I have 6 2TB Reds. They spin down fine with the settings that Alchete has posted above (my standby time is set to 60, but I've tried 5 as well and it works). A reboot is required for the setting to take effect. They are actually spinning down, I can feel the vibrations (or lack thereof) by touching a drive, and I can hear them spin back up when I go to access data or change certain settings in the GUI.

One thing of note that seems potentially inconsistent with other drives based on my reading is that they do NOT spin up on a smartctl query unless a specific test is requested. The drive reports that it is idle and is indeed spun down after a smartctl -i -H -A -l error is issued.

 

[Dusan]

I had the same problem. My setup: 5 WD REDs set to spin down after 60 minutes, smart checking every 30 minutes, power mode standby. The disks were not spinning down, however after I spun them down via smatctl -s, they would stay in standby. One of the tests I did was to set smart checking to 15 minutes, and the disk standby timers to 13, 14, 15, 16, 17 minutes (each disk had a different standby setting). The two disks with 13 and 14 minute standby timers would spin down, the 15 minutes one would also spin down, but after much longer time and the other two would never spin down. This seems to indicate that smartd with -n standby will not spin up a disk that is in standby, however the periodic checking will reset the drive's (WD RED) internal standby timer. I then set the drives to spin down after 60 minutes and the smart checking to happen every 61 minutes and the drives do spin down when the NAS is not accessed.

 

[clownphish]

That's interesting Dusan. Thanks also to the others who replied to the thread.

When you say 'smart checking' do you mean the various tests you can configure under 'SMART Tests' or the basic 'check interval' under 'SMART' (for temperature)? My check interval is set to 30 (the default) currently. I also have Offline and Conveyance test set to go off after 10 & 11 hours.

 

[Dusan]

Yes, "smart checking" = check interval under SMART. I also specified '-a' as 'SMART extra options' on the 'View Disks' screen so smartd is checking not just the temperatures, but also the SMART attributes & the results of SMART tests (I run the short test daily at 1am and the long test monthly).
Another thing I just discovered, and was actually also reported by SkyMonkey above, is that running smartctl -x on a standby WD RED drive will retrieve all the SMART data without spinning up the disk. Therefore, I changed the smartd power policy to never and the disks still spin down and stay in standby while smartd continues to poll the SMART attributes every 61 minutes.

 

[Dusan]

Hmm, in your first post you mention that you run the offline test every 2 hours. That's way too frequent. smartctl -c on my 3TB WD REDs indicates that the offline test takes more than 11 hours to complete (the number of seconds is actually slightly different for each of my 5 drives):

Total time to complete Offline
data collection: (40320) seconds.

The drive will never enter standby while it's running a test. Even if you run the test just once a day, the disk will not spin down for the next 11 hours (the test may finish faster on your 2TB drives though).

FYI: the durations of the other tests as reported by smartctl -c:

Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 404) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.

 

[joeschmuck]

Which version of FreeNAS are you guys using?

 

[Dusan]

FreeNAS-9.1.0-RELEASE-x64 (dff7d13)

 

[joeschmuck]

So after reading the posts again, is anyone stating there is a problem? It doesn't really read like there is a problem. If someone thinks there is a problem, list the details and I'll look into it but it might take some time since I'm just about to go on vacation , but there are others who can look into as well.

 

[nigma]

I have just put together a new system:


HP Proliant N54L

4x 3TB WD Red (ZRaid2) as a single pool

Two datasets : AFP for time-machine backup and the rest is for a single NFS share.

I am having difficulty in getting the drives to go into standby. By setting the HD Standby option in the WebUI the drives do not at all power down – I’m using the camcontrol command to poll power state of the drives

However, when using ataidle –S 120 /dev/ada0 the drive will immediately power down and then within 30seconds to 2mins (indeterministic amount of time) the drive will power back up again and not ever go back into standby.

I also am using “zpool iostat 2 500” to tell me if there is any activity in the pool. There seems to be a tiny amount of activity just as the drive wakes up, but am unable to track where it is coming from.

I see two possible scenarios here:

a) The HD standby timer is somehow being ignored and not going into standby

b) The “activity” is stopping the hard drive from powering down again once it has woken up from forced standby.

Any help with the above will be much appreciated.

Thanks

 

[mjws00]

Syslog and samba write to the pool in 9.2.1.7. This is a known behavior with a few dozen threads. Nothing at all to do with red drives. Either consider it 'by design', use a second pool for logs, find another hack.

 

[Wim Mistiaen]

Thanks dusan for pointing out the smart test resetting the timer. that explains why my 5 minute timeout test was working, but my drives never got to standby if set to 60 minutes :)

could've been looking for ages to find that :D

 

[ric]

I have the same situation like op as my HDDs never spin down no matter how I would choose the settings.

4x WD NAS RED HDD (RAID 5)

 

[cyberjock]

I have the same situation like op as my HDDs never spin down no matter how I would choose the settings.

4x WD NAS RED HDD (RAID 5)

Well, if you are doing RAID5 then that would explain the spindown. Hardware RAID isn't supported, and the HDD spindown would be based on the settings of the RAID controller.

 

[ric]

I configured my HDDs in the BIOS setting as AHCI mode so that Freenas able to see all my HDDs. and on Freenas under Volume Manager I then configured all HDDs as RAID 5, created pool, dataset and so on.

I went to the BIOS setting>RAID controller but no such an option found that relate to HDD spindown.

 

[cyberjock]

RAID5 is hardware RAID.

There is no RAID5 for ZFS. There is something that is somewhat equivalent called RAIDZ1.

 

[ric]

RAID5 (Hardware) same as RaidZ I think. So my current configuration right now is RaidZ > ada0, 1, ,2, 3.