If have say 6 drives configured as 3 independent mirrored pairs attached to an LSI 9211-8i, and they're all configured for HDD Standby after XX minutes, can I put my frequently accessed files on one and will the other 4 drives remain in standby while accessing the "frequent" volume? Or will FreeNAS always spin up all 6 drives regardless of which volume is being accessed?
-
Important Announcement for the TrueNAS Community.
The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums
Do Mirrored Drive Pairs Spin Up/Down Independently?
- Thread starter DevGuy
- Start date
- Status
- Joined
- May 28, 2011
- Messages
- 10,994
I can't answer this from the perspective of the LSI controller however from the hard drive perspective, when the hard drive sleeps it's normally because you have pushed to the drive a command to sleep after XX period of inactivity. The hard drives will spin up if they need to access the data on that drive or drive set. My best guess is only the drives containing the data would spin up but with the LSI controller, I'm not sure if that would make a difference,m or even if the drives would sleep in the first place. I wish I had an LSI controller, along with some other goodies.
Of course you need to evaluate if you should sleep any of the drives at all from a wear and tear perspective. Lets say you do create a system with three mirrored pairs and three separate pools, you may desire to leave the pool frequently accessed running all the time while if you only access the other two pools once a night, you could let those sleep. There are a lot of folks with different thoughts on this so you will have to figure out what your requirements are. My drives have been running 24/7 for over 3 years now and no failures.
Of course you need to evaluate if you should sleep any of the drives at all from a wear and tear perspective. Lets say you do create a system with three mirrored pairs and three separate pools, you may desire to leave the pool frequently accessed running all the time while if you only access the other two pools once a night, you could let those sleep. There are a lot of folks with different thoughts on this so you will have to figure out what your requirements are. My drives have been running 24/7 for over 3 years now and no failures.
Thanks Joe. I agree if drives are frequently accessed there can be an argument for leaving them spinning 7/24--that's especially true if they're at least "NAS grade" or, even better, enterprise-grade drives and you don't care about power consumption/heat/noise/etc. For a lot of home use, however, I think a strong case can be made for letting the drives spin down if they're likely to remain that way for hours at at time.
To provide a little more background as to my logic:
The drive reliability studies, done by Google and others, show a 3 year old 7/24 drive has (if I recall correctly) a 20% chance of failing in the next year. And, especially if they're fairly large drives and fairly full, it's frighteningly likely you may be unable to rebuild a single parity (i.e. RAID 5, etc.) striped array due to unrecoverable read errors and/or another drive failing from the "abuse" of the rebuild and being equally old and likely to fail (and often manufactured at the same time and prone to very similar failure modes). Using 2 parity drives (i.e. RAID 6, etc.) helps but doesn't really solve the problem.
Generally it's the mechanical moving bits in drives that cause most failures. And letting drives spin down and sleep can save a significant amount of wear and tear on the moving parts. Those 3 years of 7/24 is about 26K hours of wear and tear on the mechanical parts. If the drives only are spun up 4 hours a day that's only about 4K hours of wear and tear. That's a huge difference.
Working against you, there is some added thermal cycling from having the drives sleep and it's hard to know the impact that might have (it would partly depend on how hot your drives get when they're active--the cooler they run the less thermal cycling is an issue).
At least with independent mirrored pairs, there's no complex array to rebuild. And, worst case, if you have both drives in a pair fail before you can re-mirror the array, you only risk the data on that pair. Even better, because the data isn't striped, you can likely recovery nearly all of it as long as at least one of the drives is still accessible even if it has some bad blocks, etc. It's extremely unlikely both drives will fail in a way that doesn't allow you to recover all, or nearly all, the data on the pair. That's in sharp contrast to parity-based arrays where the chances of total data loss are much higher.
For home applications where usage patterns can be very different (say movies only being accessed in the evening for example), and noise, heat, and power consumption are often issues, it makes a lot of sense to use low power quiet drives (which are generally not enterprise grade and even WD Red drives are really just consumer-grade WD Green drives with different firmware and marketing). And those low power quiet drives are most likely to benefit from being spun down most of the time.
Finally, with a low power motherboard, the power consumption of 6 drives spinning but idle will often exceed the entire rest of the system. You can build a nice performing NAS with a motherboard that idles at around 10 - 15 watts. But say 6 WD Red drives will consume 10-15 watts by themselves doubling the idle power consumption of your system. By letting them sleep, you cut the power consumption roughly in half.
To provide a little more background as to my logic:
The drive reliability studies, done by Google and others, show a 3 year old 7/24 drive has (if I recall correctly) a 20% chance of failing in the next year. And, especially if they're fairly large drives and fairly full, it's frighteningly likely you may be unable to rebuild a single parity (i.e. RAID 5, etc.) striped array due to unrecoverable read errors and/or another drive failing from the "abuse" of the rebuild and being equally old and likely to fail (and often manufactured at the same time and prone to very similar failure modes). Using 2 parity drives (i.e. RAID 6, etc.) helps but doesn't really solve the problem.
Generally it's the mechanical moving bits in drives that cause most failures. And letting drives spin down and sleep can save a significant amount of wear and tear on the moving parts. Those 3 years of 7/24 is about 26K hours of wear and tear on the mechanical parts. If the drives only are spun up 4 hours a day that's only about 4K hours of wear and tear. That's a huge difference.
Working against you, there is some added thermal cycling from having the drives sleep and it's hard to know the impact that might have (it would partly depend on how hot your drives get when they're active--the cooler they run the less thermal cycling is an issue).
At least with independent mirrored pairs, there's no complex array to rebuild. And, worst case, if you have both drives in a pair fail before you can re-mirror the array, you only risk the data on that pair. Even better, because the data isn't striped, you can likely recovery nearly all of it as long as at least one of the drives is still accessible even if it has some bad blocks, etc. It's extremely unlikely both drives will fail in a way that doesn't allow you to recover all, or nearly all, the data on the pair. That's in sharp contrast to parity-based arrays where the chances of total data loss are much higher.
For home applications where usage patterns can be very different (say movies only being accessed in the evening for example), and noise, heat, and power consumption are often issues, it makes a lot of sense to use low power quiet drives (which are generally not enterprise grade and even WD Red drives are really just consumer-grade WD Green drives with different firmware and marketing). And those low power quiet drives are most likely to benefit from being spun down most of the time.
Finally, with a low power motherboard, the power consumption of 6 drives spinning but idle will often exceed the entire rest of the system. You can build a nice performing NAS with a motherboard that idles at around 10 - 15 watts. But say 6 WD Red drives will consume 10-15 watts by themselves doubling the idle power consumption of your system. By letting them sleep, you cut the power consumption roughly in half.
depasseg
FreeNAS Replicant
- Joined
- Sep 16, 2014
- Messages
- 2,874
Can you clarify "independent"? Do you mean 3 different pools? Or 3 different vdevs in the same pool? If they are different pools then the 4 drives will spin down. If they are in the same pool, then there isn't a way to restrict the writing to just 2 drives and the other 4 won't spin down separately.
solarisguy
Guru
- Joined
- Apr 4, 2014
- Messages
- 1,125
Hello [B]DevGuy[/B], I am interested! Could you please share your designs?
I was just this weekend asked for help in building a low-power FreeNAS server. Twice! However, I do not have any expertise in low power area. All I could come up with was to use WD10JFCX (1TB WD Red 2.5") drives. As they consume only 0.6W when idle.
Thank you in advance!
- Joined
- May 28, 2011
- Messages
- 10,994
With respect to if you should or shouldn't sleep the drives, we have documented it so many times on this forum about to good and bad aspects of sleeping your drives, I don't want to rehash it yet again and it will come down to what you feel is right for your situation. Heck, we all do what we think is best for our situation, that's why I have an AMD system, not an Intel system (yet).
Thanks for the reply and asking for clarification. I'm just getting started looking into FreeNAS but I don't see any obvious problems with using 3 independent ZFS pools for my application. Obviously having one single big array of all 6 drives would have some advantages for flexibility and space efficiency, but I'm gladly willing to trade those for the benefits of simple mirroring.
So, unless there's a downside to independent pools I'm not yet aware of, that's good news!
For drives you're on the right track with 2.5" drives. The WD Red 2.5" drives are the only reasonably priced supposedly "NAS-grade" 2.5" drives I'm aware of. There are some expensive enterprise 2.5" drives designed mainly for performance. And there are lots of standard consumer grade 2.5" drives including the 2 TB Samsungs which are really made by Seagate from what I can tell, very reasonably priced (< $100), and low power. You get double the capacity of the WD Red drives at roughly the same power consumption--so you need half as many and save even more power. That's what I'm going with (ultimately up to 8 of them) and can let everyone know how they work out.
Don't feel too bad about using consumer-grade drives. The whole "consumer NAS" drive category seems to mostly be a marketing effort. The drive companies have gotten together with the NAS manufactures and managed to mostly only get their NAS and enterprise drives on the certified hardware lists. And they charge significantly more for drives that are identical to their consumer drives except for the label, firmware, and sometimes warranty. You get the same platters, same heads, same spindle motor, etc. So unless your NAS requires the "NAS" drive firmware, or you care about the warranty, a WD Green is the same drive as a WD Red. My previous low power NAS systems have only used 2 or 4 drives and did not use FreeNAS.
As for the rest, it depends how much you want to spend, if you want ECC memory support, if it's purely going to run FreeNAS, many plugins (especially video transcoding), or will run some other operating system or hypervisor with FreeNAS as a virtual machine, etc. Supermicro, ASRock, Jetway, and others, make motherboards with 4 to 12 SATA ports and SoC-style processors that are nicely low power. Most Atom/SoC/embedded processors, however, don't support VT-d (not even Intel's fairly expensive C2000 family does). But FreeNAS by itself doesn't need VT-d unless you run it as a VM.
If you're only going to use a single gigabit LAN connection for your build, and 5400 or 5900 rpm low power drives, those are far more likely to be the bottle neck for pure NAS performance rather than the CPU in your system. If you want to do processor intensive things, like transcode HD video with Plex, then you need to plan accordingly. And, sometimes it makes more sense to encode your video collection in a format that's natively supported by your primary playback devices, or be limited to one stream at a time, rather than have a big high powered server solely to transcode multiple HD video streams.
Another option is an E3-xxxx-L series Xeon processor which are very power efficient. The Intel Xeon D series are even more impressive but expensive. There are some new low power "T-series" Intel Skylake CPUs like the 35 Watt i3-6100T that supports ECC memory and should have very low idle power (< 2.5 watts). They can be used in any Skylake socket 1151 motherboard (although you might have to wait before many support ECC RAM--likewise despite being "available in Q3" the T-series Skylakes are still not generally available yet). Even the "old" Haswell socket 1150 "T" processors like the i3-4130T are very power efficient (and can handle most reasonable transcoding tasks).
My latest build is a Supermicro X10SLL-F socket 1150 motherboard with a Xeon E3-1230L (a 25 watt CPU) and 6 SATA ports. The board, with a single SSD attached, and a Supermicro Gold power supply, idles around 15 watts which is really impressive considering the E3-1230L is a quad core Haswell with a Passmark rating of nearly 8000. Under full load (same config) it's still only at 35 watts which makes cooling quiet and easy. I went with a Xeon as I'm most likely going to run FreeNAS under VMWare ESXi. But, for most FreeNAS builds, it's overkill.
But, now that I've rambled on, this probably belongs in at least another thread and probably in another section of this forum.
solarisguy
Guru
- Joined
- Apr 4, 2014
- Messages
- 1,125
Thank you! (I will start another thread, but one more question...:D) How did you buy your Xeon E3-1230L ? I have looked around, and clearly buying any of E3-1220L v3, E3-1230L v3, or E3-1240L v3 was beyond my capabilities, today...
P.S.
I think I would advise those who asked me, to go for the FreeNAS Mini by iXsystems, as they were enticed by ZFS, however they wanted an appliance (not a server) with low power and low noise (50dB is not that quiet, but let's hope that with lower power hard drives everything would be OK).
P.S.
I think I would advise those who asked me, to go for the FreeNAS Mini by iXsystems, as they were enticed by ZFS, however they wanted an appliance (not a server) with low power and low noise (50dB is not that quiet, but let's hope that with lower power hard drives everything would be OK).
I got my E3-1230L from http://macsales.com (aka OWC). You can also find the Xeon L CPUs on eBay and available as a configuration option in some pre-built servers. If someone just needs a pure NAS, you don't need a Xeon. Something like this will work really well: http://www.asrockrack.com/general/productdetail.asp?Model=AD2550R/U3S3#Specifications
I only know the FreeNAS Mini is about twice the price of similar units from Synology, Qnap, Drobo, etc. which was enough to make me cross it off the list. I know that's not going to win me any points with the powers at be behind FreeNAS but it's just reality. And FYI, fan noise is often the main noise culprit in many commercial NAS products--even with the fans at their lowest speed. The manufactures don't know how they're going to be used and tend to error on the side of keeping them cool to reduce warranty claims, returns, etc. they also tend to spec fans that are not designed to be quiet. That's not to say you couldn't likely modify a commercial NAS to be quieter but you may void the warranty doing so.
depasseg
FreeNAS Replicant
- Joined
- Sep 16, 2014
- Messages
- 2,874
I'm still not sure you are talking about pools or vdevs. You can have 3 pairs of mirrored drives (3 vdevs) in a single pool. Or you can have 1 pair of mirrored drives (1 vdev) in each of 3 pools. If the former, then not, they won't spin down independently, if the latter and you write to only 1 pool, then yes, the other 4 drives will spin down independently.
I tried to make it clear I'm okay with 3 pools as long as there are no trade-offs that might be a problem with that approach? It's good news FreeNAS will let other pools remain in standby while only one pool is being accessed.
depasseg
FreeNAS Replicant
- Joined
- Sep 16, 2014
- Messages
- 2,874
As long as you don't need more than 1 disks worth of storage space and IOPS per pool, then this is fine. You know you can't aggregate storage space across pools, right? They are each handled as separate entities. I don't mean to sound difficult, but I've seen folks who use terms without realizing what exactly they were getting into.
Thanks. I'm fine with that. I know it's not the most efficient way to allocate drive space, but I'd rather have 3+ separate mirrored independent volumes/shares than deal with the trade-offs of one large array. My main goal is to get enough redundant storage in one relatively low power server. A single 6 drive NAS with 3 pools is far more energy efficient than 3 two drive NAS systems.
One other thing to be aware of is the swap space. The GUI defaults to adding a 2 GB per disk swap partition. If your system ever needs to use that swap space, all 6 disks will spin up.
If you know one pool will be busier than the rest, you may want to set a larger swap partition on those two drives and set the rest to 0. I think you may need to hit the advanced options button when you're creating your pool to see that option.
If you know one pool will be busier than the rest, you may want to set a larger swap partition on those two drives and set the rest to 0. I think you may need to hit the advanced options button when you're creating your pool to see that option.
- Status
