New NAS (Recommendations)

[michael230]

Hi FreeNas community,
I wanted to find out if I missed anything and if anyone has any recommendations for my new / current setup.

Below is my setup:
Case: Antec Twelve 1200 v3 12 bay
Motherboard: Asus P5G41T-M LX Plus
CPU: Quad CPU Q8300 @ 2.50GHz
RAM: Kingston HyperX KHX1600C9D3K2 (8GB)
Power Supply: Seasonic M12II 750W
Power Cables: NZXT premium 4pin molex to 4x sata (x 3)
SATA Cables: Mod Smart Serial ATA 90 degree (x 12)
SATA Adapters: FastTrack S150 TX4 (x 2)
Hard Drives: 12 x 2TB WD20EARX (WD Green) <--- I know not the best choice but I will eventually change them to RED if any fail.
USB: ADATA AS102-16G-RGY Pro
Raid Type: RAIDZ2
UPS: APC® Back-UPS Pro 1500 Battery Backup
Build: FreeNAS-8.2.0-RELEASE-p1-x64 (r11950)

I was wondering if I should bother with a cache or is RAIDZ2 enough to keep me out of trouble on hard drive failures etc.. any input would be greatly appreciated.

Sincerely,
Michael

 

[Stephens]

Make sure you look up the program to reset idle time to 300 seconds (5 minutes, the max setting) with those Green drives. Or just get the Samsung HD204UI, which you can probably get for the same price.

Strongly consider getting a UPS.

Did you verify the FastTrack TX4 is supported?

 

[survive]

Hi michael230,

Here are my thoughts:

I assume you already have a CPU and that's why you are going socket 775, if not get something more modern. Personally I don't like the board....the G41 is Intel's cut-rate platform and I don't like only 2 DIMM slots.

Get more RAM. 8GB is not enough for the size pool you want to do. The single best thing you can do with ZFS is give it a boatload of RAM.

Good call on the PSU, Seasonic is the Cadillac of power supplies. Same with the case....it's a bit to "pimp-my-ride" for my tastes, but there is certainly noting wrong with it. Load that sucker up with some nice Yate Loon 120mm fans. I like these guys for my fan\cooling supplies: http://www.jab-tech.com/120mm-Fans-c-81.html

Personally I would forget using the Fast Track controllers, I think they are, to be blunt, garbage...even if they were decent controllers they use the PCI bus, which is just a bad idea. Find yourself a nice PCI-e controller at least, with any luck you can get a single card that can do at least 8 SATA ports. I would find a deal on an IBM M1015 controller and connect up 8 drives to that, plus the SFF-8087 cables will make the case interior much cleaner than the rat's nest of 8 more individual SATA cables. Here's my go-to page for SAS\SATA HBA's, it's a bit dated but the HBA market isn't exactly fast moving: http://blog.zorinaq.com/?e=10

As for your pool layout....12 drives is rough, that's more drives than you should really put into a single pool, and I bet you really don't want to "burn" 4 drives to parity making 2 X 6 disk raidz2 pools. That said, if I was making a 12 disk filer 2 X 6 disk raidz2 vdevs in a single pool is exactly how I would do it.

I wouldn't buy the green drives. Given my choice I would stump up for the WD Reds at the onset. Google shopping shows that there is a ~$20 premium per drive....better to do it right from the get go, doing it wrong will cost you twice as much.

In closing, given you are spending $1500.00 on the project I would think real hard about some of the component choices and find a few more bucjks to properly "class it up" in a few areas. If you have a Microcenter local check them out as they have some excellent deals on CPU\motherboard packages....enough that going to an 1155 platform is a simple choice.

-Will

 

[michael230]

Make sure you look up the program to reset idle time to 300 seconds (5 minutes, the max setting) with those Green drives. Or just get the Samsung HD204UI, which you can probably get for the same price.

Strongly consider getting a UPS.

Did you verify the FastTrack TX4 is supported?

Hi Stephens, my reset idle time is set to "Always On" and I was thinking of buying a UPS any recommendations? Also I am not seeing anything about FastTrack but I'm using those cards as expanders and didn't even configure the hd's as JBOD or anything at all.

 

[cyberjock]

Also I am not seeing anything about FastTrack but I'm using those cards as expanders and didn't even configure the hd's as JBOD or anything at all.

Um. They aren't expanders. They are SATA RAID controllers. You are somehow very very confused.

 

[StephenFry]

> Hard Drives: 12 x 2TB WD20EARX (WD Green) <--- I know not the best choice but I will eventually change them to RED if any fail

They are actually quite brilliant and certainly an economical choice. I've built 3 FreeNAS servers this year, all with EARX drives. Don't forget to WDIDLE3 /S300 all of them, though.

 

[cyberjock]

> Hard Drives: 12 x 2TB WD20EARX (WD Green) <--- I know not the best choice but I will eventually change them to RED if any fail

They are actually quite brilliant and certainly an economical choice. I've built 3 FreeNAS servers this year, all with EARX drives. Don't forget to WDIDLE3 /S300 all of them, though.

I totally disagree with the bold. I've got a 24 drive server filled with WD20EARX and WD30EARX and I think they are an excellent choice. They may be slower than black drives, but they totally make up for it by quantity. Plus they are lower powered so they run cooler. The only "downside" is having to use the WDIDLE tool to change the sleep time(forget the exact term) to 5 mins from 8 seconds. Mine have been in my server 12-30 months and I've had zero failures with uptime of >99%.

 

[michael230]

Thanks again StephenFry and noobsauce80 for all your advise, currently in testing stages and will run the WDIDLE3 /S300 already found a LiveCD with WDIDLE http://www.jzab.de/content/wdidle-bootcd which I'll use to run on every disk.

BTW do you guys think I really need ECC as my motherboard doesn't support ECC. I've also bought a dedicated UPS "APC® Back-UPS Pro 1500 Battery Backup" for the NAS.

 

[michael230]

K guys I've ran the following:

====================================================
Step 1: Download ISO: http://www.readynas.com/contributed/CommanderQ/FreeDOS WDIDLE3.iso
Step 2: Use this tool and 'burn' the ISO to a USB drive: http://unetbootin.sourceforge.net/
Step 3: Make sure you have all the HDDs ready
Step 4: Unplug your existing PC HDDs, plug in as many WD Greens as you can into the PC
Step 5: Boot, go into BIOS and set it to boot off the USB
Step 6: Boot from USB, select BootCD (option 2)
Step 7: Type WDIDLE3 /R
It will tell you that the IDLE is set to 8seconds, default
Step 8: Type WDIDLE3 /S300
This sets the idle to 5mins (300seconds).
Step 9: Type WDIDLE3 /R (again, to make sure its changed to 300 seconds from 8 seconds)
================================================

Now all the drives are at 300 seconds

 

[StephenFry]

Sounds good, michael230! I did the WD drives one-by-one; were you able to verify this method indeed applies the setting to all drives?

btw, I've never used ECC memory and can't be bothered to :)

 

[michael230]

StephenFry correct I ran wdidle3 /r confirmed all drives were set to 8 seconds and then ran wdidle3 /s300 and confirmed all drives were set to 300 seconds and it did it in 1 shot

 

[oxyris]

Make sure you look up the program to reset idle time to 300 seconds (5 minutes, the max setting) with those Green drives. Or just get the Samsung HD204UI, which you can probably get for the same price.

Could you elaborate on why this is necessary?

 

[Joshua Parker Ruehlig]

Could you elaborate on why this is necessary?

spinups are very stressful for a drive, much more than just constantly running. So spinning down and spinning up a drive just because it hasn't been accessed for a few seconds is very costly to the life of the drive compared to the possible small energy savings of a few minutes with an idle drive.

I believe the Samsungs are like any other consumer drive and don't spin down unless told too, while the WD Greens have firmware that force them to spin down every few seconds if not accessed. I believe the default time is 9 seconds with a max of 300, though I don't own any WD myself.

 

[oxyris]

spinups are very stressful for a drive, much more than just constantly running. So spinning down and spinning up a drive just because it hasn't been accessed for a few seconds is very costly to the life of the drive compared to the possible small energy savings of a few minutes with an idle drive.

I believe the Samsungs are like any other consumer drive and don't spin down unless told too, while the WD Greens have firmware that force them to spin down every few seconds if not accessed. I believe the default time is 9 seconds with a max of 300, though I don't own any WD myself.

Thanks for your reply. Where can I find this particular setting in the web interface?

 

[Stephens]

It's not in the GUI. http://support.wdc.com/product/download.asp?groupid=609&sid=113

But I didn't do it that way. I used Ultimate Boot CD from a Flash drive (since I had no windows or DOS on my FreeNAS machine). I used a WD drive because I had it (unopened) and decided to give it a go since I was short 1 HD204UI (my preference at the time). I ended up just keeping it as that's an archive (not backup, archive) machine. It's rarely ever on, but on the day I use it, I don't want the driver powering up and down every few seconds.

 

[StephenFry]

while the WD Greens have firmware that force them to spin down every few seconds if not accessed.

This is not the issue at hand. The WD Green drives -fortunately- don't spin down THAT often! What they do, however, is park the drive heads very very quickly after being used. I think it's not 9, but 8 seconds, btw ;)

This will lead to an extremely high "Load Cycle Count", which you can read out from the drive's SMART data. With 10,000 cycles possible in a day, that adds up. The problem is that the drives are only rated for about 300,000 cycles. They really shouldn't have a problem reaching several times that, but why take the risk?

It's a strange design from WD.

Setting it to "never" can lead to strange problems. The maximum manual setting is 300s, which makes the daily maximum less than 290 cycles. Now that is completely acceptable, especially since you will likely only park a few dozen times after setting the timer to 300.

 

[Joshua Parker Ruehlig]

Ahh, thanks for the clarification. Guess it's not as bad as constantly having to wait for your drives to spinup again

 

[oxyris]

It's not in the GUI. http://support.wdc.com/product/download.asp?groupid=609&sid=113

But I didn't do it that way. I used Ultimate Boot CD from a Flash drive (since I had no windows or DOS on my FreeNAS machine). I used a WD drive because I had it (unopened) and decided to give it a go since I was short 1 HD204UI (my preference at the time). I ended up just keeping it as that's an archive (not backup, archive) machine. It's rarely ever on, but on the day I use it, I don't want the driver powering up and down every few seconds.

Thanks. Isn't this thread talking about the WD Green drives though? http://support.wdc.com/product/download.asp?groupid=608&lang=en There seems to be no equivalent utility for the Green drives. Is this setting not necessary for them?

Also, aren't there any official firmware upgrades for the WD Green drives?

 

[cyberjock]

The WDIDLE does NOT change the spindown time. WDIDLE changes the time to park the heads. Heads used to park in the park zone on the platter on power down. Since the disk head rides on a cushion of air during normal operation, during powerdown the head physically touches the media. WD added what they called "intellipark" technology to have the heads park off the media itself. Look at the following picture:

If you look at the little orange "thingy" below and to the right of the media, the head slides up on that little "shelf" when Intellipark is triggered. This, in theory, extends the life of the drive by preventing head damage and platter damage from the 2 touching one another. It also decreases power consumption because there is less drag on the air from the head, which also causes lower temperatures.

Of course, there's a drawback. The little shelf and the arm wear out from physical contact over time. WD SMART sets the "threshold" at about 300,000 cycles. If you manage to accumulate 300k cycles your SMART tests will start warning you that the drive will fail "imminently". Some people have hit that limit in just less than a month, others take years(if ever). Some have reported over 1 million cycles without problems.

The default for all of the drives I am familiar with is 8 seconds. For typical household use the technology is pretty smart. You're typically accessing the drive, or not. You open up a word document and the hard drive reads the program and the file, then sits idle until you do something that requires a hard drive read(which is usually alot more than 8 seconds). When a read or write is needed the head is placed back on the platter and the data is read.

For RAID arrays, this can be very bad. Consider an array of 6 disks in RAIDZ2. If you are streaming a movie the data may be striped over the array(RAID1, Z1, Z2, Z3 all stripe the data). As you watch the movie the data is read sequentially from disk 1, then disk 2, and so on. Theoretically you may read data from disk 1, then progressively through each disk. By the time you get back to disk 1 those 8 seconds have elapsed and the head was parked. The disk must now return the head to the media. The data is read. Then disk 2 is read, but it is also parked. The head for disk 2 must return to the media. So on and so forth.

As you can see, you begin cycling all the drives on your array. If you assume 10 seconds between reads on any given disk, a 2 hour movie would result in 720 cycles of every disk in your pool. There have been people that have claimed to have more than 50k cycles in just a few days. In just a few months of streaming movies you can rack up a significant number of load cycles.

Setting the disks to something much higher, like 300 seconds is good because loads like streaming movies will almost certainly cause load in less than 300 seconds. But if you leave the house for 8+ hours then heads will park after those 300 seconds and your disks will consume less power and run cooler.

I have heard from people in other forums that have hit the threshold for load count try to RMA the disk. Allegedly WD has denied warranty claims for JUST threshold numbers because in a desktop environment it is virtually impossible for you to rack up those 300k counts in a reasonable period of time. It's a dead giveaway that you are running the drive in an enviroment "not approved for the drive" you purchased.
But in a server environment it's very easy to hit that threshold without good planning.

Hopefully I've cleared up the situation.

Just did some quick math: If you assume 10 seconds between reads on a disk streaming 24x7, you will hit 300,000 cycles in less than 21 days.

 

[Stephens]

Thanks. Isn't this thread talking about the WD Green drives though? http://support.wdc.com/product/download.asp?groupid=608&lang=en There seems to be no equivalent utility for the Green drives. Is this setting not necessary for them?

First of all, power down does have a specific meaning when it comes to these drives, so I probably didn't use the most accurate term when describing what the drives do. However, WD itself says in addition to parking the heads, it "turns off certain drive electronics." Anyway, the correct term is head parking. That said... yes, the program is still needed on WD Green drives. But I was as confused as you while reading the web, so I decided to let reality be my guide. Run "smartctl -a" against the drives and see if your load_cycle_count balloons. Mine was. So I used WDIDLE3 and it stopped.