BUILD Need critique of a 32 drive preliminary build

[soulbabel]

I'm currently exploring a system to be used for storing transcoded video from a video capturing system. Other than the three machines that will be routinely dumping their processed video to the FreeNAS server (~18 hours/day), there will only be the occasional file access to pull data from the server. The transcoded video files are lightly compressed (~18 Mbps) x264 files at 1080p/29.97.

I was considering the following preliminary build:
Supermicro CSE-847A-R1400LPB (36 bay hot swap)
2x LSI Logic LSI00244 SAS 9201-16i
32x WD40EFRX (Probably split into 8-drive RAIDZ2 to maximize space vs moving to a 36 drive 6-disk RAIDZ2)
SUPERMICRO X9SRL-F
E5-2609 v2
ECC RAM (Amount TBD per forum advice)

I'd like to find out if people think I'm on the right track with what I have so far, or have any recommendations or suggested improvements. I chose the WD REDs because I was able to get some experience with them during my first home FreeNAS build, and I really like how cool they run, and how stable my system is after I root out the bad drives in the first few months. I'm still a FreeNAS and storage novice, so I know there's always the possibility I'm setting myself up for failure, so I'm open to any and all criticism. Thanks in advance.

 

[jgreco]

Ah, um, very tired right now but wouldn't the CSE-847A-R1400LPB be the non-SAS-expander chassis that would then require 9 SF8087's for attach?

You could probably get a bit cheaper going with the 847E16 and a single -8i SAS HBA. The downside would be limited bandwidth to the backplanes but with a 2609 and your described usage you're looking more at an archival machine anyways. Or possibly better yet an X9SRH-7TF which gives you both the HBA onboard plus 10GbE future-proofing.

For an archival storage server you could try getting away with 64GB (4 x 16GB) and then if you had performance problems buy the other 64GB that we would normally suggest.

The original E5-2609 was a contemptible sucky CPU and I don't expect its newer version to be all that much better. I can't imagine it being a problem for your described situation but I still don't like it.

 

[cyberjock]

Actually, I'd probably do 10-disk RAIDZ2s. That's an optimal config for a RAIDZ2 vdev and gives you a little more storage. Just make sure you got a sound schedule for scrubs, SMART testing, and SMART monitoring setup.

 

[jgreco]

Or for the ultimate in reliability you could do 11 disk RAIDz3's but then you'd need one more disk (and you should buy a few spares so that you can command replacements without disruption).

 

[soulbabel]

Awesome advice guys, plenty of food for thought, thanks so much for the help.

I'm curious about the SAS expander. Initially I was leaning towards an expander based solution, but I came across various pages describing potential hazards when using SATA drives on SAS expanders. I did get kinda spooked, but I also saw some threads on these forums describing successful non-problematic experiences with the SATA drives.

From what I remember, the gist was that the problems would crop up mainly due to drives going bad while attached to the expanders. What I like about FreeNAS though, is that it seems really easy to catch a drive going bad early. So are these problems kept at bay due to the application of proper FreeNAS maintenance habits, or are the dangers overblown? If a non-expander approach is taken, would the extra money spent at least yield some additional risk management?

 

[jgreco]

Please see http://forums.freenas.org/index.php...ardware-suggestions.12276/page-14#post-110425

The non-expander approach is "safer", yes. If you absolutely must have pool responsiveness under all circumstances, do not use SATA, use SAS. On one end of the spectrum you have the dual controller, dual ported SAS drives with an E26 backplane so that there are redundancies everywhere. In the middle you have SATA with dedicated HBA ports. On the cheap side you have SATA on an E16 backplane and a single controller. Basically you can run the $$$ numbers and sketch the topology to understand the ins and outs; I know for a fact that the SATA+expander route has been problematic for some specific models of drives and firmware, but the added cost as you move up the reliability ladder may not be justifiable for many applications.

 

[JerryS]

Hi,

What would be the favorite choice for processor and motherboard for a system like this? I am building something similar, have most of the other parts selected based on info from this site. I found a deal on 3TB SAS disks and will have 2 pools, 6x2 RAID 10 and 4+2 RAIDZ2. I am expecting moderate traffic and want low latency for both reads and writes. Just one controller.

thanks in advance
jerry