First Build... 24 Drives... I'm Lost :\

[Thadius Miller]

Hey guys. Coming from a Syno DS414 running 4x3tb in SHR1 and an attached RAID controller with 8x4tb in RAID 50. Pretty terrible setup, I know. I'm running out of space and eating up all my memory, to boot.

I've read quite a few builds over the last few days, read the ZFS guide a couple times (thanks, Cyberjock!), chatted with buddies, etc. But I'm still supremely lost in the swarm of options out there compared to what I need.

Looking to build primary for a home media server. The capacity to transcode a few 720p streams in case I decided to later would be a plus. I'd be looking to run at least a few extra: couchpotato, sickrage, rtorrent (well-loaded), sabnzbd, etc.

I'm thinking I need:

Single Xeon E3 1231 or 1225
32gb ECC
3x M1015
3x 8-drive RAIDZ2 or 2x 11-drive RAIDZ2 w/ 2 cold spare (WD Reds -- already own 12x 3tb)
1200VA UPS
As quiet as possible while still offering good cooling

Mobo: Should I be looking at Supermicro X9 series? If so, which?
Chassis: Which Supermicro should I be pricing against? Trying to price against the Norco RPC-2442. If used, what features do I absolutely need to have?

Many thanks in advance. This will be a very fun first build! :D

Edit: Duals NICs would be nice. My DS414 is already teamed on a SRW2048 and it would allow me to move the initial data faster over while keeping the existing NAS serving.

 

[danb35]

SuperMicro X10SL7-F, a reverse SAS breakout cable, and this chassis from eBay should do the trick. No need for any of the M1015s. Don't know how quiet it will be--rack chassis aren't known for being quiet, but a 4U model should have larger and thus quieter fans than my 1U and 2U ones.

 

[Ericloewe]

32GB might be limiting for such a large system. Consider a Haswell-EP system, with an LGA 2011-3 motherboard. A nice matching processor would be a Xeon E5-1650 v3.

If you're ok with sticking to 32GB, check out my Supermicro X10 FAQ - it has a link to guide for choosing the best one for your needs.

 

[Thadius Miller]

SuperMicro X10SL7-F, a reverse SAS breakout cable, and this chassis from eBay should do the trick. No need for any of the M1015s. Don't know how quiet it will be--rack chassis aren't known for being quiet, but a 4U model should have larger and thus quieter fans than my 1U and 2U ones.

The price point on the rig you just laid out is drool-worthy. Thanks for this!

I'm having a hard time pinpointing exactly which cable I need to get from the SATA-style SAS connectors on the X10SL7-F. Can you link one?

How would performance compare if I dropped down to a X10SLL+-F (-$82) and tossed in an IBM M1015 (+$110)? If there's no performance penalty, I'd probably throw down another $53 so I could step up to the quad NIC X10SLM+-LN4F.

32GB might be limiting for such a large system. Consider a Haswell-EP system, with an LGA 2011-3 motherboard. A nice matching processor would be a Xeon E5-1650 v3.

If you're ok with sticking to 32GB, check out my Supermicro X10 FAQ - it has a link to guide for choosing the best one for your needs.

Your guide is an absolutely awesome write-up. Huge thanks!

Stepping up to 2011 is a pretty significant price increase. I ran the numbers a little earlier and the size of the bump was shocking. Admittedly, at work, I'd tell the boss I needed it til I was blue in the face and got my funding, but at home I have to answer to the wife. She's far scarier.

I can probably only afford to fill a single Z2 8x3tb or 11x3tb vdev right now (drives already on-hand). If I later add more drives and decide I'm unhappy with the performance, would it be possible to swap out to a LGA2011 setup without losing the zpool? I'd still have a good use to put the 1155 hardware towards.

 

[Bidule0hm]

"would it be possible to swap out to a LGA2011 setup without losing the zpool?" Yes, it's one of the big advantages of FreeNAS ;)

 

[Ericloewe]

The price point on the rig you just laid out is drool-worthy. Thanks for this!

I'm having a hard time pinpointing exactly which cable I need to get from the SATA-style SAS connectors on the X10SL7-F. Can you link one?

How would performance compare if I dropped down to a X10SLL+-F (-$82) and tossed in an IBM M1015 (+$110)? If there's no performance penalty, I'd probably throw down another $53 so I could step up to the quad NIC X10SLM+-LN4F.

Probably 0 perfomance difference.

Question is, why do you want four i210s? You're not going to automagically get a 4Gb link between switch and server, unfortunately...

 

[Thadius Miller]

Question is, why do you want four i210s? You're not going to automagically get a 4Gb link between switch and server, unfortunately...

2 active and 2 for failover would be nice. Hate being faced with having to go buy addon cards later... I can count at least 6 integrated NICs I've burned out over the last 5 years. No idea why. Thankfully it seems to offset my hard drive failure rate; only lost 1 drive in 20 years, knock on wood!

I've also been toying with dual homing to a small internal network. Thinking about setting up a lab for Cisco R&S labs with an old client attached... would make it easier to practice diagnosing real-world if I had a place to pull/send data to, from an old client I have laying out.

I'm aware I won't get 4 Gbit link speed. Besides, one of my LAG groups is getting a little lonely and the 48-port looks a little sparsely populated. ;)

 

[Thadius Miller]

A little update. I ordered most the parts.

When doing price comparisons above, I was pricing against the X10SLL-F mistakenly. It took a couple days and probably the 10th time reading it before I realized that the model I wanted was the X10SLL+-F because of the matched NICs. The major retailers didn't seem to carry the +- version (or at least their search engine had a very hard time with the model number). The price difference was above $32 more than I thought.

I think I was also pricing the M1015 at $100 as a WAG. I ended up grabbing a server pull from eBay for $80 shipped.

X10SSL-F $168 + M1015 $80 = $208 (but that NIC... read too many things I didn't like and didn't want to chance)
X10SLL+-F $203 + M1015 $80 = $283
X10SL7-F $250 = $250
X10SLM+-LN4F $221 + M1015 $80 = $301

YMMV. This thread already seems to be coming up in Google search results, so I figured closure would be good.

 

[Thadius Miller]

My finalized build:

Xeon E3-1231 V3 (overkill for most people, but I'm going to hammer it)
SuperMicro 4U 24-Bay Storage Server Chassis SC846E1-R900B
---> included BPN-SAS-846EL1 SAS expander backplane, single 900W PSU, and trays
SuperMicro X10SLM+-LN4F motherboard
32gb PC3-12800 ECC unregistered (two 2x8 kits -- Crucial CT2KIT102472BA160B -- from mobo QVL)
12x 3tb Western Digital Red WD30EFRX (already own) in RAIDZ2
IBM Serveraid M1015 (cross flashed)
OCZ ARC 100 240gb SSD

The HDD trays require special screws, so don't forget them or you won't be able to fit the trays with regular screws. They're the standard 6-32 threads but with a reduced head size. A 100 bag and 24 bay labels will set you back about $6. Look for part number MCP-410-00005-0N.

The chassis arrived next-day with standard shipping (150 miles) and in great condition. I've already dry racked it and am awaiting the rest of the parts tomorrow/early next week.

The SSD is just to play with while I wait for a drive sale. I have too much data on my existing RAID 50 and SHR 1 arrays right now to migrate without a couple extra drives to help do the lifting.