BUILD Hardware selection - 1st FreeNAS build

[DataKeeper]

So here is the hardware list I'm looking at for my FreeNAS build. I'm looking for a build which will last many years. This is a Home/Family storage server for a large media collection of dvd and blu-ray backups, just under 200k music library from both CD and vinyl, personal/business documents and my 180k photography collection. Photos are worked on from my office desktop, archived on the NAS and backed up to a backup server. Music will be streamed to a few locations in the house 24/7 through a home audio system being installed with 9 zones. Plex server to stream video to as many as 5 TVs as well as to my personal home theater and the kids iPhones or iPads while on the go. A personal Cloud for 6 user, provide iscsi shares to half a dozen computers/laptops (both Microsoft and Debian based) in the house and act as a backup for 2 home systems. Will also need VPN access for myself and possibly the kids.

Server will be located in a dedicated server room in the basement where it remains about 58 degrees year round. New Cat6a shielded cabling is being run throughout the house, outdoors and to the garage to support 10GbE with all rooms receiving anywhere from 2-8 separate runs. Most of the house will remain on 1GbE for at least another few years however I hope to have the freenas, backup, ESXi, web & email servers, my office desktop and home theater on 10GbE at some point this year.

• Chassis:
  • • SUPERMICRO CSE-846E16-R1200B 1200W PSUs
    • eBay - Seller certifiedservers - $320.00
• Mainboard:
  • • SUPERMICRO MBD-X10SRL-F
    • Newegg - $269.99
• Processor:
  • • Intel Xeon E5-1650 v3 Haswell-EP 3.5GHz
    • Newegg - $599.99
• CPU Cooler:
  • • Noctua NH-U9DX i4
    • Newegg - $59.99
• Ram:
  • • 1 x SAMSUNG 32GB SDRAM ECC Reg DDR4 Model M386A4G40DM0-CPB (LRDIMM)
    • 2 x SAMSUNG 16GB SDRAM ECC Reg DDR4 Model M393A2G40DB0-CPB (RDIMM)
    • Newegg - $184.99ea x 2(32GB) = $369.98 ; x4(64GB) = $739.96
• Install Drive:
  • • 2 x SUPERMICRO SSD-DM064-PHI SATA DOM
    • Newegg - $97.95ea x 2 - $195.90
• Data Drives:
  • • 12 x WD Reds WD40EFRX 4TB (Once they drop in price again)
    • Amazon - $160.49ea x 12 = $1925.88
    • Newegg - $179.99 - $20.00 Promo = $159.99ea
    • Currently have 18 of these Red Drives +1 Spare
• Data Cables:
  • • 2 x Supermicro CBL-0281L 75cm Mini-SAS (SFF-8087) to Mini-SAS
    • eBay Seller tarpontech.com - Make an Offer $7.50ea x 2 $15.00
• HBA:
  • • IBM ServeRAID M1015 SAS/SATA Controller
• NIC:
  • • Intel Ethernet Converged Network Adapter X540-T1 or X540-T2
    • Newegg T1 = $299 ; Newegg T2 = $484.99

The chassis and mainboard provide lots of future expansion for storage, ram, processing power, etc. At this point I think I'll run 12 x 4TB raidz2. I can then go with another 12 drive array later or break it down to 2x6 drive arrays if needed/wanted. I thought about the HGST NAS drives however the added heat, power requirements and noise brought me back to the WD REDs though prices in the last 2 weeks have increased $20.. *YAY! Amazon just dropped from 180 back to $160! I know I'll order 8 more (I have 5 now) and the price will drop again.

Anyone know if 2 of the Sata DOMs will fit correctly on the X10SRL-F mainboard? Looks like they might be tight and I'd like to mirror them.

Any thoughts or glaringly wrong decisions please let me know. Thanks!

Edit1: Changed Ram selection from LRDIMM to RDIMM
Edit2: Added sources and current pricing

 

[Jailer]

Absolutely drool worthy build. And nice work on the component selection, you obviously did your homework before selecting them.

There is plenty of room for 2 of the supermicro SATA DOMS on the board, that's why the yellow slots are spaced the way they are. I have one in my build that you can just see in the bottom left corner below to get a relative size comparison.

 

[Darren Myers]

Mhmmmmm server parts.....money... <insert home simpson drooling here> Love it love it love! My only suggestion is try and stagger when you get the 4TB hdds, or stagger where you purchase from (to try and avoid getting from the same batch)

 

[SweetAndLow]

awesome build! basically the same build I'm working on. I recently got my 4TB drives from newegg at $150 each so the deals can be found out there just keep looking. Some other things to think about.

1. do you need the 1650 over the 1620? you get more cores but I'm not sure you need that much for transcoding ~5 videos. But if you have the money they yes get that CPU.
2. Make sure that cooler can fit in that case. The height is something to consider. I was just going to go with the SM cooler, SNK-P0048AP4.
3. I doubt you need LRDIMM. Just getting 16GB modules will work up to 128GB.
4. Do you need the extra nic right now or would it be better to wait?

 

[Darren Myers]

I can confirm the large noctua's fit in a 4U case (i have 3 4U cases and all of them have Noctua heatsinks)

Photo from the build pic thread of my server.

 

[marbus90]

SweetAndLow:
1. for 5 plex transcodings I'd get a 6core. a Xeon E5 gets little over 2000 passmarks per core, so that fits.
4. it's a 10Gbe NIC.

second on the DIMM choice, ideally you want to install DIMMs in pairs of 4.

 

[SweetAndLow]

SweetAndLow:
1. for 5 plex transcodings I'd get a 6core. a Xeon E5 gets little over 2000 passmarks per core, so that fits.
4. it's a 10Gbe NIC.

second on the DIMM choice, ideally you want to install DIMMs in pairs of 4.

I still think the cpu might be overkill but at the same time i like it!

 

[DataKeeper]

Jailer.. Thanks. Its been a long time since I last built a system and relearning what fits and works again was a bit confusing.
Btw.. That's the X10SL7 not the X10SRL-F I'm looking at. The 2 yellow Sata ports are side by side on the X10SRL-F not stacked. I'm sure they will fit but it looks very close! If someone cant confirm I may give Supermicro a call and have them confirm before ordering. I won't hotlink but below if a link to the X10SRL-F photo showing the difference.
http://www.bsicomputer.com/prodimg/9504/x10srl-f-motherboard.jpg

Darren.. Yeah it hurts for sure! I've always preferred building something once, with the correct parts and having it both work and last. Doesn't matter what it is.. it costs! I bought 5 drives last month. I'll order another 5 in a few days along with everything else and then likely 4 or 5 more a couple days later. I plan to have 2 spare drives (14 in total for now) sitting in trays ready to swap out if needed. I'll have an order via Newegg and Amazon for the parts and both do enough business that from day to day you'll be pulled from different batches for sure.

S&L.. Thanks. Yeah I'd like to find them at 150 for sure! Been at 180 for the last 2 weeks though Amazon dropped last night back to 160.
1. I looked at both to be honest the 1620 would likely be enough. That said I'm still not 100% certain of what all I'm going to use the system for. I'd rather over build now and not deal with just having enough.
2. Initially had the SM cooler listed but changed to the Noctua after finding 2 others online running it in this chassis. I see Darren makes the 3rd person to confirm this. :)
3. I wanted to drop 32GB in right away and looked at the Samsung 16GB RDIMM. They were $195ea so $390 for 32GB. The LRDIMM 32GB is only $9 more at $399. Two RDIMM modules produce more heat then 1 LRDIMM module. The costs of a single 32GB RDIMM was over $600 compared to the $399 32Gb LRDIMM. The LRDIMM has been shown to process fast enough that the CPU at times doesn't need to step up another core. While single module idle voltage is higher on the LRDIMM it is lower then the consumption of 2 RDIMM modules so 1 32GB LRDIMM uses less power then 2 16GB RDIMM modules. (I hope I got all that correct. Just spent the last 30 minutes looking for the article I read a few weeks ago on this and couldn't find it..)
4. Need? Heck no but I do WANT 10GbE. Spending this kinda cash the extra 300 now isn't gonna kill me.. though shush.. The wife may!

 

[DataKeeper]

Hmm.. so was my entire understanding of LRDIMMs wrong? Would I be better off with purchasing 2 16GB DIMMs for 32GB instead of the LRDIMM?

 

[DataKeeper]

S&L.. I should also add.... The 5 TVs are current TV in the house.. office, family, master bedroom & 2 other bedrooms. We (I) plan to finish the basement next year and 1/2 of it (900sq/ft) will be an old Irish design sports bar with 4 or 5 extra TVs. Pulling reruns and game/fight highlights via Plex to them all. If the guys are over or a party going we could have quite a few TVs on. Thats still 2-3 years out however and who knows what those years bring.

 

[Darren Myers]

If the guys are over or a party going we could have quite a few TVs on. Thats still 2-3 years out however and who knows what those years bring.

Soooooo should we expect invites via FreeNAS forums.....email.....snail mail for a more personalized approach? :P

 

[DataKeeper]

Ouch.. bad memories! I made the mistake of mentioning a party years ago to someone who had it announced over the flipping radio. When the cops arrived there were roughly 2000 people attending and more showing up. Amazingly there was very little damage overall. Zero problems until the police moved in to disperse everyone. This is still brought up by friends and family.

Edit.. btw that was YEARS ago! I believe these days radio stations will cut the address while on-air. Back then it was just "Hey There is a Party at this address"

 

[DataKeeper]

Crap! I just added it all up!

My look to the wife later if (when) she looks at the CC bill.. LOL

 

[SweetAndLow]

ha that e5-1620 is starting to look quite nice now huh. ;)

You almost have me talked into getting the 1650 but then i look at the price and decided not to get it.

 

[DataKeeper]

Gives one pause for certain! I'll stick with the E5-1650v3 however.

Do it! You know you want it :D


Modification!
OK so I changed to rdimms. Once I gave it another thought I realized that I had only accounted for the single 32GB lrdimm and not a pair! I do know better lol.:) Change noted in original post.

 

[marbus90]

Socket 2011 systems are quadchannel = 4 DIMMs per socket preferable, by the way. But I still wouldn't go lower than 16GB RDIMMs.

 

[DataKeeper]

Now that I didn't realize! I knew it was quad channel but thought you could still install in pairs only.

 

[Ericloewe]

Now that I didn't realize! I knew it was quad channel but thought you could still install in pairs only.

You can, but groups of four are faster. In practice, it's better to make your future life easier by going with 16GB DIMMs now, since the performance difference probably won't be noticeable.

 

[jgreco]

ha that e5-1620 is starting to look quite nice now huh. ;)

You almost have me talked into getting the 1650 but then i look at the price and decided not to get it.

The E5-1650v3 is, per dollar, the best value you can get in the E5's if you need lots of CPU. At about $550 for 21GHz and 15MB cache, it is the top of the 1600's. Going over to the 2600's, everything becomes incredibly expensive. That E5-2697V3 may be 36GHz and 35MB cache, but it's also $2600. The 2643V3 is fairly similar to the 1650 but costs about 3x as much.

 

[jgreco]

I still think the cpu might be overkill but at the same time i like it!

By the way, the 1650V3 was the CPU I was using to get 600MByte/sec throughput to a single VM and the thing wasn't really breaking a sweat. It's definitely futureproof for a few years.