BUILD Newbie Supermicro X10SRL-F Build

[Stefan Krieger]

I'm considering the following build:

SUPERMICRO MBD-X10SRL-F Server Motherboard LGA 2011 R3 ($234)

Intel Xeon E5-1620 v3 Haswell-EP 3.5 GHz LGA 2011-3 ($308)

Crucial 64GB (4 x 16GB = 64GB) 288-Pin DDR4 SDRAM ECC DDR4 2133 (PC4 17000) ($464)

SeaSonic Platinum SS-860XP2 860W ATX12V / EPS12V ($150)

2x SanDisk Cruzer Fit 32 GB USB 2.0 Flash Drive (2x $15) as mirrored boot drives

SAMSUNG 850 EVO 2.5" 250GB SATA III 3-D Vertical Internal Solid State Drive (SSD) MZ-75E250B/AM ($95) for jails

Fractal Design Define R5 FD-CA-DEF-R5-BK Black Computer Case ($90)

2x Fractal Design Silent Series R2 FD-FAN-SSR2-140 140mm 140mm Fan (2x $15) as extra fans

8x WD Red 6TB HDD for a RaidZ2 ($245)

Researching FreeNas I had my usual cost escalating enthusiasms: looking Supermicro boards with SAS controllers, separate SAS/SATA controller cards, SAS drives, faster CPUs, X11 boards, Supermicro chassis with more bays. The one cost escalating enthusiasm left is buying lots of RAM since ZFS is RAM hungry. I could scale back and just get 32GB now, adding more RAM later if useful, but it appears that is asking for noticeably worse performance. I did go with a LGA 2011-3 board to have the ability go beyond the X10 LGA 1150 board 32GB limit (and not experience driver frustration with 64GB limited LGA 1151 X11 boards). Still thinking of going with the E5-1650 v3, but it is a big price bump at $570 (not at Newegg, using Google shopping instead here, I assume 4/5 star vendors are okay). I did consider SAS drives, but for my use cases (see below) it very much doesn't seem worth the extra $600+ for disks and controller.

The purpose of this NAS would be, for the most part, to be one of my backups (mostly many TB of photos, some music) and a video server (currently running Plex and Serviio on my Win 7 machine) for non-backed up video. I assume I'll find things to do in jails beyond that. 4K streaming should show up in the next two years. I found FreeNas by thinking I could do better than paying QNAP or Synology $999 for an 8-bay machine with a slow CPU and little RAM. And I'd learn more and have more fun if I built a FreeNas machine. My current desktop is no longer doubly backed up for my important data and some stuff isn't backed up at all anymore. 13 disks on a Win 7 is probably enough too...I don't want to add to that.

I'm putting in 8x 6TB immediately, since the overhead cost of RaidZ2 seems to warrant it. The case won't take more than 3 more drives (with an adapter for the 5.25 bay), so I'll need to move to a new case for a 2nd vdev, but that's probably far enough in the future that I don't need to plan for it now (and the R3 case is only $90). I could see using a smaller PSU, but I like being on the larger safe side (I could be talked in to a larger PSU) and I might end up using this PSU elsewhere in the future as well. I am planning on using the stock cooler, but I might go with a narrow R2011-3 cooler, most likely Noctua NH-U12DX i4 ($64), instead if the stock cooler is loud.

So 32GB now with the option to go to 64GB, or just go to 64GB now? E5-1620 v3 or E5-1650 v3? Anything else? Am I being stupid/silly somewhere?

And looking at cost of 8x 6TB disks the QNAP/Synology approach with pay-as-you-go raid expansion does seem attractive...

Thanks!

 

[BigDave]

I am planning on using the stock cooler, but I might go with a narrow R2011-3 cooler, most likely Noctua NH-U12DX i4, instead if the stock cooler is loud.

I'm looking at a future build with that mobo/cpu.
My research indicates that the board will fit SNK-P0050AP4 which is the narrow mounted cooler.
It has to be narrow due to the RAM slots.
The E5-1620 v3 does not come with a stock cooler AFAIK

 

[Ericloewe]

It has to be narrow due to the RAM slots.

More directly, the socket itself is different. LGA 2011 comes in square and narrow versions, the square being the typical one.

 

[BigDave]

the socket itself is different

I was under the impression that the difference between square and narrow ILM was the
RAM slots on some models being so close to the socket, they made the cooler mounting holes
in the PCB different?

edit: correct spelling

 

[jgreco]

Yes, the narrow ILM is more of a rectangular sort of pattern.

 

[Ericloewe]

I was under the impression that the difference between square and narrow IML was the
RAM slots on some models being so close to the socket, they made the cooler mounting holes
in the PCB different?

Kinda. LGA 2011 includes mounting hardware for the cooler, unlike LGA 115x (and LGA 1366 and LGA 775).

So, the socket is electrically the same, but it needs to be different, since coolers are attached directly to thew socket (similar to AMD).

 

[jgreco]

Kinda. LGA 2011 includes mounting hardware for the cooler, unlike LGA 115x (and LGA 1366 and LGA 775).

So, the socket is electrically the same, but it needs to be different, since coolers are attached directly to thew socket (similar to AMD).

What's "kinda" about that? The holes are in a different pattern and are not compatible with standard ILM. BigDave's description is correct.



click the pic for more.

 

[Ericloewe]

What's "kinda" about that? The holes are in a different pattern and are not compatible with standard ILM. BigDave's description is correct.



click the pic for more.

Yeah, I'm just pointing out that the socket itself is different, not just the PCB.

 

[BigDave]

Kinda. LGA 2011 includes mounting hardware for the cooler, unlike LGA 115x (and LGA 1366 and LGA 775).

So, the socket is electrically the same, but it needs to be different, since coolers are attached directly to thew socket (similar to AMD).

This does make things a little clearer for me. When gathering up my research for this build, I kept asking sellers of the coolers,
"does it come with the back plate for mounting"? I see now that the cooler mount is now an extention of the lockdown hardware.
I'm an idiot! Excuse me while I look at some pictures of the underside of this awsome board.

 

[Ericloewe]

This does make things a little clearer for me. When gathering up my research for this build, I kept asking sellers of the coolers,
"does it come with the back plate for mounting"? I see now that the cooler mount is now an extention of the lockdown hardware.
I'm an idiot! Excuse me while I look at some pictures of the underside of this awsome board.

It's a very pleasant change, since coolers are much easier to assemble. I have unpleasant memories of messing around with coolers on LGA 1366, made bearable only by a then huge cutout in the chassis (they've gotten even bigger because motherboard manufacturers acquired a love of moving the CPU socket around - the one in the HAF X now isn't even large enough to allow for access to the whole socket area on many Z97 boards, which is why I'm glad I'm using a Xeon E5-1650 v3 now).

But don't feel bad. Last year, when I was about to assemble the new hardware and went to grab the cooler's LGA 2011 hardware I thought to myself "Where's the backplate? The old one is too small for this socket." I actually had to read the instructions to remember that the backplate was part of the socket assembly. I'd completely forgotten...

 

[BigDave]

In my own defense, the glitter of all those Gen3 PCIe slots blinded me. :D

 

[jgreco]

In my own defense, the glitter of all those Gen3 PCIe slots blinded me. :D

Then don't click on this link, it'll probably kill you:

http://www.supermicro.com/products/motherboard/Xeon/C600/X10DRG-OT_-CPU.cfm

 

[Ericloewe]

Then don't click on this link, it'll probably kill you:

http://www.supermicro.com/products/motherboard/Xeon/C600/X10DRG-OT_-CPU.cfm

"We have some room left on the PCI-e breakout board."
"Did you route both CPUs' lanes?"
"Yeah, and the PCH lanes too..."
"Well then, just throw in a few PLX PCI-e switches and fill that sucker."

 

[jgreco]

I imagine if you needed to actually talk to a lot of high end GPU's all at the same time, it might be meaningful. For me, I have to remember that my displays could mostly be DisplayLink-on-USB2 and I probably would never notice. As a matter of fact, at least one of the displays on my desk right now *IS* DisplayLink-on-USB2 (the Mac Mini triple head). I can't imagine what anyone would need to push 16GBytes/sec to/from a card for.

Though I am having some fun thinking about what I want to do for a next generation workstation.

 

[Stefan Krieger]

I ordered the build. I went with the E5-1650 v3, 64GB RAM and the Noctua H-U9DX i4.

Noctua support emails that

For Narrow-ILM LGA2011-0/-3 motherboards we have the DX i4 line of coolers, as you already discovered. However, there can indeed be compatibility issues with memory modules on certain motherboard models like yours.

In vertical orientation (airflow towards the I/O panel) the heatpipes of the NH-U12DX i4 will interfere with the memory slots closest to the socket but this is not a problem in horizontal orientation.
If vertical orientation is a must, then I'd suggest to take the NH-U9DX i4 instead.
It will overhang the nearest slots too but will leave a space of ~33mm underneath, thus standard sized memory modules common in server environments will fit underneath.

 

[jgreco]

Yes, you should be using the Supermicro designed coolers. :-/

 

[DJ9]

Those Noctua coolers are excellent. Can't recall the exact model I have on my workstation, but the temps are much better than the stock Intel cooler that came with my Xeon E3-1230v2. I'm using the same R5 case, just added a additional front fan (same as the one that came with the case).

 

[Stefan Krieger]

Also, what's the deal with lots of places offering the E5-1650 v3 above the MSRP price of $586?

 

[jgreco]

Lots of consumers are idiots.

 

[jgreco]

Those Noctua coolers are excellent. Can't recall the exact model I have on my workstation, but the temps are much better than the stock Intel cooler that came with my Xeon E3-1230v2. I'm using the same R5 case, just added a additional front fan (same as the one that came with the case).

Yes, Noctua coolers are nice, but if they're interfering with the system board, then that's bad.