Supermicro X10SRH-CF-O Build

[bhcelt]

I'm compiling a hardware list for a FreeNAS build that will be used for general storage, a Plex server able to handle 4-5 devices streaming, playing around with other jailed applications (testing joomla site devs, forum development, etc.) and backing up household computers. I prefer to rip BluRay to mkv files without further compression and am anticipating 4k / UHD Blu-ray media as it becomes available. The goal is to have a box that will be good for a long time (i.e. substantial expandability) and outperforms the comparably priced commercial NAS options.

I am aiming for a system that will start with 6 x 6 TB hard drive storage and will handle adding a second set of 36 TB raw storage when needed. I've researched various platform options as best I can, and arrived at the conclusion that I want not to be locked into a 32 GB memory ceiling, want a minimum of 14 SATA ports available (12 hard drives plus room for SSD's should they be desired), and Intel GbE NIC (x2 even better).

I know lots of folk don't like DDR4 based boards due to cost, but in pricing out an alternative system I saw a total cost difference of $124.00 between DDR3 and DDR4 when going for 64 GB (4X16GB sticks) of Supermicro approved list memory .

I also wanted everything to fit in one tower case. The Fractal Define XL R2 seems to fit the bill, when adding the AMS DS-346TL-BLUE 4in3 Trayless SATA/SAS backplane Module, which will use 3 of the case's 5.25 bays to give me the total of 12 3.5 bays. If I do add SSD's the AMS DS-524SSBK 2.5" X4 SATA Backplane Module for the last 5.25 bay would be a natural.

So even adding SSD's to 12 hard drives should be fine given the 10 base SATA ports together with the 8 from the onboard LSI 3008 controller.

I have a fair amount of experience putting together systems, but this will be my first foray into this type of server/NAS build. Any help in flaws in thinking and hardware selection would be appreciated. So here is my list:

Processor
Intel Xeon E5-1620 v3 Haswell-EP 3.5GHz $310.98
Motherboard
SUPERMICRO MBD-X10SRH-CF-O ATX $401.98
Memory
Samsung DDR4-2133 16GB/2Gx72 ECC/REG CL15 (M393A2G40DB0-CPB) $188.98 x 4 = $755.92
Case
Fractal Define XL R2 $142.78
Power Supply
Seasonic SS-660XP2 ATX 80 PLUS PLATINUM $147.00
SAS / SATA Cables / Drive Adapter
StarTech Model SAS8087S450 19" (50cm) Serial Attached SCSI SAS Cable -SFF-8087 to 4x Latching SATA
$28.99
Rosewill RCAB-11050 - 18" Flat Red SATA III Cable $4.99X8= $39.92
AMS DS-346TL-BLUE 4in3 Trayless SATA/SAS backplane Module $89.99

System Cost Without Drives $1,896.48

ooops.. forgot I am thinking of a NH-U9DX I4 for CPU cooling

 

[marbus90]

If you are going to use an old chassis with 6Gbps expander, don't get the SAS 12Gbps chipsets but the X10SRi-F or X10SRL-F with an LSI 9207 or 9211, IBM M1015, Dell H200 HBA.

http://www.supermicro.nl/products/nfo/SATADOM.cfm are ideal for bootdevices, 16GB is enough.

 

[bhcelt]

Oh yes, I also did not mention that this could be started for the 36TB array with 32GB of memory for $1400.00, then can be expanded as needs demand up to 128GB using 16GB sticks. I'd like the performance of plenty of DRAM though, so it will probably be $1800 to start. The drives I'm looking at most strongly are the HGST Deskstar NAS 6TB.

 

[Ericloewe]

Hardware looks good, as long as the board fits in the case.

It sounds like you won't be using expanders, so no need to worry about SAS3. If you decide to use expanders, stick to SAS3 expanders or SAS2 HBAs + SAS2 expanders, as marbus90 suggested.

 

[bhcelt]

Thanks, marbus90, yes I have considered a SATA DOM as a thumb drive alternative, sounds like it has some advantages from what I've gathered. Can you explain more on motherboard and expander comment? Are you saying the MDB-X10SRH-CF with the LSI 3008 SW controller with a breakout mini-SAS to SATA cable is a poor choice compared to a mother without the LSI 3008 and buying an add-on board? I was not planning on using an expander, using the Fractal case and just cabling directly to all drives off the motherboard, which I thought would be more than enough capacity (18 total SATA ports available )for the number of drives I envision. I thought that flashing the LSI 3008 on board to IT mode would get me where I want to be. Am I missing something?

 

[bhcelt]

Ha, just was composing my question regarding marbus90's expander feedback and had not seen your post, Ericloewe, thanks for the answer. Yikes, had not considered the motherboard might not fit, since it's listed as an ATX, and the Define XL case supposedly has "ATX, Micro ATX, mini-ITX, E-ATX and XL-ATX motherboard compatibility".

I had originally thought of going with the smaller Define R5 case, which would work well for up to 10 or 11 drives cannibalizing it's two 5.25 bays, but from what I've gleaned, adding sets of 6 drives when using Raidz2 in Freenas / ZFS is preferable to a single pool of 10-11 drives or a set of 5 and 5, 6 and 4, etc. Have I given that too much weight as a system design factor?

A 10 x 6 GB capacity would probably be enough storage. I had seen conflicting opinions on this recently, with some saying that when using lz4 compression, it may be a less important consideration.

 

[Ericloewe]

Ha, just was composing my question regarding marbus90's expander feedback and had not seen your post, Ericloewe, thanks fr the answer. Yikes, had not considered the motherboard might not fit, since it's listed as an ATX, and the Define XL case supposedly has "ATX, Micro ATX, mini-ITX, E-ATX and XL-ATX motherboard compatibility".

In that case, it'll fit (no pun intended).

 

[bhcelt]

If it won't fit, you must must acquit!

 

[marbus90]

The SATADOMs are basically little SSDs on a single PCB with a direct-attached SATA connector, trough which these are getting their power. SATA is more reliabe than USB, hence the preference.

Usually I'm looking for rackmount chassis for most things, I absolutely loathe towerchassis with aftermarket hotswap trays. nothing like a proper supermicro chassis... With that SAS board I'd look for a 16 or 24bay chassis and use 8disk raidz2 vdevs out of 4TB HDDs. or 6TB if you want that space... then 2 ports would still be free, that's what the SATADOMs would be used for. SSD cache usually doesn't work.

Seems like I missed out on the 4-5 plex streams: I'd go with an E5-1650 v3 for that. Also if you really want a futureproof system whilst trowing unconverted bluerays at it, you might want to look into the 36bay variants right away. Supermicro would have a barebone ready for that, CPU coolers, cables and controllers included, just drop in your CPU, DIMMs and HDDs.

Lastly: go 16GB RDIMMs from the start. The performance difference between RDIMM and UDIMM isn't that big.

 

[bhcelt]

Thanks marbus90, I had a rough idea about SATADOMs, but your description makes that a lot more clear to me. The MDB-X10SRH-CF has "2x SuperDOM with built-in power" ports on it, so easy peasy to go that route!

 

[Ravindra]

Hi, Did you considered "Supermicro ATX DDR4 LGA 2011 Motherboards X10SRH-CLN4F-O"? For little more you will be getting four gigabit Ethernet ports. Any reason to select that MB?