Hardware opinion

[Bulldog]

Hi, I'm building anther FreeNAS box. System is going to be used for an iSCSI target to my WMC server as recording storage. It will also serve as iSCSI targets to other HyperV lab servers and CIFS shares. I'm having a hard time deciding which direction to go Xeon V2 or V3. It will eventually house around 48TB total, 24x 2TB drives. (3 x RAIDZ2 vdevs 8drives).

I've looked at the supermicro MBD-X10SRL-F with 4x16GB (64GB) DDR4 ECC RDIMMs and a Xeon E5 2620V3.
http://www.newegg.com/Product/Product.aspx?Item=N82E16813182927
http://ark.intel.com/products/83352/Intel-Xeon-Processor-E5-2620-v3-15M-Cache-2_40-GHz

I could save some money going with a Xeon V2 system and a used CPU on ebay, but is that money saved really worth it in the long run vs spending a little more now and having it last longer? I also don't want to be limited to a 32GB memory board which leaves no expansion if I ever went with bigger drives.

Other question is the memory. If I go with DDR4, I originally only wanted to start out with 2x 16GB RDIMMs, but not sure if this will cut performance in half only utilizing 1 memory channel vs if I went with say 4DIMMs upfront whether it be 4x8GB or 4x16GB.

Any reasonable opinions always welcome.

Thanks,

 

[SweetAndLow]

You won't notice the memory channel difference.

For your use case you might need mirrors not raidz3.

I don't think v2 or v3 will matter but I would go with newer stuff when possible. Do you need a dual socket like your have suggested or can you just get a single socket CPU?

 

[Bulldog]

Could you give me little detail on why doing mirrors? Are you saying like 12 vdevs of 2drives each in mirror?

Im only considering a single socket cpu. Im leaning towards the V3 since its newer and not much more. The memory being ddr4 is where the price increase is.

 

[Bulldog]

How important is 15MB L3 cache and faster core speeds? The 1620v3 is 3.5ghz quad core vs the 2620v3 2.4ghz 6core 15MB L3cache.

http://ark.intel.com/m/products/827...v3-10M-Cache-3_50-GHz#@product/specifications

 

[SweetAndLow]

Because you are going to use cifs I would say the higher clock speed is better. I don't know what iscsi likes though. The 1620 is a single socket and the 2620 is a dual socket.

 

[jgreco]

I could save some money going with a Xeon V2 system and a used CPU on ebay, but is that money saved really worth it in the long run vs spending a little more now and having it last longer? I also don't want to be limited to a 32GB memory board which leaves no expansion if I ever went with bigger drives.

Other question is the memory. If I go with DDR4, I originally only wanted to start out with 2x 16GB RDIMMs, but not sure if this will cut performance in half only utilizing 1 memory channel vs if I went with say 4DIMMs upfront whether it be 4x8GB or 4x16GB.

The 32GB RDIMM's are priced around the same $/GB as the 16's. Take advantage of that and do not waste money on 16GB RDIMM's. The memory subsystem is a hell of a lot faster than your network interface. Obsessing about memory channels is worrying about the wrong thing.

Do not feel pressured to make decisions about v2 vs v3 based on "newer." Make these decisions based on "smarter." The Sandy era Xeons and the Skylake era Xeons are not particularly different, despite five years of evolutionary improvements. If there is some particularly compelling new feature (Skylake - 64GB UDIMM) then of course consider that. If prices are similar between newer and older, then newer is probably more attractive.

 

[Bulldog]

A single 32gb RDIMM used is around $190. I would need 2 at that point. I see what you're saying going full LRDIMM right away. I really can't spend that much on memory alone unfortunately.

I was considering the LGA2011 Haswell eras. More cores and L3 cache? or 4 less cores and less L3 cache for over 1Ghz faster core speeds? As nice as 12threads sounds, I don't think it'll ever get used in a freenas box.

E5- 2620V3 $367 http://ark.intel.com/products/83352/Intel-Xeon-Processor-E5-2620-v3-15M-Cache-2_40-GHz
E5- 1620V3 $260 http://ark.intel.com/m/products/827...v3-10M-Cache-3_50-GHz#@product/specifications

 

[jgreco]

Oh, and why are you going with E5-26xx? The E5-16xx stuff is cheaper, the E5-1650v3 (~$550) is actually the best bang-for-buck you can get for a NAS. Be aware the E5-16's don't do LRDIMM, so going RDIMM may be smarter for future options. RDIMM pricing is about $100/16GB or $200/32GB last I checked. Buy a single DIMM and use that, I'm pretty sure that should be supported even if it is "nonoptimal."

 

[cbalmer]

Could you give me little detail on why doing mirrors? Are you saying like 12 vdevs of 2drives each in mirror?

VMs tend to like IOPS more than pure read/write. More vdevs = more IOPS. I believe the slowest IOPS of each vdev can be added together for the pool IOPS. So if each vdev gave you 50 IOPS and you had 12 vdevs, you'd see 600 IOPS. Whereas if you only had 3 vdevs (your RAIDZ2 from OP), you'd get 150 IOPS. These are made up numbers as an example.

 

[Bulldog]

I dunno if I want to spend $550 on the CPU alone. If I went with a 1600 V3, The max memory I could use would be 32GB RDIMMs per slot correct? I was under the impression for some reason you could only get anything bigger than 16GB in LRDIMMs.

 

[Ericloewe]

I dunno if I want to spend $550 on the CPU alone. If I went with a 1600 V3, The max memory I could use would be 32GB RDIMMs per slot correct? I was under the impression for some reason you could only get anything bigger than 16GB in LRDIMMs.

Samsung even has 64GB RDIMMs for sale right now. They're rare and rather expensive, though.

32GB RDIMMs are fine, though.

 

[jgreco]

I dunno if I want to spend $550 on the CPU alone. If I went with a 1600 V3, The max memory I could use would be 32GB RDIMMs per slot correct? I was under the impression for some reason you could only get anything bigger than 16GB in LRDIMMs.

Use the parts listed on the Supermicro web site for the board you purchase. The 32's are about $200 each at the moment.

You can use a cheaper CPU like the 1620. Same high clock rate but fewer cores.

 

[Bulldog]

Thanks for all your feedback so far.

Here's what i'm thinking of doing.

I can get used/tested 2x DDR4 32GB RDIMMS Samsung (SM approved M393A4K40BB0-CPB) for $350 total.
Motherbard: SUPERMICRO MBD-X10SRL-F $233.99
CPU: Xeon E5-1620v3 $260
or Xeon E5-1650V3 $530
Is the extra 2 cores and the additional 5MB L3 cache worth another $270 in CPU cost?

CPU cooler I've only looked at the SUPERMICRO SNK-P0050AP4 Heatsink $39.99 /Newegg.
Had decent reviews and does both narrow and square ILM.

Persuade me on the CPU please .

Thanks,
Mike

 

[jgreco]

The extra cache and cores are only worth it if you're going to be doing processing (jails, etc) on the machine. I spec'ed a 1650 in our VM filer in order to match other gear already here but I have to say it's usually 90%++ idle, and I'm not sure I've seen it less than 80% idle except while we were burning it in.