Sanity check - E5-2650L v3 build

[pro lamer]

Hello!

Long story short: I've found a deal I like on E5-2650L v3 1.8-2.5GHz. Are there any serious pitfalls I missed?

I'm finally building a new rig. A first one since 2003 (except for one latop I bough 1,5 years ago but that was not a build ;) ).

Planned setup:
Chassis: For now: a DIY open case based on a codegen 6013 "shucked" motherboard tray. In future - some real chassis. Additionally some HDD enclosures or something (not sure which ones, initially some Codegen chassis leftovers ;-) Later - some enclosures with fans or sth. I've seen plenty of types in our forums :) )...
HSF: Thermalright AXP-100RH and Narrow ILM adapter Or some HSF more capable if this one doesn't do the trick...
Motherboard: In future X10SRM-T/-TF if the HSF above can can't cool the CPU enough (but temporarily: X10DRL-i with only one CPU socket populated - already on its way). I can test the HSF with the uATX one for a few days soon to learn if the HSF does its job.
RAM: 1xCrucial CT32G4LFD4266 32GB LRDIMM (from Crucial X10SRM-T/-TF compatibility lists) or the 64GB version if can't use X10SRM (the 64GB version is not in Crucial's X10SRM-T/-TF compatibility list but is in X10DRL's). If the AXP-100RH HSF doesn't do the trick then I can't use X10SRM and will stick to the X10DRL for longer...
PSU: Corsair RM850i (already bought - overkill but at least future proof and I'm in love with it ;-) )
HDDs: Initially 2x6TB NAS HDDs mirror :) (already bought and burnt in)
Boot pool: to be done, I guess initially a single Kingston A400 SSD 240GB which I already have, later I think I'll "downgrade" it to a 120GB one
UPS: to be bought later
Backups: not sure, still: initially probably HDDs rotated off-site. I have 3-4 more 6TB HDDs already. Thus I'll probably buy some low profile 12V-enabled eSATA bracket (this one IIRC), more HDD enclosures and some eSATA 12V cables

Disclaimer - the hyperlinks above are for information purpose - most of the things I'm byuing in local store except for the Narrow ILM adapter so far

CPU's pitfalls I'm already aware of:

• the E5-2650L v3 basic freq. is less than 2GHz which may result in not optimal SMB performance (or poor performance? context: clients are 1Gbps) Turbo is 2.5GHz though
• Haswell performance will be impacted by Spectre/Meltdown/zombieland/... patches (but this could be possibly a bit minimized by using 2nd gen Xeon Scalables or E-2200 or something like this (can't recall the generation codename ;-) OTOH some benchmarks that already exist may proove me wrong, can't recall them now)

Show : The rig purpose/my preferences

1. NAS
2. not too loud so Dynatron R14 Narrow ILM HSF doesn't seem to be a choice for me
3. low profile - the shelf is a short-height one
4. one of them not bigger than uATX (micro-ATX)
5. future proof
6. home media server
7. home lab
8. lots of native snapshots (2500) and snapshot or rsync incremental replication possibilities -> I think I need plenty of RAM to cache metadata
9. possible future parts compatibility with a dual socket system.
   1. Dual SKT has been my dream :)
   2. Compatibility would let me exchange parts between rigs easily.
10. Haswell CPUs have recently got significantly cheaper then before in my area thus Socket R3 choice
11. 10Gbps ethernet version on X10DRL is expensive but X10SRM-TF is not very expensive comparing to X10SRM-F
    1. and the X55x controller uses some otherwise unused PCIe lanes IIRC Update: I've checked the mobo's manual diagram and the former sentence is wrong. Either X550 and i350 use 8 lanes
    2. and I have Cat.6 cables and sockets home - edit: but my pool config plan wouldn't saturate it and I can't imagine when I may need 10Gbps

Show : Other pitfalls I'm already aware of

• systems are more efficient when RAM modules are in pairs (pairs are strongly recommended)
• X10SRM uses narrow ILM and only few low profile HSFs support narrow ones so if the AXP-100RH doesn't work I need a square-ILM equipped motherboard
• an open case may need frequent vacuum cleaning ;) dust removal (don't know how often yet)
• I may need to remove and replace the HSF every time I do something to the RAM modules
• I need the PSU attached to the motherboard tray since I want to put the rig on a shelf where access is limited - so I drilled a few additional holes and used some additional screws, nuts and some metal parts which names I don't know even in my mother language ;-) but they are widely available in metal and furniture stores
• eSATA bracket may cause some issues but it's for backups only
• backup drives taken off-site may need to be some archive-oriented drives (not NAS ones). I don't know. So far I have 6 NAS drives. If this is a pitfall then it's a one I have ignored/haven't checked.
• Corsair PSUs have some pitfall, for example described here
• LRDIMMs don't work with E5-16xx v2/v3/v4 ones so I'm limited do 26xx (or maybe 46xx - I don't care of 46xx for now ;-) ) nor with Atom C3xxx (possible future separate rig for other purposes). But at least if an RDIMM/C3xxx set didn't work together I could test RDIMMs with my X10 system so I could rule out defective RDIMM - a long shot)
• supermicro server motherboards have some weird front panel/controller LED connector which I am not familiar with (yet?)
• if I ever want to use 10Gbps ethernet I either can't use SMB for this or need a CPU upgrade (I might be a bit wrong - I've seen some thread somewhere else mentioning they were able to saturate 10Gbps using E5-2620v2 2.1-2.5GHz so I may "almost" saturate it. BTW 2650Lv3 has better single threaded passmark score than 2620v2 but I don't know if it's applicable to SMB)

I've been so excited writing this then I initially tried to use < and > to create spoiler tags ;-) Hopefully few errors left in this post.

Any pitfalls regarding the CPU then? Is the SMB performance going to be very poor (1Gbps-wise) or just sub-optimal? ("e5-2650L v3 SMB" google query was not helpful to me :( ) Or any other pitfalls I missed?

 

[PhiloEpisteme]

an open case may need frequent vacuum cleaning ;) dust removal (don't know how often yet)

Partially for my own education, but would you be worried about an open case also having poorer cooling performance for the amount of power draw and sound? It'll be more difficult to move cool air only in the areas you want cool air to be moving in an open case than in a closed case, right?

if I ever want to use 10Gbps ethernet I either can't use SMB for this or need a CPU upgrade (I might be a bit wrong - I've seen some thread somewhere else mentioning they were able to saturate 10Gbps using E5-2620v2 2.1-2.5GHz so I may "almost" saturate it. BTW 2650Lv3 has better single threaded passmark score than 2620v2 but I don't know if it's applicable to SMB)

Ah, this is interesting. I would've assumed your bigger issue saturating 10Gbps NICs would be your pool configuration, drive speed, and number of concurrent connections and not the cpu, especially if you go with that 12 core CPU you posted. I use 10Gbps NIC's between my main and backup server just for replication tasks. I don't saturate the 10Gbps link yet but I certain do better than 1Gbps.

Backups: not sure, still: initially probably HDDs rotated off-site. I have 3-4 more 6TB HDDs already. Thus I'll probably buy some low profile 12V-enabled eSATA bracket (this one IIRC), more HDD enclosures and some eSATA 12V cables

Depending on your budget you can build a full backup FreeNAS system to use as your backup. I'd bet you could do it quite cheaply with parts you may already have or basic used parts off ebay. This'll give you several benefits including zfs send|receive, hot swap if you go with the appropriate chassis, and better reliability than eSATA. I suppose I'm asking more for why you are leaning toward this eSATA solution than something more robust. Obviously, for backups many folks opt for less reliable solutions, so this is more a question than a criticism of your build. :)

backup drives taken off-site may need to be some archive-oriented drives (not NAS ones). I don't know. So far I have 6 NAS drives. If this is a pitfall then it's a one I have ignored/haven't checked.

Interesting thought here. I went cheap with my backup drives and used desktop drives since failure is less critical and the price was much cheaper. I'd be curious to see what you settle on and why.

 

[pro lamer]

Ah, this is interesting

Now I could find it. It was this reddit thread
Edit: one more thought on this - what they could achieve back then with 2620v1 might need a beefier CPU after applying Spectre/zombieland mitigations.

It'll be more difficult to move cool air only in the areas you want cool air to be moving in an open case than in a closed case, right?

Well, on one hand I need to learn it, too, since I am a noob. On the other hand - or maybe exactly becasue I am a noob - I can only imagine I need to cool the CPU, HDDs and possibly the PCH... I hope I'm wrong but RAM may need cool air, too...

Moreover now I am becoming a bit worried the PCH temperature may not be available by the means of IPMI because the first 3 results after googling for "x10 pch temperature ipmi" have listed problems reading this temperature :/ Maybe I get a cheap pyrometer :/ The rig is to be kept on a top, short-height shelf so it may be difficult to be accessed so I may have problems using a pyrometer :/

I think I just need to pull the trigger. Then I'll learn about the PCH temperature readouts. Maybe I can buy a special way mounted fan additionally. I hope I don't need them 'cos some of them (like inwin mars) are not cheap. Some seem to be cheap enough (like Zalman FB123 92x92x25mm + bracket) but I can't find a store ATM. I've also read about people placing small fans directly on top of a HBA chip heatspreader - maybe they can work with the PCH too..

The drives would have their own fan/fans.

Regarding the sound/noise loudness - it is me who's learning here again ;-)

your bigger issue saturating 10Gbps NICs would be your pool configuration, drive speed,

Actually I can't imagine a 10Gbps use case for me yet so I guess I may not need 10Gbps now :)

Depending on your budget you can build a full backup FreeNAS system to use as your backup

That's the plan for future but it'll be onsite. Off-site backup would still involve HDDs rotation as I didn't like the idea of carrying a whole rig - I got scared of ESD. There are ESD protecting sleeves big enough for a miniITX rig but they are ugly :)

better reliability than eSATA

The eSATA solution I'm considering is a kind of hot-swap replacement/substitute. The eSATA solution for me would be just a simple connector and bracket attached directly to the motherboard SATA ports and molex power. These extra items would not involve any special eSATA chip. So I don't consider this less reliable. Just simple wires. On one hand it is an additional point of failure, on the other hand they may protect my HDDs SATA connectors and my motheboard SATA connectors - which would need to be connected/disconnected every time I rotate offsite disks... I would just need to learn how to carefully carry drives with wires sticking out :p Or carry them toghether with some cage/enclosure. Another point of failure. Looks like there is no perfect solution for rotating off-site drives other then a third rig and lots of internet bandwidth ;-) And the limited room at the shelf is an additional trouble I need to deal with :p

I'd be curious to see what you settle on and why.

So far I already have some NAS drives, maybe I buy some more drives which may be cheaper... Time shall tell. I'll try to remeber to post back :)