Life cycle cost vs CPU generation

[Kevin Horton]

I am looking at a couple of options to acquire a second FreeNAS server to act as remote backup for the data on my current FreeNAS server. I'm wondering which CPU generation I should consider. Skylake CPUs would use less power, but the acquisition cost for the CPU, motherboard, etc would be higher. An older CPU would cost less up front, but burn more power.

The power required to spin and cool the hard drives will be independent of the CPU and motherboard generation.

Where should I be looking for the sweet spot for total life cycle cost over 5 years, for a system that would support 8 disks and that would be idle the vast majority of the time?

Power costs me $USD 0.16/kWh (including all taxes, distribution costs, and other add-on costs). Thus a 10W continuous power increase would cost about $70 over five years.

 

[Ericloewe]

If you need eight disks, Skylake is going to easily draw 10W less, since you'd need an HBA on a Haswell system.

 

[BigDave]

You sound as though you are serious about accuracy in regards to this, so consider this.
Your current costs of electrical service will rise in that 5 year period, the increase in the
cost to power your server will mean you get to buy better, more efficient hardware...

 

[Kevin Horton]

If you need eight disks, Skylake is going to easily draw 10W less, since you'd need an HBA on a Haswell system.

That is very useful info. Thanks!

What is the max number of disks without needing an HBA for each of the recent CPU generations? I tried Google, but didn't strike gold.

 

[Ericloewe]

That is very useful info. Thanks!

What is the max number of disks without needing an HBA for each of the recent CPU generations? I tried Google, but didn't strike gold.

6 as far back as is relevant until Broadwell, 8 with C236 on Skylake.

 

[jgreco]

I am looking at a couple of options to acquire a second FreeNAS server to act as remote backup for the data on my current FreeNAS server. I'm wondering which CPU generation I should consider. Skylake CPUs would use less power, but the acquisition cost for the CPU, motherboard, etc would be higher. An older CPU would cost less up front, but burn more power.

Unless you're talking Xeon D, there are only very modest power improvements in the five years between Sandy Bridge and Skylake, which is essentially five years of near-stagnation. An older CPU will not burn all that much more power. Handful of watts, probably.

You are better off concerning yourself over things like whether or not there are HBA's etc to be concerned with, as @Ericloewe noted.

If you really want something more efficient today, consider one of the Xeon D's. There's a price premium on that, but they idle at a low wattage. 37 watts for an 8 core system with dual 10 gigabit ethernet, a 16 port SAS HBA, onboard M.2 options, etc. At $900 it's pretty pricey but they make a version with half the cores:

http://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-4C-7TP4F.cfm

about $450. The compromise here is that the core speeds on these Xeon D's are fairly low, but the 1537 will be overall faster than an E3. The 1517 won't.

 

[joeschmuck]

If you really want something more efficient today, consider one of the Xeon D's. There's a price premium on that, but they idle at a low wattage. 37 watts for an 8 core system with dual 10 gigabit ethernet, a 16 port SAS HBA, onboard M.2 options, etc. At $900 it's pretty pricey but they make a version with half the cores:

http://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-4C-7TP4F.cfm

about $450. The compromise here is that the core speeds on these Xeon D's are fairly low, but the 1537 will be overall faster than an E3. The 1517 won't.

But as a remote backup system, how fast does the system really need to be?

I am looking at a couple of options to acquire a second FreeNAS server to act as remote backup for the data on my current FreeNAS server.

What kind of connectivity are you expecting to have? I could make an assumption but that just gets me into trouble.

 

[jgreco]

But as a remote backup system, how fast does the system really need to be?

Well, right, that 's correct. I think the 1517 could be a good fit here. However, I don't want anyone to come away thinking of this as a glowing review of the Xeon D for every possible use. Frankly it sucks for any task where core speeds are important, because it just doesn't have them.

 

[joeschmuck]

Well, right, that 's correct. I think the 1517 could be a good fit here. However, I don't want anyone to come away thinking of this as a glowing review of the Xeon D for every possible use. Frankly it sucks for any task where core speeds are important, because it just doesn't have them.

I agree with you but putting into perspective the end user goal, the half priced solution seems to fit the bill. However if the OP wants to do more with it in the future, maybe the faster CPU would be the better choice. But I personally think you gave the OP something to think about.

 

[Dice]

Another option not yet mentioned in the thread are the Avoton boards, which might be interesting due to their low power consumption and high count of onboard SATA ports.

To provoke a thought - does your second NAS really need to be online 24/7? If it would be enough to run 'manual syncs' every week or two - then power consumption is indeed a non issue. On top of that, performance requirements would drop drastically with a 'single purpose unit' of this kind.

 

[jgreco]

I agree with you but putting into perspective the end user goal, the half priced solution seems to fit the bill. However if the OP wants to do more with it in the future, maybe the faster CPU would be the better choice. But I personally think you gave the OP something to think about.

The real downside to the Xeon D is that you're locked in to your CPU choice. I expect that for a pure backup server even the two core Pentium D would be sufficient since it'll get done "sooner or later" but I don't see that for sale yet. Had a hell of a time getting ahold of even the 1537 but I see them for retail sale at a few places now.

 

[Kevin Horton]

I've been distracted with other (non-FreeNAS) issues, hence the delayed responses.

But as a remote backup system, how fast does the system really need to be?

What kind of connectivity are you expecting to have? I could make an assumption but that just gets me into trouble.

My home internet is only 10mbps up, so that is likely the bottleneck.

I don't think the capital cost of a Xeon D system makes sense for this purpose. I doubt I need much CPU power for a system to be used only as a replication target. Would something on the order of a G3220 (or equivalent for different CPU generations) be appropriate?

Another option not yet mentioned in the thread are the Avoton boards, which might be interesting due to their low power consumption and high count of onboard SATA ports.

To provoke a thought - does your second NAS really need to be online 24/7? If it would be enough to run 'manual syncs' every week or two - then power consumption is indeed a non issue. On top of that, performance requirements would drop drastically with a 'single purpose unit' of this kind.

I'm hoping to have this system live at someone else's house or business. I need it to work essentially unattended, to minimize the demand on the other party. Thus I plan to have it running 24/7. I'll return the favour by allowing the host to back up to my FreeNAS, either using replication or CrashPlan.

 

[Dice]

What about noise?
Is 2nd hand server gear out of question?

edit:
For reference I priced out X10 vs X11.

X10SLL-F
Intel Pentium G3240 3.1GHz DualCore, 3MB, 53W TDP
2x Samsung 8GB DDR3L ECC 1600MHz 1.35V UDIMM

X11SSL-F
Intel Core i3-6100 3.7GHz DualCore HT AES AVX2 3MB
1x Samsung 16GB DDR4 ECC 2133Mhz 1.2V UDIMM x8 DR

The price difference (locally, foreign) is about ~50USD.

In order to lower prices further, I'd imagine 2nd hand is the way to go.
I couldn't find any X9 components retailed.

 

[Kevin Horton]

What about noise?
Is 2nd hand server gear out of question?

edit:
For reference I priced out X10 vs X11.

X10SLL-F
Intel Pentium G3240 3.1GHz DualCore, 3MB, 53W TDP
2x Samsung 8GB DDR3L ECC 1600MHz 1.35V UDIMM

X11SSL-F
Intel Core i3-6100 3.7GHz DualCore HT AES AVX2 3MB
1x Samsung 16GB DDR4 ECC 2133Mhz 1.2V UDIMM x8 DR

The price difference (locally, foreign) is about ~50USD.

In order to lower prices further, I'd imagine 2nd hand is the way to go.
I couldn't find any X9 components retailed.

I haven't contacted anyone yet to discuss putting this new server at their place, so I don't know how quiet it needs to be be. It probably won't be somewhere with an existing rack of noisy servers though, so I'm assuming the new server must be fairly quiet or it will lose its welcome.

Used equipment - that is certainly possible, if I can find something suitable that has at least five years of life left in it.

 

[Dice]

if I can find something suitable that has at least five years of life left in it.

I'd look for platforms that support a solid amount of RAM, ie, =>64GB.
The drawback of "older" x9 E5 systems (featuring significant bump in RAM capacity) is their power consumption.
I played with a 2xL5630 SM 847 box, it idled at around 4x the power consumption compared to my X11 Skylake rig that additionally includes a really power hungry LSI 9201-16i (about 2x the IBM 1015)! That was a clear message to me. :0

In my book, power consumption is acceptable on higher count HDD systems. But cannot be considered a viable option on low count (<8drives) HDD systems (what I suspect you are looking for).

 

[jgreco]

What about noise?
Is 2nd hand server gear out of question?

Well, motherboards are typically dead silent. It's usually the chassis, cooling, and hard drives that make the noise.

In order to lower prices further, I'd imagine 2nd hand is the way to go.
I couldn't find any X9 components retailed.

I punched X9SCL and X9SCM in on eBay and came up with all sorts of hits. $55 for an X9SCL is pretty awesome. http://www.ebay.com/itm/Supermicro-...C202-Chipset-Socket-H2-LGA-1155-/231836795521

The E3-1230 is a little more pricey at $135 http://www.ebay.com/itm/intel-E3-1230-SR00H-3-2GHz-Socket-1155-Quad-Core-CPU-Processor-/142041944610

With two 2.5" HDD's, the above platform was measured at 44 watts five years ago, so figure this is around 40-42 watts idle. This compares favorably to the Xeon D-1537 board I measured at 37 watts. Of course the Xeon D is a better platform with its 2x10GbE and HBA all built in, but the five year old E3 is still pretty awesome even today.

I wouldn't be particularly shocked to find that a similar Skylake system would be in the 30-35 watt range but I haven't had the chance to run that test.

 

[Dice]

That is a lot of value for 2nd hand!

 

[Kevin Horton]

Well, motherboards are typically dead silent. It's usually the chassis, cooling, and hard drives that make the noise.



I punched X9SCL and X9SCM in on eBay and came up with all sorts of hits. $55 for an X9SCL is pretty awesome. http://www.ebay.com/itm/Supermicro-...C202-Chipset-Socket-H2-LGA-1155-/231836795521

The E3-1230 is a little more pricey at $135 http://www.ebay.com/itm/intel-E3-1230-SR00H-3-2GHz-Socket-1155-Quad-Core-CPU-Processor-/142041944610

With two 2.5" HDD's, the above platform was measured at 44 watts five years ago, so figure this is around 40-42 watts idle. This compares favorably to the Xeon D-1537 board I measured at 37 watts. Of course the Xeon D is a better platform with its 2x10GbE and HBA all built in, but the five year old E3 is still pretty awesome even today.

I wouldn't be particularly shocked to find that a similar Skylake system would be in the 30-35 watt range but I haven't had the chance to run that test.

Thanks for the ideas.

I've got enough info now so I can move to the next phase, which is to determine exactly where this new server will live, and find out whether the host will want to use its services as well, or whether it will only be doing back up duties for me. That will determine how much CPU processing power and disk space it will need.

 

[jgreco]

I'd look for platforms that support a solid amount of RAM, ie, =>64GB.
The drawback of "older" x9 E5 systems (featuring significant bump in RAM capacity) is their power consumption.
I played with a 2xL5630 SM 847 box, it idled at around 4x the power consumption compared to my X11 Skylake rig that additionally includes a really power hungry LSI 9201-16i (about 2x the IBM 1015)! That was a clear message to me. :0

In my book, power consumption is acceptable on higher count HDD systems. But cannot be considered a viable option on low count (<8drives) HDD systems (what I suspect you are looking for).

Yes, the mid tier E5 Xeon's chow current. This isn't limited to X9, by the way. I really like the E5-1650 v3 on the Supermicro X10SRW platform but the damn boxes eat like a minimum of 120 watts (though there's a RAID controller and 10G card in there).

I've got direct comparisons on a dual socket system that had a single E5-2609 in there and then had a single E5-2697v2, the idle on the 2609 was 190 watts but became 225 with the 2697v2. Not doin' anything. Again the platform had "other hardware" in there but that's a direct reflection of the difference in idle power between those two CPU's.

 

[Kevin Horton]

Yes, the mid tier E5 Xeon's chow current. This isn't limited to X9, by the way. I really like the E5-1650 v3 on the Supermicro X10SRW platform but the damn boxes eat like a minimum of 120 watts (though there's a RAID controller and 10G card in there).

I've got direct comparisons on a dual socket system that had a single E5-2609 in there and then had a single E5-2697v2, the idle on the 2609 was 190 watts but became 225 with the 2697v2. Not doin' anything. Again the platform had "other hardware" in there but that's a direct reflection of the difference in idle power between those two CPU's.

Holy cr*p that is a lot of power consumed at idle! And then in summer, you need to consume more air conditioning power to get rid of that heat. I found it bad enough with my current system which idles at 60W (including six drives spinning, all fans, etc). It was originally in my office, and the 60W continuous heat output noticeably increased the temperature in that room. I finally cracked and ran ethernet to the basement and moved it down there.