I finally got around to setting one of these up...

[jgreco]

Anyone who's been reading the FreeNAS forums knows I've got a bit of a thing about power consumption.

I've been working through a backlog to get to a point where I could deploy some new gear, and thought I'd share some results.

My ultimate goal is to replace some aging Opteron storage server gear that hovers around 100-110W. Nice, six years ago, with Opteron 240EE CPU's, they were never deficient before when serving files, but with ZFS, the 8GB memory limit for a uP box, and ZFS's pigginess on a uP system meant I was pretty unhappy with FreeNAS on my old gear.

Well, tackling the 1U form factor is problematic, we've been mostly an AMD shop for years and no recent Pentium/Xeon gear is familiar. So I took some time to review ... lots of stuff ... and then built us some new ESXi nodes. So I've been playing with this for about a week now.

The initial build was a Supermicro X9SCL+-F with a Xeon E3-1230, two Crucial 4GB 512Mx72 DDR3PC3-10600, a P2M-6600P 500W power supply, two Seagate Momentus XT 500GB drives, and an 8GB microSD USB flash. Cooling via two Cooler Master Excalibur 120MM fans, and processor cooling with a Xigmatek Loki 92MM heatsink/fan, all thrown into a generic chassis. This landed me with a platform that seemed to run around 120W peak, and (when fans, etc were throttled) idled around 57W.

That by itself kind of blew my socks off. Adding two more sticks of RAM added 2W to the total (59W). This burned in pretty happily under ESXi with a FreeNAS guest serving up the two Momentus drives via Intel VT-d.

But today saw the arrival of a pair of Kingwin STR-500's. These are extremely efficient (90%+) power supplies, and I swapped one in. Idle power dropped from 59W to 44W. Under peak CPU load, power went up to 85W. It is throwing off so little heat that the fans are still running at low.

So this is just crazy.

Of course, this rig would take more power with some serious 3.5" storage in it, but for a fairly high end rig to consume so little power... I just had to post this.

 

[jgreco]

So this prompted me to think about the N36L, which eats about 67W loaded with 4x WD Caviar drives which all eat about 9W spinning. 72W when the CPU is 100% busy.

Pulled the drives out of the N36L, because I was *so* curious... idle, 29.5W. Okay, that's about right. Five watts more if maxxing out the CPU.

Basically if I pull the drives off the Supermicro to make it "equal", the Supermicro will probably idle at 42W. And if it only had the 8GB, that'd be more like 40W. I didn't try it but the math is sound. The Supermicro eats an extra 40 watts if maxxing out the CPU.

Now I hate benchmarks, one can make them say whatever. I don't like any of them, but I had to pick something, so Geekbench is kind of OK. Charitably, the N36L pulls in at around 2100. The X9SCL/Xeon combo pulls in at 9700 - or four times faster.

So here's some "conclusions."

The N36L uses only 75% of the SM's watts during idle (not counting drives). The N36L uses only 41% of the SM during peak CPU.

But the Supermicro kicks it around the yard.

The N36L cost a little over $400, decked out with memory, no drives. The Supermicro, already having a generic chassis in stock, was $700 in parts.

It's probably stupid in most cases to waste a Xeon on a fileserver like this. BUT WAIT ... there are two different things to consider here.

1) Running VMware with FreeNAS as a VM might allow one to make good use of the extra Xeon cores. This is exceedingly enticing IMHO.

2) The E3-1230 is theoretically an 80W part. Now I was only seeing 85W used by the entire system at peak, so it's not really entirely clear, but the board is light on chips and I suppose it's really possible that the mainboard only eats maybe 15-20W, which'd be *awe*some. I really wonder what'd happen if one were to use a part like the i3-2120T (35W part) instead.

That part seems to Geekbench in the mid 6000's, so still maybe 3x the speed of the N36L ... but now costwise we're getting really pretty close ($160/$120/$60/$150 ~= $500) so it's only a 25% price premium, assuming you already have a chassis.

Dang. Now I almost want to buy an i3-2120T to see what's what.