BUILD Supermicro X9SRL-F Build Check

[Bidule0hm]

The RAM should work without any problem at the frequency and voltage specified, if it doesn't then there's a problem. Of course if you overclock it it'll be less stable at 1.35 V than 1.5 V but who overclocks his server?

 

[Black Ninja]

You are right. No one with a right mind will.

Like jgreco mention, Supermicro ( as well as other vendors) have 1.35V options REG ECC at 1600MHz , when you chose a 1866MHz REG ECC option then they are showing only 1.5V memory. I guess when it get's faster (1866Mhz) most likely can't be 1.35V ?!

 

[jgreco]

I can't currently find any DDR3L parts at 1866 "at size." It's quite possible they never bothered to do this in any significant volume, or at all, or whatever.

In any case, with ZFS, having memory, and lots of it, trumps minor issues like the speed. You'd rather have 128GB of 1333 than 64GB of 1866.

 

[jgreco]

I can guarantee you that the supermicro 1u stock cooler for 1155 will cool better. Dont buy it you can have mine.:)

Thanks for the offer, but the 46's are going for like ten bucks used on fleaBay. Kind of a no-brainer.

 

[Black Ninja]

Thanks for the offer, but the 46's are going for like ten bucks used on fleaBay. Kind of a no-brainer.

No problem, I didn't know you need that many.
Just keep in mind it won't cool as good as supermicro 1u cooler.

 

[Black Ninja]

I can't currently find any DDR3L parts at 1866 "at size." It's quite possible they never bothered to do this in any significant volume, or at all, or whatever.

In any case, with ZFS, having memory, and lots of it, trumps minor issues like the speed. You'd rather have 128GB of 1333 than 64GB of 1866.

Or maybe DDR3 at 1866Mhz need to be at 1.5V. Cause I couldn't find it with other vendors too.
I'll start with 64GB and if need more will get up to 128GB.

It seems power consumption is around 1W more per memory stick on 1.35V vs 1.5V, so 4 stick around 4W more I can live with that number easy.I can buy 1866Mhz 1.5V REG ECC 64GB (samsung or hynix) from supermicro recomended list, would you do that or rather have 1600Mhz 1.35V ?

 

[jgreco]

I can buy 1866Mhz 1.5V REG ECC 64GB (samsung or hynix) from supermicro recomended list, would you do that or rather have 1600Mhz 1.35V ?

I really don't think it matters that much.

 

[jgreco]

No problem, I didn't know you need that many.:)
Just keep in mind it won't cool as good as supermicro 1u cooler.

The 46 *is* the Supermicro cooler. At a certain point I get tired of all the extraneous fluff in their part numbers...

 

[Black Ninja]

The 46 *is* the Supermicro cooler. At a certain point I get tired of all the extraneous fluff in their part numbers...

Oh I got it now. I don't remember the number but we know which one is it. I thought you meat as quantity - 46 heatsinks.
Anyways I am saying supermicro is better. I can even find some testing screenshot I keep from my tests.
On the first pic I let it run for 30min then stop it when it reached 79 C , but second one with supermicro I let run for 2 hours and never got above 76 C.

 

[Black Ninja]

I really don't think it matters that much.

Well it wouldn't matter at all if this was for a customer. But when you are doing for your self, trying to make something nice you can enjoy, and using your own money , then is kind of a different story. At least the way I am.

But still I want to know what would you chose if this was for your own build.(with the intention to be freenas, but could be re-purposed as SAN or ESXI host)

 

[Black Ninja]

I got a find myself a web site with memory speed test results like Geekbench for CPU, any suggestions guys ?

 

[jgreco]

Well it wouldn't matter at all if this was for a customer. But when you are doing for your self, trying to make something nice you can enjoy, and using your own money , then is kind of a different story. At least the way I am.

But still I want to know what would you chose if this was for your own build.(with the intention to be freenas, but could be re-purposed as SAN or ESXI host)

OCD much?

 

[Black Ninja]

OCD much? :p

Not sure what OCD stands for but hope is not "Obsessive-Compulsive Disorder"

 

[Black Ninja]

When I check Supermicro recommendations for reg ecc 1600 1.35V 16GB, it show 2 Samsung models. I think one just has a heat-sink on it for few bucks more, unless is the wrong picture. Should I get one with heat sink ? All older server memory has a nave a nice heat-sinks on them , what do you think ?

 

[Black Ninja]

I order 4 sticks of M393B2G70DB0-YK0 from Supermicro recommended list: Samsung Registered ECC 1600 Mhz 1.35V.

There is no turning back now.

Thanks everybody who helped in RAM discussion. Now we can continue to the next component of our attempt to build the best freenas server.

 

[jgreco]

No, no, no! Disaster! You should have ordered the 1866!

Heh.

 

[Black Ninja]

Perhaps I should of say before placing the order: "Those who have something to say against placing the order should speak now , or silence forever"
After the order is placed, only support I will accept


P.S. Well speaking honestly, I have around 24hours to respond before order it's shipped. Like the last time I cancel my 1650 v2 in favor of 1620 v2 , if you remember.

This ram should be good , I even play scenarios for future use in dual X9 MB , or get a fancy WIO 1U ESXI like yours but on X9 platform and this memory was listed there too, so it will flexible.

 

[Black Ninja]

We're using these as hypervisors. Works out really nice. A nice 2GB LSI 3108 RAID controller for 8 of the bays provides HDD and SSD RAID1's, and two more non-RAID drives for less critical storage. Toss a dual 10GbE card in there and there's STILL a full slot free.

Now you made me jealous again, I was hoping it will go away in a day or two , but it didn't, so tell me more about this 1u WIO hypervisor. What drives you are running in it ? Is it all local storage ?

 

[jgreco]

Yeah, those were 2014's hypervisor design, which I was still having to buy even at the end of 2015, because the Xeon D stuff just wasn't really quite ready for prime time.

The X10SRW is a great platform because it has immense flexibility. The board itself can go either E5-16xx or E5-26xx, of course, which means that you can either do an inexpensive E5-1650v3 (one of the best bang-for-buck in the Xeon lineup) or something crazy on the far side of the E5-269x's like the 2697 v4, which gets you approximately 41GHz of bang, or double the 1650v3. 128GB (4x32) of full speed memory is only ~$800, can be doubled to 256GB but at reduced speed. There's a pricey full speed option too, which probably only makes sense for the E5-26xx's.

On the expansion side of things, three PCIe x8 slots in the 1U is pretty good expansion capability, and there's 5 in a 2U.

I chose to stick a high quality RAID controller (the AOC-S3108L-H8iR) in there with the LSI supercap option, which gives ESXi a nice write cache. The 2GB RAM is a great Supermicro bonus, the LSI 9361-8i only comes with 1GB. This might be unnecessary for the application, because a lot of the storage is actually SSD. Now part of this that you have to remember is that when operating gear at a significant distance (14 hours away in this case), you do things redundantly where possible. So there are three WD Red 2.5" 1TB HDD's, two in RAID1 and a standby. Those are for "slow" bulk storage. Then there's two 480GB datastores made out of 5 Intel 535 480GB's, two sets of two mirrors and a standby. Now the thing is, the 535's have a relatively low write endurance ... 40GB/day. So the system also has a nonredundant Intel S3710 that's rated for 10 drive writes per day. That's what shoulders the noisy stuff.

I had originally intended to go with Intel S3500's for the SSD RAID1 datastores, but you know me, kinda cheap.

So in the end, what's been happening over the years is that we've been slowly downsizing at our locations. As an example, what used to occupy ~8 racks back in the 1990's was downsized to ~2 by the mid-2000's, and now is 97 VM's on a handful of hypervisors, though we still have some physical machines left in addition right now. Those should go away this year, replaced by some more VM's and a NAS box.

 

[Black Ninja]

On the expansion side of things, three PCIe x8 slots in the 1U is pretty good expansion capability, and there's 5 in a 2U.

I just want to say here that on 1u form factor WIO is fantastic - to get 3 expansion cards in it, but for 2u chassis isn't it better to stick with standard board size and have as many as cards as you MB allows , as long as they are low profile since it will be used vertically , yes ?