SUPERMICRO 4U 846E16-R1200B 2x Xeon L5640, 48GB RAM good starter rack server?

[Murac]

How's it going people. A few days ago I was researching simple little builds for a pfsense firewall to add some proper protection to my home network which contains my FreeNAS build which is:

AsRock C2550D4I
6x4TB WD RED in RAIDZ2
16gb ECC RAM

And before long, I got to thinking how I was already at 80% of the tank volume, and plex really loves to transcode, so what if I finally switched over to one of those professional looking rack servers to handle the growth of my pool and use the extra resources after the CPU upgrade for some VMs to accommodate my personal projects. I admit I'm green when it comes to this enterprise hardware but knew of all the used parts on ebay and found this: http://www.ebay.com/itm/SUPERMICRO-...444539?hash=item281ecd053b:g:L5gAAOSwV0RXu0Ia

SYSTEM SUPERMICRO: 4U 846E16-R1200B
CHASSIS: CSE-846
HDD BACKPLANE: SAS2-846EL1
SYSTEM BOARD: X8DTE-F
PROCESSOR: DUAL INTEL XEON L5640 2.26GHZ SIX CORE CPU WITH 12MB CACHE
MEMORY 48GB MEMORY ( 12X 4GB) DDR-3 ECC
HARD DRIVE NO HARD DRIVES INSTALLED (24X TRAYS W/ SCREWS)
RAID CONTROLLER LSI 9210-8I ARRAY CONTROLLER
POWER SUPPLIES DUAL 1200W PWS-1K21P-1R POWER SUPPLIES
RAILS RAILS INCLUDED

-24-bays won't be full for atleast a decade.
-the L5640 passmark score is 6584 compared to the C2550's 2329, and half the CPUs. Definitely room to grow despite the age

This this is supposedly loud as hell and I haven't figured out where I'm going to put it yet. Challenge accepted.

From what I understand this is a decent price for what i'm getting and I've heard good things about the chassis.

There are two things I'm concerned about:

1) Is the LSI 9210-8I ARRAY CONTROLLER compatible with freenas? I see lots of talk about the 9211 and the 9201 being either compatible or flashable but not much about this model. I'm okay with just replacing this component if that's what it comes to. Any suggestions?

2) Once I increase my pool to include 12 total 4TB drives, will 48GB of RAM be a bottle neck if plex is transcoding 5+ streams simultaneously plus whatever the VMs are doing? (everything in the VMs will be light weight, atleast to start, input simple sandbox use case here)

Any and all criticism or insight is welcome.

Anyways, back to pfsense. I'll probably run it on the machine i already have despite it beyond way beyond what I would need for a home network.

Thanks!

 

[MatthewSteinhoff]

The X8DTE-F motherboard support 192 GB of RAM. You'll have to rip the 12, 4GB sticks out and replace them with 16GB sticks to get there. The 8GB sticks are a lot less expensive so, if you're on a budget, consider your upper limit 96 GB of RAM. I don't know enough about Plex's memory usage to know how much you need to transcode five streams and host your virtual machines. From a straight NAS perspective, 48TB is a good amount for that much storage space.

In addition to the noise, you may be surprised at how much heat this beast produces and how much power it consumes.

Cheers,
Matt

 

[Spearfoot]

How's it going people. A few days ago I was researching simple little builds for a pfsense firewall to add some proper protection to my home network which contains my FreeNAS build which is:

AsRock C2550D4I
6x4TB WD RED in RAIDZ2
16gb ECC RAM

And before long, I got to thinking how I was already at 80% of the tank volume, and plex really loves to transcode, so what if I finally switched over to one of those professional looking rack servers to handle the growth of my pool and use the extra resources after the CPU upgrade for some VMs to accommodate my personal projects. I admit I'm green when it comes to this enterprise hardware but knew of all the used parts on ebay and found this: http://www.ebay.com/itm/SUPERMICRO-4U-846E16-R1200B-24-BAY-X8DTE-F-2x-L5640-48GB-24x-TRAYS-LSI-9210-8I-/172315444539?hash=item281ecd053b:g:L5gAAOSwV0RXu0Ia

SYSTEM SUPERMICRO: 4U 846E16-R1200B
CHASSIS: CSE-846
HDD BACKPLANE: SAS2-846EL1
SYSTEM BOARD: X8DTE-F
PROCESSOR: DUAL INTEL XEON L5640 2.26GHZ SIX CORE CPU WITH 12MB CACHE
MEMORY 48GB MEMORY ( 12X 4GB) DDR-3 ECC
HARD DRIVE NO HARD DRIVES INSTALLED (24X TRAYS W/ SCREWS)
RAID CONTROLLER LSI 9210-8I ARRAY CONTROLLER
POWER SUPPLIES DUAL 1200W PWS-1K21P-1R POWER SUPPLIES
RAILS RAILS INCLUDED

-24-bays won't be full for atleast a decade.
-the L5640 passmark score is 6584 compared to the C2550's 2329, and half the CPUs. Definitely room to grow despite the age

This this is supposedly loud as hell and I haven't figured out where I'm going to put it yet. Challenge accepted.

From what I understand this is a decent price for what i'm getting and I've heard good things about the chassis.

There are two things I'm concerned about:

1) Is the LSI 9210-8I ARRAY CONTROLLER compatible with freenas? I see lots of talk about the 9211 and the 9201 being either compatible or flashable but not much about this model. I'm okay with just replacing this component if that's what it comes to. Any suggestions?

2) Once I increase my pool to include 12 total 4TB drives, will 48GB of RAM be a bottle neck if plex is transcoding 5+ streams simultaneously plus whatever the VMs are doing? (everything in the VMs will be light weight, atleast to start, input simple sandbox use case here)

Any and all criticism or insight is welcome.

Anyways, back to pfsense. I'll probably run it on the machine i already have despite it beyond way beyond what I would need for a home network.

Thanks!

Before you make a decision, consider the special FreeNAS-ready Dell C2100 FS12-TY system that @Mirfster has cajoled DFW Parts Depot into offering for $389.99 - more details at this post in the Ode to the Dell C2100/FS12-TY thread. Enter ultra-double-top-secrect coupon code 'freenas' in the checkout cart for a 10% discount.

EDIT: If you prefer Supermicro gear, just search for "supermicro freenas" on eBay and you'll find dozens of hits...

 

[Mirfster]

This this is supposedly loud as hell and I haven't figured out where I'm going to put it yet. Challenge accepted.

Yeah, that would be a "Turn off" for me.

MEMORY 48GB MEMORY ( 12X 4GB) DDR-3 ECC

Check the motherboard to see how many slots it has. Since they are using 4GB Sticks, that is what others call "Slot Stuffers". Better to get 8GB or 16GB Sticks which will allow room for growth.

1) Is the LSI 9210-8I ARRAY CONTROLLER compatible with freenas? I see lots of talk about the 9211 and the 9201 being either compatible or flashable but not much about this model. I'm okay with just replacing this component if that's what it comes to. Any suggestions?

Per Avago, it is a HBA. Has the same chipset (LSI SAS2008) as the other more popular HBAs (9211-8i, Perc H200, etc.). Only real difference I see is that it is a card normally "Only available to OEMs through direct sales".

I *think* it can be used properly in FreeNAS, but not 100% sure. Heck it looks almost identical to a Dell Perc H200.

2) Once I increase my pool to include 12 total 4TB drives, will 48GB of RAM be a bottle neck if plex is transcoding 5+ streams simultaneously plus whatever the VMs are doing? (everything in the VMs will be light weight, atleast to start, input simple sandbox use case here)

As mentioned earlier, here is where those "Slot Stuffer" 4GB Sticks are going to become a nuisance.

 

[Dice]

This this is supposedly loud as hell and I haven't figured out where I'm going to put it yet. Challenge accepted.

It is a substantial challenge.
I'd say before establishing a location/setup that is functional, has decent air turnover or enough air space (a garage for instance, not a closet) and testing it by putting a regular vaccum cleaner or something similar in noise levels - carefully listening in other habited areas of the house and really thinking through if u'd want that type of nosie 24/7.
When that step is done - go all in for a rack solution. All day - every day.
(I failed at the first step, and have a similar SM box just sitting around waiting for a second order attempt at having that machine running while at home)

 

[Murac]

Thank you everyone for your replies thus far. These are some very sensible suggestions an excited mind doesn't always take into account.

Is it safe to keep a rackserver in the garage? I live in south florida, and the garage is not ventilated, but there is certainly plenty of room in there. I'd just have to run cat6 through the house from the router. If the answer to this question is yes, I may be good to go.

@Spearfoot that box looks great but I've heard such conflicting views about the C2100 all over the place. It's been a few months since he purchased the build and still hasn't provided much feedback other than "it's running." Which i guess is always a victory...


No one has commented much on the processors themselves. I suppose those are sufficient to get the job done, eh?

Live long and prosper

 

[MatthewSteinhoff]

I, too, live in South Florida. My garage is not air-conditioned and ends up hot and humid, two factors which will hurt your server's longevity.

Cheers,
Matt

 

[Spearfoot]

Thank you everyone for your replies thus far. These are some very sensible suggestions an excited mind doesn't always take into account.

Is it safe to keep a rackserver in the garage? I live in south florida, and the garage is not ventilated, but there is certainly plenty of room in there. I'd just have to run cat6 through the house from the router. If the answer to this question is yes, I may be good to go.

@Spearfoot that box looks great but I've heard such conflicting views about the C2100 all over the place. It's been a few months since he purchased the build and still hasn't provided much feedback other than "it's running." Which i guess is always a victory...


No one has commented much on the processors themselves. I suppose those are sufficient to get the job done, eh?

Live long and prosper

Unless your garage is climate-controlled I wouldn't even consider running a server in it. It's hot and humid in Florida; both are very hard on electronic equipment. Servers run best where it's nice and cool... mine are in my shop, which is climate-controlled, and stays at a steady 70 degrees F in the Summer and 66 degrees F in the Winter.

Regarding the Dells... @Mirfster is an enthusiast and will gladly answer any questions or address any concerns you may have. I would buy one myself, just to tinker with... except I've already got two FreeNAS systems and a Synology, to boot! I just can't justify another NAS to myself - or my wife! :)

 

[Mirfster]

@Spearfoot that box looks great but I've heard such conflicting views about the C2100 all over the place. It's been a few months since he purchased the build and still hasn't provided much feedback other than "it's running." Which i guess is always a victory...

Yeah, sorry about that. I am pretty busy with other stuff to get the full review done. However, I am very happy with the systems. Fits well with all my other ones. :)

 

[Murac]

Maybe I approached this the wrong way. Here's my goal, maybe one of you can atleast point me in the right direction. All I want is atleast a 12 bay enclosure to add another 6 drives to my pool and a build that could handle 8 plex transcodes instead of 4. Yeah I really want the option to run some VMs but damnit, cost/noise ratio really starts to run away at that point it seems. Would the C2100 deal suffice here? I'd rather not spend more than $700 but if the difference between complete replace and upgrade, and component upgrade for the future is another $300, I could justify that. Of course, I begrudgingly must accept that servers are loud and this adds another layer of "fuck."

But yeah, the generosity and patience I find even on these notoriously pretentious forums is abundant. I love this community.

 

[Stux]

FWIW, I went with a Norco RPC-4224 chassis. I wouldn't recommend one for a number of reasons, but one thing in their favor was that the 4U chassis used a 120mm fan wall and they used standard pc components.

This meant I could build a silentish server. And I did by upgrading all the fans etc. now the noisiest component in my rack is the gbe switch.

(Need to replace the fan, it's started whirring)

I use an e5-1650v4, and it runs at up to 70c flat out with a 140W TDP. Yes the fans crank up for that, but the rest of the time they're idle and it's inaudible.

 

[Mirfster]

Would the C2100 deal suffice here?

You can always opt to get more RAM and higher CPUs (Highest is the X5690); but I don't use Plex and don't really know what you would need as far a CPU power. I'll leave that to others.

There are tons of CPUs that it supports, so plenty to pick and choose from.
Intel Xeon 5500 Series Family
Intel Xeon 5600 Series Family

 

[Chuck Remes]

Plex transcodes, as a rule of thumb, require a Passmark of 2000 per stream. That is documented here: https://support.plex.tv/hc/en-us/articles/201774043

You can check out the passmark ratings of various cpus here: https://www.cpubenchmark.net/cpu_list.php

The C2100 deal is using an older and slower xeon. Combined they have a passmark rating of around 10k which would be enough for 5 streams (maxed out). If you want to do 8 streams and anything else on that box then you will want to upgrade the CPUs. Good luck.

 

[MatthewSteinhoff]

At Passmark 2000 per stream, eight streams is an expensive proposition. You're going to blow your $700 budget on CPU power alone.

Cheers,
Matt

 

[ChriZ]

With 2 x5690 xeons there is plenty horsepower for 8 streams - even 10 I reckon, but as @MatthewSteinhoff said, it will cost more than your $700 budget, since a matched pair of them would cost half that money.
However you can go with the Dell, see how much it can handle and upgrade the CPUs later when you find a good deal for them...
However I believe that you can handle 8 streams even with a pair of x5660 CPUs, which are fairly cheap, but don't quote me on that, neither, of course spend your money based on my estimates.
Perhaps someone with similar CPUs can tell how much they can handle..

 

[Spearfoot]

Eight simultaneous Plex transcodes? Are you setting up a media server for your apartment building?

 

[Mirfster]

Meh, I still run PopCornHour A-110 driveless boxes for my TVs and YAMJ (Yet Another Media JukeBox)... :P

 

[droeders]

FWIW, I went with a Norco RPC-4224 chassis. I wouldn't recommend one for a number of reasons, but one thing in their favor was that the 4U chassis used a 120mm fan wall and they used standard pc components.

Not to hijack this thread, but I'd be interested to hear your thoughts on the Norco RPC-4224. I've been considering one for my backup server.

I've got Supermicro gear for my main NAS, but it's pretty loud.

 

[Stux]

The biggest issue I had was their rack rails don't work in a standard 4 s-post rack.

Will post a detailed review soonish.

 

[Mlovelace]

I have two Norco RPS-4224s and they are just fine. No issues with temps but I swapped out the fans (120mm and the 80mm).