BUILD Building a server for FreeNas primary for Plex

Status
Not open for further replies.

gpsguy

Active Member
Joined
Jan 22, 2012
Messages
4,472
You probably are using the vSphere Web Client now. Unfortunately, it's a feature you need to pay for - it's not in the free version. That being said, there is no Mac version of the vSphere client. There is one (free) for Windows.

That being said, I believe you could run VMware Fusion on a Mac to access ESXi. List price for it is ~$150 USD.

Any Mac vSphere or access to the Hypervisor?
 

AltecBX

Patron
Joined
Nov 3, 2014
Messages
285
vSpere is going to take a back seat for now. ~So I decided on the processor and just bought it. After reading throughout the forum I'm deciding not to VM this machine for now. I'm basically going to build it and run FreeNAS directly from a USB 3.0 stick. My built starts the following week after the holidays. As of right now, I don't want to take a chance with all my media as I don't have the resource to built another one to clone the data. As hard drive come down in price after the new year, I think I might go ahead and build another machine with VM in mind as well as backing up this current machine. SuperBiiz.com was a great site to get most of the components I needed for this built. Not even eBay, Amazon, or Newegg came close to their prices for server grade parts.

Here are the parts I'm using to built my Plex/Backup/File server on FreeNAS 9.3
Motherboard
- SUPERMICRO MBD-X10SRH-CLN4F-O ATX Server Motherboard LGA 201
Processor - Intel Xeon E5-1650 v3 Six-Core Processor 3.5GHz 0GT/s 15MB LGA 2011-v3 CPU, Retail
Heatsink - Supermicro SNK-P0048AP4 CPU Heatsink For LGA2011
Memory - Crucial 646GB DDR4 2133 MT/s (PC4-2133) CL15 DR x4 ECC Registered DIMM 288-Pin
Flash Drive - Super Talent 16GB USB 3.0 Express ST4 Flash Drive (MLC)
Hard Drive - (10) WD Red WD60EFRX 6TB 64MB Cache SATA 6.0Gb/s 3.5" NAS Hard Drive
Chassis - 3U Supermicro 16 bays Storage Server Chassis SC836E26-R1200 SAS2-836EL2 6Gbs
Rack - Tripp Lite 12U SmartRack WallMount Enclosure Rack Cabinet SRW12US33-800115632

Complete change from my first post on the parts I wanted to buy to build this server.

* Thank you all for all the help in guiding me to the right direction and parts for this build.
 
Last edited:

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
Congrats! I think you'll be happy. I'll race ya, as my chassis arrives next week and I plan to start my build the following week as well. I've been lining up a bunch of burn in test threads that I can share in a separate post. As well as some best practices (AD, permissions, raid types, preventing full-pool lockup).

Also, would you mind sharing the component costs from superbiiz?
 

AltecBX

Patron
Joined
Nov 3, 2014
Messages
285
Congrats! I think you'll be happy. I'll race ya, as my chassis arrives next week and I plan to start my build the following week as well. I've been lining up a bunch of burn in test threads that I can share in a separate post. As well as some best practices (AD, permissions, raid types, preventing full-pool lockup).

Also, would you mind sharing the component costs from superbiiz?
Yes I would like that. Let me know when your ready.
Also I notice that your building yours with the same board. How much was your rig and breakdown?

Here's my list.
Motherboard
- SUPERMICRO MBD-X10SRH-CLN4F-O $389.99
Processor - Intel Xeon E5-1650 v3 Six-Core Processor 3.5GHz 15MB LGA 2011-v3 $569.99
Heatsink - Supermicro SNK-P0048AP4 CPU Heatsink For LGA2011 $29.99
Memory - (4) Crucial 16GB DDR4 (PC4-2133) CL15 DR x4 ECC Registered DIMM 288-Pin $743.96
Flash Drive - Super Talent 16GB USB 3.0 Express ST4 Flash Drive (MLC) $24.00
Shipping - $26.00
SuperBiiz Total - $1,783.93
Hard Drive - (10) WD Red WD60EFRX 6TB 64MB Cache SATA 6.0Gb/s 3.5" NAS Hard Drive NewEgg - $2,799.90
Chassis - 3U Supermicro 16 bays Storage Server Chassis SC836E26-R1200 SAS2-836EL2 6Gbs eBay Total - $425.00
Rack - Tripp Lite 12U SmartRack WallMount Enclosure Rack Cabinet SRW12US33 eBay Total - $255.00

All together this build came out to $5,263.83

I think if some users recycle their old hard drives and go with a regular case, you can spend under $1,800 on really great server hardware. I think it's time for an updated Hardware recommendations thread.
 
Last edited:

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
This is what I went with. I don't have a component price, since I bought it assembled from Ace. Includes DDR4 (which drove the cost up compared to the "spoiler" config at the bottom), SAS3 and E5 v3 but is single CPU (which I did to prevent me from trying to do a combined ESX/FN box):
Chassis/motherboard - 1x SSG-5028R-E1CR12L SUPERMICRO Super Storage Server - 12 drive barebones - Single Socket E5-2600 v3 w/ X10SRH-CLN4F motherboard
misc chassis add-on - 1x MCP-220-82609-0N Rear 2x 2.5" Hot-Swap Hard Drives / SSD's carriers
Memory - 8x MEM-DR416L-HL01-ER21 SUPERMICRO 16GB DDR4 2133MHz ECC RDIMM
CPU - 1x P4X-DPE52637V3-SR202 SUPERMICRO / INTEL Intel Xeon E5-2637 v3 (4C, 3.5GHz, 15MB)
Total Server: $4,409.00 (Ace Computers)

ZIL/L2ARC - 2- Intel DC S3700 2.5-Inch 200 GB Internal Solid State Drive SSDSC2BA200G301 $378 $756
Data - 12-WD RE SAS 4 TB Enterprise Hard Drive: 3.5 Inch, 7200 RPM, SAS, 32 MB Cache - WD4001FYYG $228.72 $2,744.64
OS - 1- SanDisk Cruzer Fit CZ33 32GB USB 2.0 Low-Profile Flash Drive- SDCZ33-032G-B35 $15.50 $15.50
Total Storage: $3,516.14 (Amazon)

Grand total: $7,925.14

I also picked up a Dell SAS/SATA 2.0 6Gb/s 8-Port 2x4 PCI-e HBA LSISAS2008-IT from ebay for $80, just in case I can't cross-flash the on-board SAS controller to IT-mode.

The plan is to daisy-chain additional storage-only chassis' as needed off this unit. I'm also looking into the possibility of a SATA DOM, but want to see how much room there is to figure out which orientation to get.

This was the first server config. I didn't end getting this one though.
1- SSG-6027R-E1R12L Super Storage Server - Barebones $1,421.88 $1,421.88
8- MEM-DR316L-HL01-ER18 16GB DDR3 ECC RDIMM 1866MHZ (128GB) $150.35 $1,202.80
2- P4X-DPE52637V2-SR1B7 Intel Xeon E5-2637 v2 $973.08 $1,946.16
1- MCP-220-82609-0N Rear 2x 2.5" Hot-Swap Hard Drives / SSD's $51.00 $51.00
1- FREIGHT Ground Freight $60.00 $60.00
Total $4,681.84 (Ace Computers)
 

AltecBX

Patron
Joined
Nov 3, 2014
Messages
285
What do you guys think of me adding ZIL/L2ARC drives for my setup? Think is worth the performance? If so, what drives do you recommend since many places have black friday deals. MLC or SLC drives? Should the ZIL be smaller and the L2ARC be a bigger drive? So many questions regarding setting this up if necessary.
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
Zil/slog is useful for sync writes. You only need an 8-32GB ssd, but it needs to be incredibly fast at writes (SLC with battery/capacitor backup). Generally you use a larger capacity SSD (64-128GB and only partition part of it to increase the lifespan.

L2ARC is useful for caching files. I recall needing at least 64GB of RAM to handle the cache index/metadata. There is an equation somewhere.
 

depasseg

FreeNAS Replicant
Joined
Sep 16, 2014
Messages
2,874
You can easily add them later. Try it without, check the performance, and there is a cache statistic you can check to see if an L2ARC would be helpful.

In short, unless you know you need one, you don't need one. :smile:
 

AltecBX

Patron
Joined
Nov 3, 2014
Messages
285
You can easily add them later. Try it without, check the performance, and there is a cache statistic you can check to see if an L2ARC would be helpful.

In short, unless you know you need one, you don't need one. :)
Thanks depasseg. Figured some sites had sales today and this weekend for black friday and cyber monday, but I guess I'll hold off to see if I need to go this route or not.
 

Tywin

Contributor
Joined
Sep 19, 2014
Messages
163
Thanks depasseg. Figured some sites had sales today and this weekend for black friday and cyber monday, but I guess I'll hold off to see if I need to go this route or not.

It's only a deal if you were going to buy it anyway ;)
 

AltecBX

Patron
Joined
Nov 3, 2014
Messages
285
Ok guys. Got the board in today and just finish putting the board, memory, and cpu on the chassis. Put in 10 hd drives in the 16 bay chassis as well. A few questions?

1. I have one cable coming from my backplane. You can see in the picture it's a blue cable. Where the hell do I connect this too?
2. In the other picture you can see it has 6 connections in the backplane (BPN-SAS2-836EL2 Backplane). Again what to do?
3. The blue cable doesn't look like it fits the MB LSI 3008 connection. WTF?
4. I put 4 Dimm of 16GB each on A1, A2, B1, and B2. It's that my best performance setup?
 

Attachments

  • IMAG0534.jpg
    IMAG0534.jpg
    156.9 KB · Views: 389
  • IMAG0536.jpg
    IMAG0536.jpg
    231.7 KB · Views: 368
  • IMAG0537.jpg
    IMAG0537.jpg
    168.3 KB · Views: 370
Last edited:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
Oh, right. SAS 3 uses newer connectors.

You need to buy an SFF-whateverthenewconnectoris to SFF8087 cable. Signalling is backward compatible, but they felt the need for new connectors.

The backplane doesn't seem to have an expander, so you'll either need to get one or connect whatever drives are left to the PCH SATA ports using a reverse breakout cable.

As for the RAM, that's doubtful. Read the motherboard's manual and do as they say.
 

AltecBX

Patron
Joined
Nov 3, 2014
Messages
285
Oh, right. SAS 3 uses newer connectors.

You need to buy an SFF-whateverthenewconnectoris to SFF8087 cable. Signalling is backward compatible, but they felt the need for new connectors.
- My motherboard has a LSI 3008. It looks like it has two ports out. Do you think I need this cable here (Mini SAS High Density HD SFF-8643 TO SFF8087 )?
Will one cable do?

50cm-Internal-Mini-SAS-SFF-8087-to-Mini-SAS-SFF-8643-High-Density-HD-Data-Cable.jpg


The backplane doesn't seem to have an expander, so you'll either need to get one or connect whatever drives are left to the PCH SATA ports using a reverse breakout cable.
- I thought the SAS2-836EL2 was a Backplane 6Gbs Expander
- The backplane (BPN-SAS2-836EL2 Backplane) has 6 SFF8087 connections (PRI_J0), (PRI_J1), (PRI_J2), (SEC_J0), (SEC_J1), (SEC_J2). How do I hook up the LSI 3008 to see all 16 drives in the bay? Will 1 cable(SFF-8643 to SFF8087) from the LSI 3008 to the backplane do? Maybe to port PRI_J0?


As for the RAM, that's doubtful. Read the motherboard's manual and do as they say.
- As for the RAM, the manual is not that clear. It just says to start at A1. I figured since this community mostly use Supermicro boards, they will have more insight on this.

This is the chassis that I have.
(http://www.supermicro.com/products/chassis/3U/836/SC836E26-R1200.cfm?parts=SHOW)

I'm attaching the pdf to the backplane to see if someone understands it better than I can.
(http://www.supermicro.com/manuals/other/BPN-SAS-836EL.pdf)
 
Last edited:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194

AltecBX

Patron
Joined
Nov 3, 2014
Messages
285
That oughtta work.
Cool. Do you mean getting 1 (Mini SAS High Density HD SFF-8643 TO SFF8087) cable and hooking it to what port on the Backplane? The PRI_J0 port?
The LSI 3008 says it supports 8 SAS drive. 4 drives through each port. How will it see and handle all 16 drives through that one cable?
 

mjws00

Guru
Joined
Jul 25, 2014
Messages
798
All the extra ports on the backplane are for cascading and redundancy. You actually have two expanders as far as I can tell with a quick glance at the specs. The single 8087 port will let you see all the drives. It drives the expander, the expander signals the drives. I'd just unplug the existing cable and plug the new 8643 to 8087 cable in there. Truth is all the ports would likely work... but PRI_0 should hit all the defaults.

It is a little odd they don't specify any order on the ram. I'd probably hit a1a2 and c1c2 on the other side of the cpu out of habit unless I could find better information.

Looks nice.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
6 connectors for failover? Doesn't seem likely. Typically there's two, maybe four. No harm in trying, though.

As for RAM, I'd do A1 B1 C1 D1, if A and B are on one side and C and D on the other.
 

mjws00

Guru
Joined
Jul 25, 2014
Messages
798
I hear ya, Eric. It is a pretty spiffy case with dual expanders as far as I can tell. Your memory layout was my other thought and may be the ticket.

I'm happy to be wrong. Sitting on the lanai watching the surf pound wasn't inspiring deeper reading. ;)
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
Talk about odd design choices... Four upstream channels per expander plus eight for cascading?
 

AltecBX

Patron
Joined
Nov 3, 2014
Messages
285
So I just got off the phone with SuperMicro. They said the LSI 3008 cannot be used with a expander; nor can't it recognize the hard drives with a Mini SAS High Density HD SFF-8643 TO SFF8087 cable .
They told me my best bet is to get a SAS2 controller. They recommended 2 that will work with my Backplane expander. A AOC-S2208L-H8iR or a SAS2LP-H8IR. He said they both are LSI controller.
MJWS00 yo were right, he told me that (PRI_J0), is the primary connection and (PRI_J1) is for additional bandwidth. (PRI_J2) is for daisy chain to other chassis, (SEC_J0), (SEC_J1), and (SEC_J2) are basically the same as the primary but for redundancy with an additional controller card..

So now guys, what do I do? I really don't want t keep shelling out money. What route do you suggest or what controller do you recommend?


ps - For anyone with this board or similar from SuperMicro. They recommend me to leave the memory exactly how i had it. A1, A2, B1, B2. basically on one side of the CPU. This way it gives me the best performance and it operates in Memory Interleaving. Any other way, it won't do Memory Interleaving.
 
Last edited:
Status
Not open for further replies.
Top