New FreeNAS build

[Valdhor]

Well, if @Mikey J hasn't bought anything yet I certainly have. That motherboard on eBay was too good to pass up (https://www.ebay.com/itm/132762499937). I went for Cisco (Hynix) memory (https://www.ebay.com/itm/323400067110) as that module has been tested with this motherboard and you can make an offer to get a few bucks taken off.

I'll probably go with the Intel Xeon E5-2650 V2 2.6GHz 8 Core (https://www.ebay.com/itm/113166298794) as this should be fine with Plex, Sonarr and SABnzbd. Do you think a E5-2680 v2 would be more suitable for double the cost?

I already have a Define R5 case, EVGA Supernova 550 G3 and M1015 SAS card. I also have 12 x WD Red drives for a RAID Z3. I guess I should take pictures and make a build thread.

 

[Chris Moore]

I guess I should take pictures and make a build thread.

Sure, I love to see a good build thread. I only wish I was better at taking photos so I could do one.

I also have 12 x WD Red drives for a RAID Z3.

You should use one of these SAS expander cards so you can run all those drives from your SAS controller.
https://www.ebay.com/itm/NEW-IBM-46...ion-Adapter-16-Port-SAS-Expander/132448152088

M1015 SAS card

Did you flash it to the IT firmware from LSI?

 

[Valdhor]

Sure, I love to see a good build thread. I only wish I was better at taking photos so I could do one.

I'm no photographer. I'll just be using my iPhone camera so we'll see how good I am.

You should use one of these SAS expander cards so you can run all those drives from your SAS controller.
https://www.ebay.com/itm/NEW-IBM-46...ion-Adapter-16-Port-SAS-Expander/132448152088

I was just going to use four of the ports on the motherboard. Is using the expander better?

Also, I tried finding a manual for that card. How is the cabling meant to work? I think the top two connectors are inputs and would connect to the two ports on the M1015. Then the four ports on the right are outputs and would connect to the 12 drives (Using just three of the ports). So a couple of these cables (https://www.ebay.com/itm/231337420398) and three SAS to SATA breakout cables should do the trick, right?

Did you flash it to the IT firmware from LSI?

Yes, although that was a while back. I will check the firmware version during the build.

 

[Chris Moore]

I was just going to use four of the ports on the motherboard. Is using the expander better?

I found my storage system worked faster with all the drives on a single controller. I was using two controllers before, but my new server chassis included a SAS expander, so I did a comparison of the performance. It was not very scientific, but the amount of time it takes to run a scrub dropped by an hour with all the drives on a single controller with an expander vs when the drives were on two separate controllers.

Also, I tried finding a manual for that card. How is the cabling meant to work? I think the top two connectors are inputs and would connect to the two ports on the M1015. Then the four ports on the right are outputs and would connect to the 12 drives (Using just three of the ports). So a couple of these cables (https://www.ebay.com/itm/231337420398) and three SAS to SATA breakout cables should do the trick, right?

Exactly. I used cables like this to connect from the SAS card to the expander card:
https://www.ebay.com/itm/Mini-SAS-3...AS-36-Pin-SFF-8087-Data-Cable-1M/201484956159
It is one meter (three feet) and I kind of wish I went with a shorter cable, but it isn't a problem.

As for the breakout cable, I used this kind the first couple server builds I did:
https://www.ebay.com/itm/2Pcs-Mini-...IN-HD-Splitter-Breakout-Cable-1m/202067758247
I found them to be a little on the fragile side because they are so thin. I managed to break one. The latest set I bought is this style and they are much more durable, so far:
https://www.ebay.com/itm/Lot-of-2-M...-Forward-Breakout-Internal-Cable/371681252206
Just check the length you need so you don't come up short. The set I bought was barely long enough to reach the drives.

 

[Mikey J]

Chris, Yes I will be building a new system soon. I will likely go with a lot of the hardware you suggested. I think I will do 11 x 6TB drives in Z3. I do plan on going with at least 64GB of memory, rather than my current 32GB.

Valdhor, How do you plan to fir 12 drives in a Define R5?

 

[Valdhor]

Valdhor, How do you plan to fit 12 drives in a Define R5?

The Define R5 has eight 3.5 bays and two 5.25 bays. I intend to swap out the 5.25 bays with a EverCool Dual 5.25 in. Drive Bay to Triple 3.5 in. HDD Cooling Box. This will give me another three 3.5 bays. There should be enough space inside the box to fit another 3.5 somewhere (Even if I don't have a bracket I can velcro it to the side of the drive bays).

Best Regards
--Steve

 

[IQless]

(Even if I don't have a bracket I can velcro it to the side of the drive bays)

This might restrict your airflow quite a bit, if you need to velcro it somewhere, It might be best to place it on top of the PSU or something.

 

[Chris Moore]

Just my thoughts on RAIDz3. I tried it before, and I don't use it now because I found it to be slow. I have 12 drives in my main pool, but it is two vdevs of six drives each at RAIDz2. It means I have four drives for redundancy instead of 3, and you might think of that as wasted space, but it also means I have two vdevs instead of one and that gives me more IOPS, and that means the pool is faster and more responsive.

 

[Valdhor]

This might restrict your airflow quite a bit, if you need to velcro it somewhere, It might be best to place it on top of the PSU or something.

I was somewhat joking with the velcro remark. That was kinda a last resort kind of thing.

I'm fairly sure I can get a bracket that would fit somewhere. I am planning on adding other cooling fans (Already have an extra Noctua for the front) as the case allows (There are many fan mounting points in the Define R5).

 

[Valdhor]

Seeing as the 550G3 is a short power supply there is enough room in the bottom of the case for a 120/140mm fan. I found the ICY DOCK SATA Hard Drive Mobile Rack and swapped out the 80mm fan for a 120mm fan. Now my 12th HDD has its own fan.

The final item that I need and would like recommendation for is the CPU cooler. I'm looking for something that's close to silent as noise level is what this build is about. I'm going for either the Intel Xeon E5-2650 V2 or the E5-2680 v2 (Still debating in my head). I was looking at the Noctua NH-D15 but wonder if that's overkill. Then I looked at the Cooler Master Hyper 612 and the be quiet! BK009.

Any recommendations would be most appreciated.

 

[anmnz]

Be aware that the X9SRL-F has a "narrow ILM" heatsink mount, not the larger "square ILM" that is more common with LGA2011 sockets.

I have a Noctua NH-U12DX i4 on that motherboard, with an (EDIT: fixed typo here) E5-2680 v2 CPU, and it's great. Very very quiet nearly all the time, keeps the temperature well under control during stress testing. Fits fine in the Define R5 case.

The only slight caveat I found is this. You can mount the cooler either horizontally so the airflow is towards the back of the case or vertically so the air flows toward the top. I prefer the horizontal arrangement -- might be wrong but I think it will give better airflow when the top of the case is closed, and I keep it closed to minimize noise.

And in the horizontal arrangement the cooler blocks the inner 2 RAM slots. (EDIT: corrected detail here) If you're populating the RAM slots in the recommended order, those belong to the 3rd and 4th DIMM pairs, so the cooler will get in the way only if you're using more than 4 DIMMs. Not a problem for me.

 

[Valdhor]

Be aware that the X9SRL-F has a "narrow ILM" heatsink mount, not the larger "square ILM" that is more common with LGA2011 sockets.

I have a Noctua NH-U12DX i4 on that motherboard, with an E2-2680 v2 CPU, and it's great. Very very quiet nearly all the time, keeps the temperature well under control during stress testing. Fits fine in the Define R5 case.

The only slight caveat I found is this. You can mount the cooler either horizontally so the airflow is towards the back of the case or vertically so the air flows toward the top. I prefer the horizontal arrangement -- might be wrong but I think it will give better airflow when the top of the case is closed, and I keep it closed to minimize noise.

And in the horizontal arrangement the cooler blocks the inner 2 RAM slots. If you're populating the RAM slots in the correct order those are the 5th and 6th, so it will only get in the way if you're using more than 4 DIMMs. Not a problem for me.

Now that's the kind of info I need. Thank you sir.

I'm really only going to populate the four blue slots (Interleaved memory) with 4 x 16GB RAM. Hopefully I should be fine.

Thank you for the recommendation. I will almost certainly use the Noctua NH-U12DX i4.

 

[anmnz]

 

[Valdhor]

I have a Noctua NH-U12DX i4 on that motherboard, with an E2-2680 v2 CPU

Of course I did make the assumption that you meant E5-2680 V2 ;)

 

[anmnz]

Of course I did make the assumption that you meant E5-2680 V2 ;)

Yes. Sorry, typo. I make lots of typos when on mobile. :-/ Fixed.

 

[Stux]

I was somewhat joking with the velcro remark. That was kinda a last resort kind of thing.

You might’ve been joking, but I’ve successfully used 3M command strips in similar applications a number of times.

 

[pashairfan]

Updated Links on: 4 Aug 2018.

These are older components, but still very powerful.
I use similar components in both my 48 bay primary NAS and in my 24 bay backup NAS that runs ESXi with FreeNAS in a VM.

This one comes with a CPU and fan, but it is really slow, so probably want to replace it:

Supermicro X9SRL-F Motherboard LGA2011 System Board w/Intel E5-2650L 0 @ 1.80GHz
https://www.ebay.com/itm/132762499937
Price: US $174.99

Nothing special about this, and the price is higher, but an option:

Supermicro X9SRL-F Motherboard Socket LGA2011 System Board w/ I/O Shield
https://www.ebay.com/itm/401593992194
Price: US $199.99


SAMSUNG 16GB PC3L-12800R DDR3-1600 ECC Registered 1.35V RDIMM...
https://www.ebay.com/itm/302606459277
Price: US $44.95 x 8 = $359.60 for 128GB of RAM !!

note: You could go for the 32GB memory modules, but they are more than twice the price.

If the board above doesn't come with a cooler, or you want a better one, I use this model on two of my systems.
It is only slightly louder than the Noctua cooler I have on my wife's desktop PC.

Dynatron R27 Side Fan CPU Cooler 3U for Intel Socket LGA2011 (Narrow ILM)
https://www.ebay.com/itm/401284811045
Price: US $39.59

You have a lot of options for CPUs to go in this board, but I recently bought one of these for my system:

PassMark score of 13073... If you are wondering... This is the model I use for the NAS I run Plex in.

Intel Xeon E5-2650 V2 2.6GHz 8 Core 20MB 8GT/s SR1A8 LGA2011 ( Ivy Bridge ) Processor
https://www.ebay.com/itm/113166298794
Price: US $91.99

Only it was $30 more when I bought it... It works great. Plenty of resources for all the things I am doing.

If you want more, you can get a 10 core model like this:

Intel Xeon E5-2680 v2 Ten Core 2.8GHz 25MB CPU PROCESSOR LGA2011 SR1A6
PassMark score of 15796... If you are wondering...
https://www.ebay.com/itm/192637399687
Price: US $189.99

For the drive controller, I would suggest a SAS controller, there are 4 SCA ports on the system board in addition to the SATA ports, but only two of the SATA ports are SATA III, so you will still be a little short.

SAS PCI-E 3.0 HBA LSI 9207-8i P20 IT Mode for ZFS FreeNAS unRAID
https://www.ebay.com/itm/162862201664
Price: US $69.55

I would suggest one of these SSDs for the boot drive.
These are used data-center drives, but as a boot drive in FreeNAS, it should last as long as the server, if not longer:

Intel SSD DC S3500 Series 2.5" SATA 6Gb/s, 20nm, MLC 80GB
https://www.ebay.com/itm/183416267073
Price: US $29.99

Just rough math in my head, I think that is all under $1000.... Still, you will need some odd bits like the case and power supply, but it should be simple to get there from here and this should save you a buck or two vs buying new and still do the job.

Hi Chris,
Thanks for the list up there. I have been reading the forums lately and going to be building my first ever server. I am going with the stuff you have suggested above. I have ordered the board and the processor above. I had a question regarding the SAS controller. I see that you have mentioned two different controllers on this forum one is the Dell H310 and the other is the HP H220. Is there any difference between the two ? which one is better of the two? Can i pick anyone of the two or do I have to go with a specific one.

I still need to order the ram and the cpu cooler can you point me at any good deals that would be compatible with this configuration. I got the Intel Xeon E5-2650v2 2.6GHz CPU.

Lastly since I am ordering used parts for the first time. Do i need to order any screws etc for putting these things together or they come with the screws. Sorry for the Newbie questions this is my first ever server build with Freenas.

 

[Chris Moore]

I see that you have mentioned two different controllers on this forum one is the Dell H310 and the other is the HP H220. Is there any difference between the two ?

Yes, the chipset in the HP H220 is newer and it uses PCIe 3.0 where the Dell H310 is older and uses PCIe 2.0. If I recall correctly, there are some minor differences beyond that but I don't recall what all of them are. I have observed both in operation, in my systems, and the newer cards appear to run a few degrees cooler. I have moved from the older card to the newer card in both of my personal FreeNAS systems, but either card is a perfectly valid option.

I still need to order the ram and the cpu cooler can you point me at any good deals that would be compatible with this configuration. I got the Intel Xeon E5-2650v2 2.6GHz CPU.

If you got the system board I am thinking you got, it needs a 'narrow ILM' cooler. This is the model I bought to use on both of my systems that take the narrow ILM cooler and I am very happy with them:
https://www.ebay.com/itm/Dynatron-R...et-Narrow-2011-Intel-Xeon-Series/321375083251

Lastly since I am ordering used parts for the first time. Do i need to order any screws etc for putting these things together or they come with the screws.

That depends a lot on the case because most of the miscellaneous parts that you need in a system build come with the case, unless it is something specific to the installation of a particular component. The CPU cooler I pointed you at screws directly to a bracket that is integrated into the CPU socket on the socket 2011 boards. Nothing extra needed. System board standoffs usually come with the case. You might need some cable-management gear for keeping your wires tidy, but don't make anything too tight because it makes pulling it apart later difficult and there is always going to be a time when you need to take it apart for some reason. It is kind of like Murphy's Law of building a computer. Once you get it perfectly cable managed, you will need to take it apart for something.

You might’ve been joking, but I’ve successfully used 3M command strips in similar applications a number of times.

Same here. I didn't have good luck with them staying attached though. I ended up with things dangling by the wire. I have had poor results with adhesives over the years.

 

[LimeCrusher]

Hi Chris,
Thanks for the list up there. I have been reading the forums lately and going to be building my first ever server. I am going with the stuff you have suggested above. I have ordered the board and the processor above.

Please, let us know how it goes. I am curious about the feasibility of such a build for a newbie like myself.

 

[pashairfan]

Yes, the chipset in the HP H220 is newer and it uses PCIe 3.0 where the Dell H310 is older and uses PCIe 2.0. If I recall correctly, there are some minor differences beyond that but I don't recall what all of them are. I have observed both in operation, in my systems, and the newer cards appear to run a few degrees cooler. I have moved from the older card to the newer card in both of my personal FreeNAS systems, but either card is a perfectly valid option.

If you got the system board I am thinking you got, it needs a 'narrow ILM' cooler. This is the model I bought to use on both of my systems that take the narrow ILM cooler and I am very happy with them:
https://www.ebay.com/itm/Dynatron-R...et-Narrow-2011-Intel-Xeon-Series/321375083251

That depends a lot on the case because most of the miscellaneous parts that you need in a system build come with the case, unless it is something specific to the installation of a particular component. The CPU cooler I pointed you at screws directly to a bracket that is integrated into the CPU socket on the socket 2011 boards. Nothing extra needed. System board standoffs usually come with the case. You might need some cable-management gear for keeping your wires tidy, but don't make anything too tight because it makes pulling it apart later difficult and there is always going to be a time when you need to take it apart for some reason. It is kind of like Murphy's Law of building a computer. Once you get it perfectly cable managed, you will need to take it apart for something.

Same here. I didn't have good luck with them staying attached though. I ended up with things dangling by the wire. I have had poor results with adhesives over the years.

Thanks Chris, I went ahead and ordered everything. I went with the HP220 controller and the cooler you recommended. Estimated shipping date is between 6-10th Dec since I am in Canada it just takes longer. I will be using a Node 804 Chassis for now. Will upgrade to a bigger chassis sometime in the next year when I have a dedicated media room in the new house.

I am still looking for a power supply would you recommend anything specific?
I am thinking Seasonic PSU below
https://www.newegg.ca/Product/Product.aspx?Item=N82E16817151192&ignorebbr=1

Would it be too much for the system I am building?