New X9SRL-F build - Fine tuning

[racielrod]

After auto-hijacking my post here https://www.ixsystems.com/community/threads/memory-at-hand-will-it-freenas.74488/ I decided to move this to a new thread - apologies.

I finally pulled the trigger and bought an X9SRL-F from here:
https://www.ebay.com/itm/Supermicro...974587&hash=item5d85e7b4b9:g:Y3wAAOSwDmxcLiog

This is what I'm planning to buy for the rest of the build. Please let me know what you think

Memory
32GB
2 x HYNIX HMT42GR7MFR4A-H9 16GB (1X16GB) 1333MHZ PC3-10600 ECC
https://www.serversupply.com/MEMORY/PC3-10600/16GB/HYNIX/HMT42GR7MFR4A-H9.htm
I'm not buying this memory, I have it at hand.

Processor
Intel-Xeon-E5-2650V2
https://www.ebay.com/itm/Intel-Xeon...m4433fd7f5a:g:VEkAAOSw-sVcR45O&frcectupt=true
Question: I'm not expecting this NAS to do a lot in the short term. If keeping this as power efficient as possible is one of my goals, is there another compatible processor (with the board and memory) that I can pick?

Processor Cooler
Noctua NH-L9i

https://www.newegg.com/Product/Product.aspx?Item=9SIAADY4B02495
I want this build to be as silent as possible, hence my noctua pick. I'm not sure if there is anything better out there that is as quiet.

Boot Drives
I'm planning to use any low capacity SSD I can find online. Are there any specific requirements/specs I need to observe here?

Drive Controller
If I'm planning to use 4-6 HDD (still undecided) do I need this? Would you include it on your initial build if you were on my position?

PSU
Still undecided, but I think I'm going with a Seasonic G-550, which seems to be the go to option here in the forum.
Should I consider anything else?
Update:
This PSU looks like a strong candidate
Seasonic FOCUS Plus 550 Gold SSR-550FX 550W 80+ Gold ATX12V & EPS12V Full Modular
https://www.amazon.com/dp/B073GY89G5/ref=cm_sw_r_sms_c_api_i_a6.GCbHVXSYCB

Case
Undecided. I want a Fractal R6, but I might consider an R5 to save a few bucks if possible.
Thoughts?

The board must be arriving next week, so I would like to order the rest of the components ASAP to be able to test everything - just in case I need to return any defective part, considering most of the parts are used.
Is there any guidance here in the forums on how to approach this hardware test? I have put many PCs together in the past, but this is my first "server" grade built. I'm not sure if there are things I should be considering... (definitely a lot to learn).

Thanks for any feedback
R. Rod

 

[PhiloEpisteme]

If I'm planning to use 4-6 HDD (still undecided) do I need this? Would you include it on your initial build if you were on my position?

I wouldn't include it if I didn't have to. If you don't plan on using more drives than your board will support no need to grab an HBA now. Though, if you plan to grow your number of disks you might want to keep an eye out for exceptional deals.

Are there any specific requirements/specs I need to observe here?

I think you've got the right idea; low capacity drives that aren't too expensive. 64GB SSDs shouldn't break the bank.

Processor
Intel-Xeon-E5-2650V2
https://www.ebay.com/itm/Intel-Xeon...m4433fd7f5a:g:VEkAAOSw-sVcR45O&frcectupt=true
Question: I'm not expecting this NAS to do a lot in the short term. If keeping this as power efficient as possible is one of my goals, is there another compatible processor (with the board and memory) that I can pick?

For that I'd just check out ark.intel.com and pick the lowest power consumption CPU at a clock speed I can tolerate. The E5-2630 V2 may be a good option balance, it is 80W 6 cores at 2.60 GHz vs 95W 8 cores at 2.60 GHz of E5-2650 V2 if you wanna go even lo

Undecided. I want a Fractal R6, but I might consider an R5 to save a few bucks if possible.
Thoughts?

That is the case I use. If you want more than 6 HDD you have to buy extra trays. It'll go up to 12 HDD without much difficulty and is quiet and does a good job of keeping the drives cool.

2 x HYNIX HMT42GR7MFR4A-H9 16GB (1X16GB) 1333MHZ PC3-10600 ECC
https://www.serversupply.com/MEMORY/PC3-10600/16GB/HYNIX/HMT42GR7MFR4A-H9.htm
I'm not buying this memory, I have it at hand.

If you've got the memory on hand and it is compatible go for it. You'll still have plenty of room to grow.


Hope this helps. I'm sure some other folks might have other useful advice as well.

 

[Turgin]

You should double check your heat sink compatibility. That board requires a narrow ilm socket 2011 cooler.

You may want to look at the newer focus plus series seasonic power supplies.

 

[racielrod]

For that I'd just check out ark.intel.com and pick the lowest power consumption CPU at a clock speed I can tolerate. The E5-2630 V2 may be a good option balance, it is 80W 6 cores at 2.60 GHz vs 95W 8 cores at 2.60 GHz of E5-2650 V2 if you wanna go even lo

So, every E5-26xx v2 and E5-16xx v2 seem to be compatible here and I only need to understand the memory restrictions for the E5-16xx v2 - as posted on the board specs?
It looks like there are a lot of processor options on that family. I will try to find the right balance performance/power consumption and report back.

Thanks!

 

[racielrod]

You should double check your heat sink compatibility. That board requires a narrow ilm socket 2011 cooler.

Something like this maybe?
https://noctua.at/en/nh-u12dx-i4.html

Is it a bad idea to consider passive cooling?

You may want to look at the newer focus plus series seasonic power supplies.

Thanks for the recommendation. I'm going to read the PSU sizing that is being referred to here frequently and try to find a suitable Seasonic PSU for my build.

 

[anmnz]

Something like this maybe?
https://noctua.at/en/nh-u12dx-i4.html

I use that on an X9SRL-F with E5-2680v2. It's great. The only slight quirk, if I can even call it that, is that you have a choice of orientations for the cooler, and if you mount it so it blows air towards the back of the motherboard where the I/O ports are (which was my preference), then it blocks the inner two DIMM slots. This is only a problem if you have more than 4 DIMMs, and if you do you can just rotate the cooler by 90 degrees.

Is it a bad idea to consider passive cooling?

It's important to cool the disks properly, not just the CPU. You need to get air flowing through the case and over the disks, so you need to have fans in there. So no benefit in passive cooling.

 

[anmnz]

If keeping this as power efficient as possible is one of my goals, is there another compatible processor (with the board and memory) that I can pick?

For that I'd just check out ark.intel.com and pick the lowest power consumption CPU at a clock speed I can tolerate. The E5-2630 V2 may be a good option balance, it is 80W 6 cores at 2.60 GHz vs 95W 8 cores at 2.60 GHz of E5-2650 V2 if you wanna go even lo

I wouldn't do that. Those power consumption figures are TDP, which relates to maximum heat dissipation, and your NAS will almost never be running like that.

What you should care about is idle power consumption, where your NAS will be almost all the time. At least since Sandy Bridge, all Xeon CPUs of the same generation will idle at pretty much the same power consumption. So there is not going to be a significant benefit in terms of power consumption in switching from one "E2-26xx v2" to another.

 

[PhiloEpisteme]

What you should care about is idle power consumption, where your NAS will be almost all the time. At least since Sandy Bridge, all Xeon CPUs of the same generation will idle at pretty much the same power consumption. So there is not going to be a significant benefit in terms of power consumption in switching from one "E2-26xx v2" to another.

Ah, thanks for clarifying. :)

 

[racielrod]

It's important to cool the disks properly, not just the CPU. You need to get air flowing through the case and over the disks, so you need to have fans in there. So no benefit in passive cooling.

Yes, I'm planning to (learn to) guarantee good airflow for the disks, I was just wondering if passive cooling was an option for the processor assuming there will be a good airflow through the case.
BTW - Were the OEM fans -and placement- of the R5 good enough to keep the disks cool or did you have to add or modify anything?

Thanks a ton for the feedback

 

[racielrod]

So there is not going to be a significant benefit in terms of power consumption in switching from one "E2-26xx v2" to another.

Thanks for the info, I thought there was going to be significant long term power savings if going with a less powerful processor.
Is the E5-2650v2 the best option in the E5-26xxv2 family or would you consider any other processor if you were on my situation? I'm pretty much sold on it, but wanted to double check.

 

[Constantin]

I'm a big believer in keeping hardrives and CPUs cool. Whether you use a shroud with a fan or some similar mechanism, many motherboards seem to be designed with high air flows across them in mind - see the chassis made by Supermicro as an example: They offer clear shrouds that fit over the motherboard to minimize the 'open' headroom and hence direct as much air flow across the board and its components as possible. Naturally, OEM shrouds will be specific to motherboards, i.e. choose wisely.

Similarly, I like space between hard drives so that I can use low static pressure fans to keep them cool. That approach grows the case size (more space needed between drives = more volume for a given # of hard drives) but the fans can run at much quieter settings than if they have to deal with a lot of static pressure drop (like tightly-packed drives). Additionally, I spend some time sealing the case on the inside to ensure that the intended air flow actually goes where it ought to. Black duct tape or even Al tape work great for this application.

Lastly, I'm a big fan of the Noctua line - they offer a great variety of high-quality fans at relatively-competitive prices. I use them for everything from my computer cases to upgrading the cooling fan on the DIY humidifier's external power supply to make it run quieter and cooler. A external temperature-based fan controller may or may not help - I suppose it depends in part on how good you are implementing the PID control systems that others here have developed.

 

[anmnz]

Were the OEM fans -and placement- of the R5 good enough to keep the disks cool or did you have to add or modify anything?

They seemed good enough, and not too loud. But I'm a bit of a Noctua junkie and was always going to replace them.

 

[anmnz]

Is the E5-2650v2 the best option in the E5-26xxv2 family or would you consider any other processor if you were on my situation? I'm pretty much sold on it, but wanted to double check.

Seems to me like a fine choice, and popular around here. Although I bought an E5-2680v2 instead, I don't feel like I have any compelling reason to recommend it ahead of the '50.

 

[racielrod]

Similarly, I like space between hard drives so that I can use low static pressure fans to keep them cool. That approach grows the case size (more space needed between drives = more volume for a given # of hard drives) but the fans can run at much quieter settings than if they have to deal with a lot of static pressure drop (like tightly-packed drives). Additionally, I spend some time sealing the case on the inside to ensure that the intended air flow actually goes where it ought to. Black duct tape or even Al tape work great for this application.

With the Fractal Design R5 or R6 I'm expecting to have plenty of room in between the 5-6 disks I'm expecting to use.
My plan is to keep this build as quiet as possible by using bigger/low-static pressure fans to move air around.
Thanks for the tip about sealing the case as much as possible.

Lastly, I'm a big fan of the Noctua line - they offer a great variety of high-quality fans at relatively-competitive prices. I use them for everything from my computer cases to upgrading the cooling fan on the DIY humidifier's external power supply to make it run quieter and cooler. A external temperature-based fan controller may or may not help - I suppose it depends in part on how good you are implementing the PID control systems that others here have developed.

I'm planning to buy a noctua processor cooler and I'll eventually replace every fan in the case with noctua fans.
Thanks for pointer to the PID control system, I have a lot to learn....

Is there anything else I should be considering after I put this build together?
Again, I have build many pcs, but never used server grade hardware before.
Other than updating the BIOS, and running the manufacturer tests on the board, is there anything else?

 

[Constantin]

Starting this business week, I will be right behind you on this learning curve when my next motherboard comes in. I’ve never configured a board from scratch, so it should be an interesting experience.

I’m really looking forward to having an extended life supermicro board that will hopefully serve me well for the next 10 years or more.

 

[racielrod]

Thank you everyone who offered some feedback.
It looks like I have a good learning curve ahead of me, but I'm excited.

So far I have decided to buy these components:
PSU
Seasonic FOCUS Plus 550 Gold SSR-550FX 550W 80+ Gold ATX12V & EPS12V Full Modular
https://www.amazon.com/dp/B073GY89G5/ref=cm_sw_r_sms_c_api_i_a6.GCbHVXSYCB

Processor
E5-25650 V2
https://www.ebay.com/itm/SR1A8-INTEL-XEON-E5-2650V2-2-6GHZ-20MB-CACHE-8-CORE-LGA-2011-PROCESSOR/183022002957?epid=1823003776&hash=item2a9cf6230d:g:MhYAAOSwUPlcZbr9:sc:UPSGround!30024!US!-1&LH_FR=1&LH_ItemCondition=1000|1500|2000|2500|3000

Processor Cooler
https://www.amazon.com/Noctua-Cooler-LGA2011-Platforms-NH-U12DXi4/dp/B00DWFQ42I/ref=sr_1_1?keywords=NH-U12DX+i4&qid=1552176087&refinements=p_85:2470955011&rnid=2470954011&rps=1&s=gateway&sr=8-1
OR
https://www.amazon.com/Noctua-NH-D9DX-LGA2011-0-LGA2011-3-Backplate/dp/B00PIPCGWC/ref=sr_1_3?keywords=NH-U12DX+i4&qid=1552176087&refinements=p_85:2470955011&rnid=2470954011&rps=1&s=gateway&sr=8-3

 

[racielrod]

Something I forgot to ask:
Do I need 1 or 2 SSD to boot FreeNAS from? I'm asking because I think I remember reading about the possibility of having these drives mirrored?
I think it was here:
https://www.ixsystems.com/community/resources/specific-build-components-list-up-to-32gb-ram.109/

Thoughts?

 

[chris crude]

Mirroring boot drives is definatly an option. It is reccomend for USBs but not SSDs because of the difference in failure rates. Honestly since this seems like a home build and not an enterprise machine where uptime has to be 100% I would just save the config file often. If your boot does die just install FreeNAS to a new drive, upload saved config and everything will be back to normal.

 

[racielrod]

Mirroring boot drives is definatly an option. It is reccomend for USBs but not SSDs because of the difference in failure rates. Honestly since this seems like a home build and not an enterprise machine where uptime has to be 100% I would just save the config file often. If your boot does die just install FreeNAS to a new drive, upload saved config and everything will be back to normal.

Thanks. It makes sense.

 

[Constantin]

On my next motherboard I'll mirror my boot drives for several reasons:

1. The SuperMIcro motherboard has two built-in SATADOM connectors
2. Encrypted pool makes a functional boot drive (with config, keys and so on) even more important.
3. The motherboard has more SATA / SAS connectors than I can make use of
4. Provisioning/confectioning a SATADOM with a new copy of FreeNAS, transferring the config file, etc. is not as easy as it is with a SSD.

I put mirrored boot and SLOG drives in the same category - it's a good idea if motherboard infrastructure, power, budget, etc. allow it.

For example, while my next motherboard will feature two 64GB SATADOMs that fit into their designated orange connectors, the board only features one M.2 PCIe 3.0x4 connector. A fast SLOG will thus either require the use of a PCIe slot (for a AIC, U2, or carrier card approach) and/or the M2 slot. For now, I plan on using a single P4801x in the M2 slot.

My current motherboard only features a single boot drive on a 3GB/s SATA interface and mirrored SLOGs running on a 6GB/s SATA interface. My hope is that between the much faster interface (PCIe 3.0x4) and the actual SLOG (which claims 2GB/s for large, sustained writes) that SLOG-related performance will improve. I also wonder to what extent a dedicated HBA (LSI 2116) on the new motherboard will make a difference vs. the motley crew of Marvell and like controllers on the current C2750D4I.