New X9SRL-F build - Fine tuning

[racielrod]

On my next motherboard I'll mirror my boot drives for several reasons:

1. The SuperMIcro motherboard has two built-in SATADOM connectors
2. Encrypted pool makes a functional boot drive (with config, keys and so on) even more important.
3. The motherboard has more SATA / SAS connectors than I can make use of
4. Provisioning/confectioning a SATADOM with a new copy of FreeNAS, transferring the config file, etc. is not as easy as it is with a SSD.

I put mirrored boot and SLOG drives in the same category - it's a good idea if motherboard infrastructure, power, budget, etc. allow it.

For example, while my next motherboard will feature two 64GB SATADOMs that fit into their designated orange connectors, the board only features one M.2 PCIe 3.0x4 connector. A fast SLOG will thus either require the use of a PCIe slot (for a AIC, U2, or carrier card approach) and/or the M2 slot. For now, I plan on using a single P4801x in the M2 slot.

My current motherboard only features a single boot drive on a 3GB/s SATA interface and mirrored SLOGs running on a 6GB/s SATA interface. My hope is that between the much faster interface (PCIe 3.0x4) and the actual SLOG (which claims 2GB/s for large, sustained writes) that SLOG-related performance will improve. I also wonder to what extent a dedicated HBA (LSI 2116) on the new motherboard will make a difference vs. the motley crew of Marvell and like controllers on the current C2750D4I.

The fact that almost every sentence in this post is triggering a google/forum search on my part is making me wonder how ready I really am for FreeNAS

 

[racielrod]

I'm ordering 2 SSDs for mirrored boot drives.
With my 5 hdds plus these 2 that will be 7 drives total.
The Seasonic Focus Plus Gold 550w has 6 SATA power ports on it, while the same PSU but 650w has 8 ports available and it is only $10 more. (Extra cables are ~$30 in CableMod).
If my goal is never to exceed 6 HDDs (if I need more space I would probably upgrade to bigger HDDs) should I consider the 650W PSU or would this be oversized and therefore not as efficient? (Maybe my assumptions are wrong here).

Thoughts?

 

[racielrod]

I'm between these 2 ssds:

https://www.ebay.com/itm/Intel-320-...m34031585d6:g:rtwAAOSwToZcZL1n&frcectupt=true

and

https://www.amazon.com/Kingston-120...gston+120+ssd&qid=1552331232&s=gateway&sr=8-3

I'm leaning towards the Kingstons because they are new and have 40GB more for an extra $10. But I'm not sure if people have had bad experiences with these in the past or not.
I know the Intel ones are recommended very often here in the forums.

Should I be worrying about the Kingstons?
Should I stick to the 550W PSU or is the 650W a better bet given the extra cables? Or would I wasting the savings in cables on added power consumption over the years?

 

[PhiloEpisteme]

Should I stick to the 550W PSU or is the 650W a better bet given the extra cables? Or would I wasting the savings in cables on added power consumption over the years?

I personally opted for the larger PSU for the cables and reliability. I wanted to be able to easily add additional drives, cards, etc. Though, I was not thinking about cable/psu cost vs long-term power costs. If you've done power consumption calculation estimates how much headroom does the 550W give you?

 

[racielrod]

These are the numbers based on @jgreco 's "Proper Power Supply Sizing Guidance":

Total Potential Watts
X9SRL-F: 25W
E5-2650 v2: 95W (TDP)
Memory: 2 x 16GB x 6w = 12W
HDD: 6 Max x 35W = 210W
Fans: 4 x 15W = 60W
Total: 402W
Target: 402W * 1.25 = 502.5 W

I didn't include the 2 SSDs I'm now considering, but I don't think they will move the needle that much.
I believe I have read somewhere else the guidance I'm using is VERY conservative. So a 550W should be more than plenty for my setup and might have some room to scale? I'm not planning to push it though.
As a reference I checked Seasonics' Wattage Calculator and it is giving me these numbers:

Load Wattage:289 W
Recommended PSU Wattage:339 W

I hope this helps.
Thanks,
R. Rod

 

[jgreco]

The main problem with most wattage calculators is that they do not provide elbow room for multiple HDD's simultaneously spinning up, so their numbers are usually trash for more than maybe three or four drives.

 

[racielrod]

The main problem with most wattage calculators is that they do not provide elbow room for multiple HDD's simultaneously spinning up, so their numbers are usually trash for more than maybe three or four drives.

Thanks for the info. And thank you for putting the guidelines together.

Do you think going from 550W to 650W makes sense in my case?

- If I use the "Lazy Geek Guess" it points me to a 650W.
- If I calculate my target size based on potential watts, that gives me ~500W.
-- 1/5 of that is 100W and this is more than my AVG idle number 88W (I guessed fans will be 1/2, not sure if this ok).
-- So the "sweet spot" if I start at my AVG Idle Watts is more like 450W.

Does this make sense? If so, is going to 650W just a plain waste of money and power?

Thanks!
R. Rod

 

[jgreco]

Thanks for the info. And thank you for putting the guidelines together.

Do you think going from 550W to 650W makes sense in my case?

- If I use the "Lazy Geek Guess" it points me to a 650W.
- If I calculate my target size based on potential watts, that gives me ~500W.
-- 1/5 of that is 100W and this is more than my AVG idle number 88W (I guessed fans will be 1/2, not sure if this ok).
-- So the "sweet spot" if I start at my AVG Idle Watts is more like 450W.

Does this make sense? If so, is going to 650W just a plain waste of money and power?

Thanks!
R. Rod

For where you're just planning on 6 drives, or where you might someday do more?

So the big idea I had was to give people numbers where they were pretty sure they wouldn't kill their hardware and drives. This was precipitated when a particular user here decided to go around spouting "expertise" and quoting people PSU wattage numbers I deemed to be irresponsible and dangerous. His basic strategy was to size to almost the smallest PSU possible, and when I told him to stop, some bad things happened, and then I decided to go with the informed-decision model.

I did not recalculate your numbers. If you followed my methodology and the resulting numbers gave you 500-550, I am *confident* in that.

This seems to fit well with the Lazy Geek Guess being sized a bit bigger, where there are more potential "unknowns," so usually those numbers are a bit bigger. But very safe.

On the cheap end of supplies, the PSU efficiency curve starts to get crappy once you get down past maybe 20% idle load. So especially with the cheap supplies, that's a bit of a factor.

So the thing that you have to grapple with is that PSU's aren't free, and usually bigger PSU's are more costly, and this is even more true for reputable pro-sumer PSU's like the Seasonics. But the PSU efficiency curve is better, and you can get down to ~10-15% without as much loss of efficiency.

You can read this particular post:

https://www.ixsystems.com/community/threads/i-need-some-data-points.32829/post-204099

So that's output power: power measured as a sum of the amps times volts of all the DC supplies. A 60 watt system is operating at 16% of rated load on a 360W supply looks to be around 86% efficient, while that same system is only around 11% rated load on a 550W supply, which is 83% efficient. I've picked the 550 because I was able to find numbers for it - not because it's a good idea. I have to assume the G-450 would perform "better" but I am giving your position a head start.

The 360 watt supply supplying 60W at 86% efficiency is therefore consuming 70W at the wall. The 550 watt supply at 83% efficiency is consuming 72W at the wall.

That's TWO WATTS MORE due to the inefficiency of the larger supply.

At a cost of 14c/kWh, it would cost you about $12.26 over the next five years for that larger supply. Probably less if you went for the G-450. The "green" savings is exceedingly minimal.

So here's the thing. You can definitely take cost into account. If you found a 650W PSU and you really only needed a 500W, you might end up consuming a watt or two more due to inefficiency. That does have a cost. But if the 650W PSU is on sale for $20 off, it's probably a wash.

I'll also tell you I'm paranoid about this sort of thing. But I did the homework and I showed all my work, explained my reasoning. There is a safety margin in there.

 

[racielrod]

For where you're just planning on 6 drives, or where you might someday do more?

I don't see myself adding more than 6 HDDs -and maybe a lot of people has started here with the same idea and ended up with massive systems :) - this would be already a bigger system that I originally intended (got here from considering a QNAP appliance... go figure).

On the cheap end of supplies, the PSU efficiency curve starts to get crappy once you get down past maybe 20% idle load. So especially with the cheap supplies, that's a bit of a factor.

So the thing that you have to grapple with is that PSU's aren't free, and usually bigger PSU's are more costly, and this is even more true for reputable pro-sumer PSU's like the Seasonics. But the PSU efficiency curve is better, and you can get down to ~10-15% without as much loss of efficiency.

I'm aiming for the Seasonic Focus Plus 550W Gold and its 650W alternative. I checked the efficiency numbers and they seem to be very close.
My main concern was not about saving money upfront, but about buying an oversized PSU I don't need and wasting money up-front and on a monthly basis.
I don't think I can go wrong with either one... it is probably about getting as close to the sweet spot as possible.

I'll also tell you I'm paranoid about this sort of thing. But I did the homework and I showed all my work, explained my reasoning. There is a safety margin in there.

Appreciated! I'm the same when it comes to safety... Thanks again for your work!

 

[jgreco]

this would be already a bigger system that I originally intended (got here from considering a QNAP appliance... go figure).

Isn't that how it always is, though?

Appreciated! I'm the same when it comes to safety... Thanks again for your work!

I enjoy talking tech with anyone willing to listen. ;-) It's my pleasure.

 

[racielrod]

Build update:

I just finished installing the drives.
I ran 5 passes of Memtest86 and memory seems to be OK.

It is taking shape (images attached).
I'm downloading FreeNAS and installing tonight.

 

[ali_v001]

What cooler did you go for in the end? Any issues I've gone for the same board and also a fractal case. Any issues mounting the motherboard etc? Very similar setup

 

[racielrod]

I'm using this one:

https://www.amazon.com/gp/product/B00E1JGFA0/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

No major issues so far, very quiet and keeps everything cool. I installed it more than a year ago, so I might be forgetting details, but the installation was straightforward.

The only nuisance is that the cooler covers the inner ram slot on each side. I'm not using these slots, so not a big deal to me. And even if I were, I don't see myself swapping memory often enough that this would be an issue.

This setup is rock solid, no issues after a 1+ year of running.