Hardware Recommendations Guide Discussion Thread Rev 2a) 2021-01-24

[Ericloewe]

Could someone check the links here:

Hardware Recommendations Guide

This is the latest edition of the FreeNAS Community hardware recommendations guide. The current major version is R2, dated January 2021, with the last minor update on 2021-01-24. The format has moved away from the forum post form factor, to...

www.truenas.com

some of them appear to be broken for some reason. I

So, I went through all the links in the latest version and they all worked for me. I replaced them all anyway to save some redirects.

 

[Yorick]

Oh super! Feedback on CL and CL Refresh Xeons:

"
Intel Xeon E-2126G Similar to the Xeon E3-1220 line, but with six cores instead of four.
Intel Xeon E-2176G As Coffee Lake i7 Xeon equivalent, has six cores with hyperthreading.

Xeon E3 / Xeon E models with iGPUs

Equivalent models that do have iGPUs exist for many of the above models, denoted by a model ending in 5. The GPU is currently of limited usefulness for TrueNAS
"

For the CL/CL Refresh, the "G" models are actually the ones with iGPU. The equivalent models without iGPU might be quad core E-2124 or E-2224 for CL Refresh, and hexa core with HT E-2136 or E-2236 for CL Refresh.

 

[mvcad]

When using SSD as boot device you should mention that SSD with Marvell controller do not work. i.e i tried to use as WD green for booting resulting in immediate degraded volume. I replaced it with a Kingston SSD now and all good.
This seems to be a known issue that has not been resolved. But If that were mentioned on the hardware recomendation guide it would have saved me some money.

 

[Yorick]

When using SSD as boot device you should mention that SSD with Maxwell controller do not work. i.e i tried to use as WD green for booting r esulting in immediate degraded volume.

That sounds like hardware failure, not incompatibility. That said, QLC and DRAMless SSDs are more prone to failure. It might me a good idea to point that out in the recommendations.

 

[artlessknave]

page 10, SAS2
is this line intended to be duplicated?
"All of these, except for the LSI SAS 9211 and 9207, must be cross-flashed. All of these, except for the LSI SAS 9211 and 9207, must be cross-flashed."
page 15, Not-quite-ancient Hardware
shouldn't there be something following "in and out of the CPU must ....?", like "must communicate through"?
"Performance will be seriously impacted by the Front-Side Bus, which every single operation in and out of the CPU must – this includes operations between the two halves of a quad-core model."

 

[Ericloewe]

When using SSD as boot device you should mention that SSD with Marvell controller do not work

A specific controller, anyway.

That sounds like hardware failure, not incompatibility. That said, QLC and DRAMless SSDs are more prone to failure. It might me a good idea to point that out in the recommendations.

WD Greens and the SanDisk equivalent do have a controller bug that affects FreeBSD.

 

[cobrakiller58]

When using SSD as boot device you should mention that SSD with Marvell controller do not work. i.e i tried to use as WD green for booting resulting in immediate degraded volume.

I do not remember the Marvell controllers having a problem, it was the SM2246, SM2256 and SM2258 if i recall correctly

A specific controller, anyway.

WD Greens and the SanDisk equivalent do have a controller bug that affects FreeBSD.

Should a note be adding about this since they are still marketed and the bug https://jira.ixsystems.com/browse/NAS-100276 has sadly been marked as "not to be fixed"

 

[Ericloewe]

Yeah, I'm planning an update for next weekend.

 

[rmccullough]

I don't know if this is an appropriate place to ask, but I noticed in the recent hardware guide it spoke about Supermicro PSUs. In particular that older models are very noisy. I have dual 920w PSUs and my server is very noisy. How can I check if I have the older "noisy" PSUs? And what do I want to look for to identify a newer PSU that may not be as noisy? Are these generally available on eBay?

 

[Ericloewe]

My understanding is that any Platinum models are new enough to not be a disaster. Keep in mind, however, that "quiet" is a relative term when it comes to servers.

 

[jgreco]

I don't know if this is an appropriate place to ask, but I noticed in the recent hardware guide it spoke about Supermicro PSUs. In particular that older models are very noisy. I have dual 920w PSUs and my server is very noisy. How can I check if I have the older "noisy" PSUs? And what do I want to look for to identify a newer PSU that may not be as noisy? Are these generally available on eBay?

Pull one of the PSU's and see if the part number is PWS-920P-SQ ("Super Quiet"). eBay sells lots of them, usually at a bit of a premium price.

And they're not "very noisy", but they aren't as quiet as the SQ's.

 

[rmccullough]

Ok, thanks I’ll pull one and take a look.

My server is in my crawl space, but I can hear it beneath the stairs and behind the closet in the guest room. The fans are pretty noisy. Not sure if it’s just the PSU fans or the others in the chassis.

Am I chasing a dream with this chassis as far as getting it quieter?

 

[jgreco]

Well, the thing NOT to do is to just blindly pull all the fans and replace them with "Noctua" fans. Lots of gamers come in here with the idea that servers are just funny looking PC's.

From https://www.truenas.com/community/t...reenas-sitting-in-an-office.89074/post-616547

The fundamental problem with 4-drive-wide rackmount gear is that there is a need to cool the drives. This happens by sucking, hard, to force air through the tiny gaps that exist around the drives in the drive bays. Causing this static pressure differential is a process that consumes a lot of energy and that translates to noise. The fans that are used are beefy, industrial grade fans that are tolerant of such abuse and will still last a decade or more given such abuse.

Your problem is tougher because of 1U; the little 40MM fans are incredibly noisy due to the physics of it all.

A better solution for you might be to go for a Supermicro SC826 style chassis, 2U x 12 bays, specifically with the PWS-920P-SQ ("super quiet") power supplies.

A lot of people buy these and then try to toss Noctua fans in with 12 drives to keep the noise down and maximize the storage. Don't do this, it's dangerous and stupid. You will end up with a nice little oven up front, especially as the fans fail, because the Noctuas are not designed to generate
a high pressure differential. Stacking two drives on top of each other creates a hot spot between them, and if you do not have the "oomph" to forcibly move some air through the gap, it gets hot. Twelve of them is a recipe for disaster. The two drives in the middle especially have nowhere for their heat to go, despite the irony of having some fans directly behind them. If the fans cannot move significant air, the drives just warm up and bake at far too high a temperature.

But what you *can* do is to remove every other drive in a checkerboard pattern. This gives you capacity for six drives, no two drives adjacent, and six slots for air to easily flow through, so then you can use silent "gamer" fans in place of the industrial fans on the bulkhead. The drives will run a slight bit warmer, but because there is a way for heat to bleed off, it's generally possible to get this working fairly quietly.

It is also possible to do this with eight drives by leaving the middle row empty.

It's usually a better thing to see if you can figure out why the fans are noisy. Is the BMC revving them due to heat? Is the fan setting set incorrectly in the IPMI? Has one of the fans stalled, causing the others to rev up to full speed? Is it a really old board that just doesn't have the sensors to do a better job? Fixing any one of these problems is a better fix than trying to replace the fans. If you end up deciding to replace the fans, you need to make sure your fans are monitored carefully, because none of the common alternative fans are anywhere near as robust as the ones that come from Supermicro.

 

[silmor_senedlen]

Xeon E5-16xx CPUs support single-socket systems only and do not support LRDIMMs.

Xeon E5-16xx and Xeon-D additionally support RDIMMs; and Xeon E5-2xxx ,Xeon E5-4xxx, Xeon Scalable and AMD Epyc support all three.

Apparently, these statements are incorrect.
Please see my comment in this topic about LRDIMM support.

 

[jgreco]

Apparently, these statements are incorrect.
Please see my comment in this topic about LRDIMM support.

And you were corrected in that thread.

 

[Ericloewe]

Since the discussion is mostly academic (who would want to throw 1 TB of DRAM onto a board alongside a Xeon E5-16xx?) and the support is not official, I do not think it is the right move for me to tell people that LR-DIMMs will work.
A more interesting question would be Xeon Scalable... I wouldn't be surprised if not all of those supported LR-DIMMs, but Intel's segmentation has broken my will to understand it.

 

[silmor_senedlen]

the support is not official, I do not think it is the right move for me to tell people that LR-DIMMs will work.

Maybe just take a note, that some people report that it work on <some_configurations>?

 

[Ericloewe]

What would be gained from that? That's like saying "you can buy a fuel pump from Lotus for twice the price and use it on your Toyota", it is not very useful advice and promotes odd choices that are likely to not work for some people.
You can get 64 GB RDIMMs, why would you go out and buy LR-DIMMs?

 

[Rand]

Actually you can buy 128G RDimms nowadays :)

 

[Ericloewe]

Do those work with Xeon E5? The memory limits in that generation were somewhat low.