Hardware Recommendations Guide

Hardware Recommendations Guide Discussion Thread Rev 2a) 2021-01-24

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
This is the discussion thread for the new Hardware Recommendations guide, meant to replace cyberjock's old version. It can be found here - alternatively, jsut use the tabs at the top of the thread.

Please post your suggestions here, as I'll be editing it in the near future according to feedback.

Please note that this thread is not the place to discuss specific builds. Please start a new thread if you want advice for your build - The correct place to post builds is the Will it FreeNAS? section.
 
Last edited:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175

scwst

Explorer
Joined
Sep 23, 2016
Messages
59
First, thanks for the work!

The sentence on page 9 Basically, the community’s approach is “If you’re going to do it, do it right” got me thinking - I wondering if some of the discussions that keep coming up with new people in the forums are based on different standards of keeping data safe - the ZFS/FreeNAS people ("us") really, really want to secure their data, whereas there are a lot of people who just, you know, like, want a place to sorta-kinda put their files. Maybe we could add something to that effect in the intro, so the people know what they are getting into, and those who are just looking for something more casual realize early on that they might be in the wrong place?

Since a big red label "Warning: Paranoid Data Security Freaks" is probably out of the question, maybe we put something like that in the second paragraph:

This document assumes that the user wants a very, very reliable server to keep data as safe as humanly possible down to the very last bit, not just some place to dump a bunch of files. Such a server will be built to take maximum advantage of FreeNAS' -- and thus ZFS' --strengths. Accordingly, these recommendations are very different from typical gaming or general work builds.​

Then, we could split the third paragraph and add the "big red label" part to the new paragraph:

The community is very welcoming of users asking for help with their builds, as long as the suggested build has been researched – it’s normal that questions will arise or that details will be overlooked and the community will gladly help. Having said that, it should be noted that quite a number of people in the community are proudly paranoid with respect to data safety, and tend to expect this mindset in others as well. Basically, the approach here is “If you’re going to do it, do it right”.​

Then we can delete the last line from the EEC entry - it's a really good one, I feel, and should come a lot earlier.

Again, thanks for all the work! I hope to read the whole thing this evening.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
First, thanks for the work!

The sentence on page 9 Basically, the community’s approach is “If you’re going to do it, do it right” got me thinking - I wondering if some of the discussions that keep coming up with new people in the forums are based on different standards of keeping data safe - the ZFS/FreeNAS people ("us") really, really want to secure their data, whereas there are a lot of people who just, you know, like, want a place to sorta-kinda put their files. Maybe we could add something to that effect in the intro, so the people know what they are getting into, and those who are just looking for something more casual realize early on that they might be in the wrong place?

Since a big red label "Warning: Paranoid Data Security Freaks" is probably out of the question, maybe we put something like that in the second paragraph:

This document assumes that the user wants a very, very reliable server to keep data as safe as humanly possible down to the very last bit, not just some place to dump a bunch of files. Such a server will be built to take maximum advantage of FreeNAS' -- and thus ZFS' --strengths. Accordingly, these recommendations are very different from typical gaming or general work builds.​

Then, we could split the third paragraph and add the "big red label" part to the new paragraph:

The community is very welcoming of users asking for help with their builds, as long as the suggested build has been researched – it’s normal that questions will arise or that details will be overlooked and the community will gladly help. Having said that, it should be noted that quite a number of people in the community are proudly paranoid with respect to data safety, and tend to expect this mindset in others as well. Basically, the approach here is “If you’re going to do it, do it right”.​

Then we can delete the last line from the EEC entry - it's a really good one, I feel, and should come a lot earlier.

Again, thanks for all the work! I hope to read the whole thing this evening.
I'll take a look at your suggestions, thanks!
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,358
Great update. Clear, concise and as far as I can tell, correct :)

As a selector of a few of the mentioned X10SRi-F motherboard, I thought I'd contribute to this

Supermicro X10SRi-F

The X10SRi-F is essentially the same as the X10SRL-F, but with a single dual-port Intel i350 NIC, which supports higher end networking features, mostly related to virtualization.

The major reason I selected the SRi-F was the PCIe3 x16 slot (electrical) on the SRi-F. The i350 was an important bonus since I'm planning on making VMs a large part of my workload once FreeNAS 10 ships.

The SRL-F provides a great number of slots, but none faster than 8x.

Why would I want x16? I wanted to guarantee that as much as possible IO would not be a future bottleneck on this system.

x16 SSDs will be a thing, probably sooner than you think:
http://www.anandtech.com/show/10125/seagate-announces-pcie-x16-ssd-capable-of-10gbs

And since PCIe4 wasn't available ;)


And some other minor things/points/suggestions:

SAS2

By far the most popular option, SAS controllers based on the LSI SAS 2008 or SAS 2308 chips provide up to eight lanes of SAS connectivity and are available in a number of configurations, with external and/or internal ports.

Some cards are configured as RAID controllers and require a crossflash to standard HBA mode.

Popular models include the LSI SAS 9211, SAS 9240 and SAS 9207 series, the IBM/Lenovo M1015 and the Dell H200.

I think you should list the M1115 with the M1015. They're essentially the same, but the M1115 might be easier/cheaper to find since its been the replacement for the M1015 for a while now.

LSI SAS firmware and drivers

The firmware used with these controllers must be flashed to the appropriate version required by the drive version used by FreeNAS. Currently-supported versions of FreeNAS provide a warning that indicates the correct firmware to use.

Think "drive version" should perhaps be "driver version"

Thankyou for mentioning the die-off ;)

Even cheaper alternatives exist, such as the Norco RPC-4224. However, build quality is not on par with Supermicro, power supplies are generally not included and thermal design is mediocre at best. Overall, the cost reduction may not be great enough to justify these chassis over a used Supermicro. These are generally poor solutions, overall.

With proper care, a very quiet thermally correct build can be made with a Norco chassis which uses the 120mm fan wall. Not sure if the same is possible, or as easily possible, with a Supermicro or some other more data-centre grade chassis.
https://forums.freenas.org/index.ph...24-supermicro-x10-sri-f-xeon-e5-1650v4.46262/
 
Last edited:

scwst

Explorer
Joined
Sep 23, 2016
Messages
59
Possible additions for the section on hard drives: I'm not sure if everybody is aware that the are not supposed to get too cool (see the failure rate in the Google Study at http://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf) - I know I was surprised. Maybe include a link to the study, which still seems to be one of its kind?

Also, is the suggestion on buying slow-spinning (5k rpm) drives not basically saying "buy WD Red"? The HGTS seem to spin faster in that size, and Seagate doesn't seem to be too popular around here, so what else is left?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Possible additions for the section on hard drives: I'm not sure if everybody is aware that the are not supposed to get too cool (see the failure rate in the Google Study at http://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf) - I know I was surprised. Maybe include a link to the study, which still seems to be one of its kind?
The drive failure rate(temperature) curve is much more favorable at lower temperatures (20-30 degrees) than at higher temperatures (40+ degrees), so brute-force cooling is a reasonable approach.

Their summary of the data is also contradicted by the data itself, from what I recall, since they mention no correlation between temperature and failure rate despite the massive drop in reliability past 40 degrees.

Most well-cooled drives end up at high-20s or low-30s at idle, so I don't see a problem worth treating in detail here.

Also, is the suggestion on buying slow-spinning (5k rpm) drives not basically saying "buy WD Red"? The HGTS seem to spin faster in that size, and Seagate doesn't seem to be too popular around here, so what else is left?
Possibly, I haven't really looked into that. Higher angular velocity won't kill anyone, but it's generally just useless.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
I've integrated a few changes and I'll probably release R1b later today or tomorrow. I'll also integrate any other fixes that might pop up.
 

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
Some thoughts. Apologies if these were already mentioned and I missed it.

Chassis Section
  • Maybe toss in some warning about noise (and possibly power consumption) for those considering running the larger systems
    • There is a small note already in there that says "(which can also be loud at full throttle)"; but I tend to think that doesn't really prepare the individuals for the reality of the situation...
  • Perhaps a warning about larger chassis with older backplanes?
    • Ones that don't see over 2.2TB
    • Ones that will be a "rat's nest" of cabling
Motherboards
  • ASRock (not that I use them)? Maybe other ones are okay?
    • Of course a warning should be there for those overseas that notes support will be significantly slower than for those U.S. based
CPUs
  • Is the information about i3 CPUs: "The *only* i3s that support ECC and are reasonably priced are Haswell i3 CPUs." still true?
    • Not sure either way, just wanted to see if that was still the general understanding...
Additional SATA/SAS Connectivity
  • Comment "If the controller you want to use is an Adaptec, Dell Perc, Highpoint, or some no-name brand you shouldn't even try to use them." could use a little updating
    • Dell Perc H200, H300 and H310 all are perfectly fine; whereas the H700, H710, H800 and H810 are not.
    • Not 100% sure on the ones above H700 (too lazy to check)
  • While the IBM M1015 is mentioned, the others should be as well. LSI 9211-8i, Dell Perc H200, H300 and H310
  • Maybe update "If you are on v16 drivers then the firmware had better be v16 too, without exception. 9.3 will give you a warning if you break this cardinal rule." so that it pertains to v20 with 9.10.1?
    • Of course there is that false-positive v21 message...

Maybe toss in some examples of "Turn Key" systems that can be bought (for those not wanting to build their own)
  • Dell T20
  • Lenovo TS140
  • If you are feeling *nice* maybe even a Dell C2100/FS12-TY
    • No worries if you don't since that is pretty old and I consider it for us "special" users. ;)
 
Last edited by a moderator:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Some thoughts. Apologies if these were already mentioned and I missed it.

Chassis Section
  • Maybe toss in some warning about noise (and possibly power consumption) for those considering running the larger systems
    • There is a small note already in there that says "(which can also be loud at full throttle)"; but I tend to think that doesn't really prepare the individuals for the reality of the situation...
  • Perhaps a warning about larger chassis with older backplanes?
    • Ones that don't see over 2.2TB
    • Ones that will be a "rat's nest" of cabling
Motherboards
  • ASRock (not that I use them)? Maybe other ones are okay?
    • Of course a warning should be there for those overseas that notes support will be significantly slower than for those U.S. based
CPUs
  • Is the information about i3 CPUs: "The *only* i3s that support ECC and are reasonably priced are Haswell i3 CPUs." still true?
    • Not sure either way, just wanted to see if that was still the general understanding...
Additional SATA/SAS Connectivity
  • Comment "If the controller you want to use is an Adaptec, Dell Perc, Highpoint, or some no-name brand you shouldn't even try to use them." could use a little updating
    • Dell Perc H200, H300 and H310 all are perfectly fine; whereas the H700, H710, H800 and H810 are not.
    • Not 100% sure on the ones above H700 (too lazy to check)
  • While the IBM M1015 is mentioned, the others should be as well. LSI 9211-8i, Dell Perc H200, H300 and H310
  • Maybe update "If you are on v16 drivers then the firmware had better be v16 too, without exception. 9.3 will give you a warning if you break this cardinal rule." so that it pertains to v20 with 9.10.1?
    • Of course there is that false-positive v21 message...

Maybe toss in some examples of "Turn Key" systems that can be bought (for those not wanting to build their own)
  • Dell T20
  • Lenovo TS140
  • If your feeling *nice* maybe even a Dell C2100/FS12-TY
    • No worries if you don't since that is pretty old and I consider it for us "special" users. ;)
Are you sure you're looking at the right version? Some of the excerpts you quote sound like they're from cyberjock's old version.
 

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
Are you sure you're looking at the right version? Some of the excerpts you quote sound like they're from cyberjock's old version.
Lol, yeah I was... Ugh... Need to drink my coffee first before reading/responding to posts... :)
 

snaptec

Guru
Joined
Nov 30, 2015
Messages
502
Heading to cpu and FN 10.
some good cpus for fn9 won't be good for bhyve in fn10.

A turnkey solution for entry class could also be a hp microserver gen8 f.e.


Gesendet von iPhone mit Tapatalk
 

Sakuru

Guru
Joined
Nov 20, 2015
Messages
527

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Heading to cpu and FN 10.
some good cpus for fn9 won't be good for bhyve in fn10.
All the recommended processors support VT-x, and thus bhyve. Feel free to crawl through ark.intel.com and try to find some weird edge case: http://ark.intel.com/search/advance...CMemory=true&VTX=true&ExtendedPageTables=true

I dunno, seems out of place in the hardware recommendations...
 

MrToddsFriends

Documentation Browser
Joined
Jan 12, 2015
Messages
1,338
Copy-and-paste error on page 7:

the C2550 uses eight Atom cores -> the C2750 uses eight Atom cores
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
Okay, my comments on the correct one.

Additional SATA/SAS connectivity - SAS2
  • H310 is also one widely used
Additional SATA/SAS connectivity - Common mistakes
  • Would this be a good place to mention not using older backplanes/cards that do not support drives over 2.2TB?
SLOG devices
  • Perhaps mention other devices aside from the "Intel P7000" (Did you mean "Intel P3700")?
    • Intel DC S3500
    • Intel DC S3710
    • Possibly even a blurb about "High", "Medium", etc. versions/suggestions (or just point them to the existing thread ;) )
SLOG devices / L2ARC devices
  • Should anything be mentioned here about not trying to use a single SSD for both roles?

Abbreviations/Definitions - Would it be pertinent to have a table at the end which lists these?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
"Intel P7000" (Did you mean "Intel P3700")?
Oops, that's a typo.

Should anything be mentioned here about not trying to use a single SSD for both roles?
I think it's reasonably clear that they need to be very different.

Now that I've thought about it, I might include a link to jgreco's guide, since it does detail the hardware for the two roles.

Abbreviations/Definitions - Would it be pertinent to have a table at the end which lists these?
I can do that. Might use that to bootstrap a new version of jgreco's guide, too.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
R1b is out. Glossary is hereby postponed to a future version, given the workload.

Now, for a quick PSA: Resources can be subscribed to by watching them, like threads:

Capture2.PNG
 
Top