New storage server build - will it FreeNAS?

Mattias Hedman

Dabbler
Joined
Sep 5, 2016
Messages
21
So I am about to rebuild my storage server at home and this is whats going in:
ASUS TUF B450M-PLUS GAMING
AMD Ryzen 3 2200G 3.7 GHz 6MB
Corsair 16GB (2x8GB) DDR4 3200Mhz CL16 Vengeance
5 WD Red 2 Tb
3 WD Black Enterprise 2 Tb
1 Samsung EVO 850 500 Gb SSD
If need: 1 Samsung 840 250 Gb SSD
Delock PCI-E SATA card
All will be powered with a Crosair 1000 W PSU.

Will it work?
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Unless you are repurposing parts that are already owned by you, this is a terrible assortment of parts.
I hope you are not going to insist on using this with FreeNAS.
 

Mattias Hedman

Dabbler
Joined
Sep 5, 2016
Messages
21
Unless you are repurposing parts that are already owned by you, this is a terrible assortment of parts.
I hope you are not going to insist on using this with FreeNAS.
All but the three first ones are things I already own.
Why are they a terrible assortment?
 

racielrod

Explorer
Joined
Mar 3, 2018
Messages
74
Hi Mattias,

I'm a total rookie here but a few things that you will find it is general consensus here in the forum:
  • Stick to Intel architecture
  • Avoid gaming hardware - I think the main reason here is the Realtek chipsets you find on these boards.
  • When possible build with server grade hardware.
  • Always use ECC memory...
I think that is why @Chris Moore referred to this as "terrible assortment" but I'm not going to talk here on his behalf.
Guys like Chris have been providing guidance for years on proven hardware to build your FreeNAS... some of that hardware is old, but reliable and very accessible.

I think a good starting point might be the link on Chris' signature:
https://www.ixsystems.com/community/resources/links-to-useful-threads.108/

I'm putting my first build together after some research and a lot of reading:
https://www.ixsystems.com/community/threads/new-x9srl-f-build-fine-tuning.74551/

Thanks,
Rasiel
 
Last edited:

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
ASUS TUF B450M-PLUS GAMING
This system board does have a Realtek integrated network adapter which, among other things, makes in unsuitable.
Delock PCI-E SATA card
This is a totally unknown factor, but I wouldn't suggest using it.

This is a fair build of hardware, but I know it won't be available everywhere:
https://www.ixsystems.com/community...anges-to-upgrade-as-high-as-512gb-of-ram.110/

These resources point to other known working hardware:

Hardware Recommendations Guide Rev. 1e) 2017-05-06
https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/

Hardware Recommendations by @cyberjock - from 26 Aug 2014 - and still valid
https://forums.freenas.org/threads/hardware-recommendations-read-this-first.23069/

Proper Power Supply Sizing Guidance
https://forums.freenas.org/index.php?threads/proper-power-supply-sizing-guidance.38811/

Don't be afraid to be SAS-sy
https://forums.freenas.org/index.php?resources/don't-be-afraid-to-be-sas-sy.48/
I think that is why @Chris Moore referred to this as "terrible assortment" but I'm not going to talk here on his behalf.
Sounds like you have a pretty good grasp on my thinking. There is nothing wrong with AMD, but most of the low-end / Gaming boards have bad hardware. It is actually funny because they market those boards as being premium products but they are not really all that good.
 
Last edited:

Mattias Hedman

Dabbler
Joined
Sep 5, 2016
Messages
21
I have to say, the answers I got here got me to reconsider what to use, and I have made the decision not to use FreeNAS.
 

racielrod

Explorer
Joined
Mar 3, 2018
Messages
74
I have been in your place -joined this forum almost a year ago and I have been going back and forth on whether I want to implement FreeNAS or just go with something else and "keep it simple"
I have friends and co-workers that are happy with their consumer grade NAS appliances. I personally find them to be way overpriced for what you get.
Others are fine with hardware raid and a basic file server or are more than happy to pay for something like UnRaid.
"Your mileage might vary"

I don't think FreeNAS is for everyone, it depends on how involved you want to be and how curious you are.
I'm still putting my build together, asking questions and reading on a daily basis; so take my comments with a grain of salt - I still have a long road ahead.

What keeps me coming back to this forum and betting on FreeNAS is the support I find in the community, the amount of documentation and resources. It might take some time to get up and running, but when you do, you will have a solid solution that can run smoothly for years (plus you would be learning a TON in the process). I guess that is my "why".

Regardless of your decision of sticking to FreeNAS or using something else I'm going to echo something I read here a LOT:
Make sure you backup your valuable data; no matter what you use sh*t happens and when it does, it doesn't matter which file-system you are using.

Good luck!
 

Mattias Hedman

Dabbler
Joined
Sep 5, 2016
Messages
21
As of now, I have got so much better answers on other forums so I will go with:
Ubuntu Server Minimal on its own SSD.
ZFS RAIDZ2 with my 8 2Tb disks and an SSD for cache.
Backup is not a part of this, that a whole separate issue but that is solved with an external cloud partner with an awesome price connected to unlimited storage, just they wait... :)
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
As of now, I have got so much better answers on other forums so I will go with:
Ubuntu Server Minimal on its own SSD.
ZFS RAIDZ2 with my 8 2Tb disks and an SSD for cache.
Backup is not a part of this, that a whole separate issue but that is solved with an external cloud partner with an awesome price connected to unlimited storage, just they wait... :)
So, a good storage solution is based on people telling you what you want to hear. Nice. That is definitely the way to get the best possible outcome.
 

Mattias Hedman

Dabbler
Joined
Sep 5, 2016
Messages
21
@chrismoore Sigh. They are not telling me what I want to hear, I am having a good discussion with them and together we solved the question I had.
Im sorry if I misunderstand you.
 

Stevie_1der

Explorer
Joined
Feb 5, 2019
Messages
80
FreeNAS is aimed at NAS purpose, and additionally at rock-solid and reliable hardware to protect your data.
And the truth is, there are chips that are supported and proven to work under FreeNAS (and the underlying FeeBSD), others that don't work at all, and others that might work, but not as stable as you'd want.
So the recommendation is to stick with the supported and proven-good stuff like Intel NICs, Broadcom SAS and so on.
And in general, server-grade hardware is recommended because it is aimed at professionals and professional use, and is built to be stable and last longer.
Of course you could build up just nearly anything, install some NAS distribution and use it as a NAS, and it will most likely work, even Raspberry Pi can be used as such somehow.
But that way, it is more likely that something might happen someday, and your data is gone, or even worse, silently corrupted.
FreeNAS users usually want their data to be as safe as possible, and a stable system that can and will run untouched for years.
So they go for professional equipment, and often buy used components.
And here are many people that know what they are talking about, because they earn their money with server administration and such.

The answers you got here may sound a bit harsh, but they are true.
There is hardware that is well-supported and proven to work, and you should stick to that for a new build.
You would't buy a new Maserati with a trailer coupling to pull your trailer, you would buy a truck instead.
Repurposing already owned stuff is another thing, an old gaming rig or something found in the depths of an attic is nice to try if FreeNAS is for you, but almost always it is better to sell that stuff and get appropriate parts instead.

Here some more detailed explanations if your selected part are ok, and if not, why they are not ok:

5 WD Red 2 Tb
3 WD Black Enterprise 2 Tb
You can use your 8x 2TB disks for FreeNAS, of course.
The Blacks may be a bit louder and run hotter than the Reds, and maybe 24/7 on-time will shorten their lifetime, but you could replace them later.

1 Samsung EVO 850 500 Gb SSD
If need: 1 Samsung 840 250 Gb SSD
Your SSDs are actually too big for FreeNAS, it only uses about 16GB (and you cannot put anything else on that disk by default), so the rest of the space would be wasted. But if you already own it and have nothing else to use it for, you can use it for sure.
The recommendation is to use two mirrored 16GB+ USB sticks, or (better) a cheap used (server-grade) SSD like for example Intel 320 or S3500 DC with 40 or 80GB.

ASUS TUF B450M-PLUS GAMING
The mainboard you picked is aimed at gaming market. Many of its advertized features like RGB illumination are absolutely useless for a server, and onboard sound is genrally not supported at all in FreeNAS. So you waste your money on totally unnecessary things for a server.
And you get components that are not well-supported (like Realtek LAN) and might result into problems later.

Corsair 16GB (2x8GB) DDR4 3200Mhz CL16 Vengeance
For the RAM, you should go for ECC to protect your data. Storage goes bigger and bigger, and so bit errors are more likely to happen. The data on disks is secured by ZFS, but if it gets transferred wrong from RAM to disk, you need additional protection there, so ECC.
And the additional cost is not so high either.

Crosair 1000 W PSU
That PSU is overpowered, but of course you can use it if you already own it.

Delock PCI-E SATA card
The Delock SATA card could possibly work, but it could also not work at all, or (even worse) give you strange errors sometimes, I don't know.
So instead, you should go for a SAS HBA card (with Broadcom SAS2008 chip for example), this allows you to connect 8 drives directly, or by using expanders, even 24 or more drives.
And those are proven-good and rock-stable, as these are used in thousands of datacenters.
They are quite expensive if bought new, but used ones are fairly cheap.

In general, Intel is supported best, but there have been successful AMD builds as well.
So the mainboard could for example be a "Supermicro" or "ASRock Rack" board with ECC support, a supported CPU that supports ECC (Celeron, i3 or Xeon), and ECC RAM.
Server boards are ugly green instead of the sleek black gaming boards with fancy heatsinks and disco lighting, but they offer really useful features instead.
Like for example a feature called IPMI, with that you can access your server via browser, you don't ever need to hook up a keyboard, mouse or monitor to your server, you can even do BIOS settings with it, this is some really useful stuff.

What exactly did the guys from the other forums advise you?

You can get really good advice for really good combinations here, if you listen to the people with a vast amount of knowledge around here.
But you have to be willing to listen to them, they make their suggestions not to offend you, but to help you, free of charge.
 

garm

Wizard
Joined
Aug 19, 2017
Messages
1,556
As of now, I have got so much better answers on other forums so I will go with:
Ubuntu Server Minimal on its own SSD.
ZFS RAIDZ2 with my 8 2Tb disks and an SSD for cache.
Backup is not a part of this, that a whole separate issue but that is solved with an external cloud partner with an awesome price connected to unlimited storage, just they wait... :)

I'm genuinely curious, what question didn't you get answered?
Linux probably have a broader hardware support then FreeBSD, and in the process of setting up my latest server I contemplated rolling my own on Ubuntu. The FreeNAS configuration database is what makes me stick to FreeNAS. I have a system that probably took 40 hours to set up and a can replace the hardware after a catastrophic failure with a 77 kb file stored in a fireproof case. Im setting up a backup pool for multiple offsite locations using ubuntu server however, and some of my services do run on Ubuntu, but for main storage I will stick with FreeNAS for the foreseeable future
 
Joined
Oct 18, 2018
Messages
969
I have to say, the answers I got here got me to reconsider what to use, and I have made the decision not to use FreeNAS.
Hey @Mattias Hedman. I do agree with the general advice outlined above but I am somewhat new here as well. I just wanted to chime in with some more concrete advice that might help you get off the ground if you change your mind and come back to us. :)

5 WD Red 2 Tb
3 WD Black Enterprise 2 Tb
1 Samsung EVO 850 500 Gb SSD
If need: 1 Samsung 840 250 Gb SSD
These SSD's may be a bit large for a boot pool but if you already own them go ahead and use one. :)

I don't recommend using them as a SLOG (separate ZIL) device because they do not have power loss protection (PLP). What you're worried about here is if data is being written to the SLOG and the power is lost any data which is in-transit and not fully committed to permanent storage will be lost. This is bad news bears.

What you can do is use one of them to experiment with a SLOG device to learn how they work and if it gives you a boost in performance. If so, you can pick up a small-ish SSD with PLP to use as a SLOG when your budget allows.

Delock PCI-E SATA card
I don't know anything about this card so I won't say anything bad about it. Instead, I will say something good about another card that has excellent support in FreeNAS. This Dell H200 LSI-9210 HBA card contains two SAS ports, each containing 4 SAS lanes. You can use the card to connect 8 SAS or SATA drives. :) And it is only $60 and is sold by a very reputable reseller on ebay. If you need more drives than that you can pick up this IBM SAS Expander for $40 which when used with the HBA above supports 24 drives. HBAs and SAS expanders are great because they allow your storage to grow and give your operations system direct access to the drives which helps your os do a better job detect issues.

ASUS TUF B450M-PLUS GAMING
This is the primary change to your build in my opinion.The motherboard determines so much else about a build. It determines your CPU, memory type, bus support, etc. A lot of people come to FreeNAS with gaming motherboards and want to use them and are disappointed to learn that it may not be a good idea. The great news is you can get great, cheap, used server motherboards such as supermicro's x9 series boards for ~$50. Of course, you need memory to go with it. Luckily you can pick up single DIMMs at a time to spread out the expense. :)

Concrete Steps
You can pick up a great motherboard such as this Super Micro X9SCM-F for $60. Depending on what you want to do with your NAS this $75 Xeon E3-1240 v2 might be enough. And with memory I recommend you get 1 DIMM of the largest supported by the motherboard. For this motherboard you could pick up this $100 DIMM. You'll note these are VERY similar to my backup system. :) I am biased. But, for a basic into NAS it works great and has room to grow.

After that you are ready to go :) You can add more drives later, add more memory, an HBA and/or SAS Expander if you need to.
 

Mattias Hedman

Dabbler
Joined
Sep 5, 2016
Messages
21
Hey @Mattias Hedman. I do agree with the general advice outlined above but I am somewhat new here as well. I just wanted to chime in with some more concrete advice that might help you get off the ground if you change your mind and come back to us. :)
Thank you for this answer I wish it had come earlier, the system with the parts mentioned is built and is working better than I thought it would.
But my decision to leave FreeNAS still stands, and I am running Ubuntu Server, with ZOL. Works beautiful.
 

xcqn

Cadet
Joined
Mar 19, 2017
Messages
9
Thank you for this answer I wish it had come earlier, the system with the parts mentioned is built and is working better than I thought it would.
But my decision to leave FreeNAS still stands, and I am running Ubuntu Server, with ZOL. Works beautiful.
Are you using ECC ram with that build at least? As i understand you are still using ZFS as your filesystem, even though you changed your mind about FN.

If not, make sure you at least run memtest for a couple of days to detect any faillures. If your RAM is bad you can loose your data. ECC prevents that.

Ant reason in particular you don’t wanna go with server hardware? It doesn’t have to be more expensive. You get IPMI, ECC support better chipset, and other goodies.

It really doesn’t make any sense to buy equipment ment for gaming when youre in fact trying to build a server. Different requirements.

Skål!
 
Last edited:

garm

Wizard
Joined
Aug 19, 2017
Messages
1,556
Thank you for this answer I wish it had come earlier, the system with the parts mentioned is built and is working better than I thought it would.
But my decision to leave FreeNAS still stands, and I am running Ubuntu Server, with ZOL. Works beautiful.
How do you measure that beauty?
 

Bozon

Contributor
Joined
Dec 5, 2018
Messages
154
As of now, I have got so much better answers on other forums so I will go with:
Ubuntu Server Minimal on its own SSD.
ZFS RAIDZ2 with my 8 2Tb disks and an SSD for cache.
Backup is not a part of this, that a whole separate issue but that is solved with an external cloud partner with an awesome price connected to unlimited storage, just they wait... :)

What is the cloud partner you are going to use?
 
Joined
Oct 18, 2018
Messages
969
If not, make sure you at least run memtest for a couple of days to detect any faillures. If your RAM is bad you can loose your data. ECC prevents that.
Definitely agree that memtest is important in a build without ECC memory, or really any build. Here is a nice study attempting to understand the actual affects of memory corruption and ZFS. An important takeaway here

Zhang et al. 2010 End-to-end Data Integrity for File Systems: A ZFS Case Study.. 29-42. said:
In the last section we showed the robustness of ZFS to disk corruptions. Although ZFS was not specifically designed to tolerate memory corruptions, we still would like to know how ZFS reacts to memory corruptions, i.e., whether ZFS can detect and recover from a single bit flip in data and metadata blocks. Our fault injection experiments indicate that ZFS has no precautions for memory corruptions: bad data blocks are returned to the user or written to disk, file system operations fail, and many times the whole system crashes

You can take precautions if you choose to go without ECC memory. If your backup scheme involves running memtest on some interval (say every 3 months) and you keep backups until the next successful memtest run you'll have reason to be confident that your recent backups were not corrupted from bad memory. If a memtest run fails you can be confident that the backup from just prior to your last successful memtest run is safe. From there it is up to you how you choose to inspect new and modified files to confirm integrity. Of course, those that are corrupted may remain so if you don't have a non-corrupt backup. This scheme is not as good as ECC but is better than nothing, I think.
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,829
If memory serves, there are also software approaches that can verify the operation of non-ECC memory as it operates vs. relying on hardware like ECC RAM does (presumably by creating a checksum for every file as it is loaded and saved - lossy re: performance and not as failsafe as ECC RAM).

It comes down to how much you love your files. If you can live without them, by all means go with non-ECC RAM. But then, why bother with ZFS, sync writes, and so on? Just start running massive RAID-0 arrays.
 
Top