NEWBIE - First build mITX, ITX Home Server

[mijki]

Hello guys,

I spend some hours searching on the web and reading articles and different forums about Home Servers and I think I would like to build one by myself. I don't really like the idea of Synology or QNAP because of their restrictions. I tried to create some configurations for myself and I want to ask your help with choosing one. I wanted them to be future-proof and expandable in the future.

My main usage will be:

• TrueNAS Scale with accessible from the internet (maybe in a VM in Proxmox or ESXi?), I would use that to store private data and the data of my research group (this is why it should be reachable from the internet). I want to use Traefix or Nginx and Cloudflare for security and Next Cloud for Access on different devices (laptops, mobile, etc.)
• Running my own website/web app --> this could run in a separate VM
• VM with Windows 11 to run different simulations (like electrical grid current flow) on Win environment
• Maybe some Plex/Jellyfin server in the future for 4-5 people

I created one Config for "minimum":

Part list	Intel	Ryzen
Case	Fractal Design Node 304	Fractal Design Node 304
Supply	EVGA SuperNOVA 550 GM SFX+ATX	EVGA SuperNOVA 550 GM SFX+ATX
Motherboard	MSI B560M PRO-VDH	ASRock B550M Pro4
CPU	Intel Pentium Gold G6405 Dual-Core 4.1GHz	AMD Ryzen 3 4300G 4-Core 3.8GHz
RAM	Kingston FURY Beast 16GB (2x8GB) DDR4 3200MHz	Kingston Server Premier DIMM 16GB, DDR4-2666 ( KSM26ED8/16HD)
Cache storage	-	-
Boot storage	Kingston A400 2.5 480GB SATA3 (2 pcs., mirrored)	Kingston A400 2.5 480GB SATA3 (2 pcs., mirrored)
Storage	Seagate IronWolf 4TB SATA3 (ST4000VN006) (4pcs.)	Seagate IronWolf 4TB SATA3 (ST4000VN006) (4pcs.)

And one for a bit more money, I called that "Medior":

Part list	Intel	Ryzen
Case	Fractal Design Node 804	Fractal Design Node 804
Supply	EVGA SuperNOVA 550 GM SFX+ATX	EVGA SuperNOVA 550 GM SFX+ATX
Motherboard	B660M Steel Legend	ASRock B550M Pro4
CPU	Intel Core i3-12100 4-Core 3.30GHz LGA1700	AMD Ryzen 5 4600G 6-Core 3.7GHz AM4
RAM	Kingston Server Premier DIMM 16GB, DDR4-2666 ( KSM26ED8/16HD)	Kingston Server Premier DIMM 16GB, DDR4-2666 ( KSM26ED8/16HD)
Cache storage	Samsung 980 1TB M.2 PCIe	Samsung 980 1TB M.2 PCIe
Boot storage	Samsung 870 QVO 2.5 1TB SATA3 (2 pcs., mirrored)	Samsung 870 QVO 2.5 1TB SATA3 (2 pcs., mirrored)
Storage	Seagate IronWolf 4TB SATA3 (ST4000VN006) (4 - 8 pcs.)	Seagate IronWolf 4TB SATA3 (ST4000VN006) (4 - 8 pcs.)

Just some notes:

• ASRock states on its website, these MoBos are working with ECC, but I want to show you guys first.
• I can leave mITX behind and get a Fractal Define if necessary.
• I would like to use the ECC if possible, but most of the server MoBos are too expensive.
• I read here that, if I want proper ECC with Ryzen, I should get one without APU or a PRO G version.

Please feel free to critique my configs and add recommendations to be better, every help is thanked.

And thank you in advance!

 

[Etorix]

Generally, we do not like consumer/gamer motherboards for a NAS build, and anything with a Realtek/Killer NIC is a big fat "no!"—unless you intend to add a proper server-grade NIC through a PCIe card.

Hardware Recommendations Guide

This is the latest edition of the FreeNAS Community hardware recommendations guide. The current major version is R2, dated January 2021, with the last minor update on 2021-01-24. The format has moved away from the forum post form factor, to...

www.truenas.com

Is my Realtek ethernet really that bad?

One of the longest running issues faced here in the forums may be people who come to these forums, wanting to recycle an old PC into a NAS. In general, this is a bad idea because typical PC hardware isn't optimized towards the task. However...

www.truenas.com

16 GB RAM is the minimum for TrueNAS… assuming it only serves storage; VMs come on top of that, so you need more RAM. And certainly more than 2 cores. 64 GB RAM is the minimum to even consider a L2ARC—if that's what you mean by "cache".
Mirroring the boot device is of limited use, but apps and VMs may use their own (mirror) NVMe pool, especially if the main HDD storage is raidz2.

I have to waste an entire drive just for booting?

With the advent of TrueNAS Scale, we seem to be getting a large number of people from the Linux community who are not quite grasping what TrueNAS is. TrueNAS is not a "Linux distro" and it is not designed to try to meet your arbitrarily spec'd...

www.truenas.com

Virtualising TrueNAS would add an extra layer of complication, especially on a consumer platform.

"Absolutely must virtualize FreeNAS!" ... a guide to not completely losing your data.

[---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some specific things known to be problematic ----] [---- 2014/12/24: Note, there is another post discussing how to deploy a small FreeNAS VM instance for basic file...

www.truenas.com

 

[Arwen]

Boot drives of 480GB or 1TB are way too much. Unless they are already available, something on the order of 32GB is all you need. It is perfectly fine if you want a name brand and their smallest size is 120GB or 240GB. This is more of "save your money for things that can use the money".

And I second @Etorix comment about both CPU, (4 cores as a minimum), and more memory if you plan on VMs or Apps.

 

[ChrisRJ]

I would avoid ITX because it is limiting in so many ways. Yes, you will end up with a bigger case. But you have a much bigger number of boards to choose from.

Also, I would look into used enterprise gear. Board, CPU, and RAM will last at least 10 years. I would only avoid HDDs and the PSU is a bit of a grey area.

 

[mijki]

@Arwen @ChrisRJ
Firstly thank you guys for your response. So let's say I would change the MoBo and the CPU for this:

For AM4:
What about Ryzen PRO 3400G / 3700 with dGPU: 1030 on these MoBo-s?
ASRock Rack X470D4U AM4, 4x DDR4 ECC, 6x SATA, 2x M.2(22110), 3x PCIe, 2x LAN, IPMI or the same just with 2 x 10 GbE, but it is much expensive:
ASRock Rack X470D4U AM4, 4x DDR4 ECC, 6x SATA, 2x M.2(22110), 3x PCIe, 2x 10Gb LAN, IPMI

Or for AM4, it is worth changing to X570 with a 5750G PRO for example:
ASRock Rack X570D4U AM4, 4x DDR4 ECC, 8x SATA, 2x M.2(22110/22080), 3x PCIe, 2x LAN, IPMI

On Intel side I found two Supermicro with LGA1200, so maybe I would use it with 10th gen i3:
SUPERMICRO MB LGA1200 (Xeon E3-2300), C256, 4xDDR4, 8xSATA3, M.2, 3xPCIe4.0 (x16, 2 x4, x2), VGA, 4x LAN, IPMI
SUPERMICRO MB LGA1200 (Xeon E3-2300), C256, 4xDDR4, 8xSATA3, M.2, 3xPCIe4.0 (x16, 2 x4, x2), VGA, 4x LAN, IPMI

And two ASrock:
ASRock W480M WS
ASRock W480D4U

And one ASUS:
ASUS P12R-M/10G-2T, P12R-M/LGA-1200,C252,MICROATX, 4*DIMM, 1*PCIe x8 slot, 1*PCIe 16 slot, 6*SATA ports, 1*M2, 1 x Dual

I don't know which would be the best for me, could you help me decide between these? On my desktop, I have a 5900X and I'm pretty satisfied with it. But I read a lot Intels are better for being a home server.

 

[mijki]

For Cache I could use an Samsung 970 M2 SSD, they are pretty cheap and for boot Kingston A400 SATA3 SSDs are cheap also.

 

[Etorix]

An AsRockRack Ryzen server board with a PRO Ryzen is the best attempt one can make to have working ECC with Ryzen. The choice for 10 GbE depends on whether you need it… A (second-hand) Intel/Chelsio/Solarflare card is also an option, and gets you SFP+ rather than copper.
As for the CPU, it depends on the use, primarily the VMs/apps, as a 3400G is more than enough to serve files.

On the Intel side, C256 boards require a Xeon E-2300 (not a Core i3), and W480 will only do ECC with a Xeon W-1200/W-1300 so there is a "Xeon tax". Savings were with the previous generations, where you could use a Core i3 with a C236 or C246 motherboard and have ECC (no W chipset in these generations). But C236/C246 boards are now hard to find, or overpriced.

Any cheap and small SSD will do for boot. Again, L2ARC requires at least 64 GB RAM to begin with: Using a L2ARC uses RAM!
Keep the 970 (or a pair of those) to host your apps/VMs. Don't bother with a cache as first intention; you can add one later if you do find in actual use that L2ARC would help.

 

[mijki]

So if I choose the Ryzen path, should I go with the X470 or the newer B550, X570? I guess it is worth the price because I can use later Ryzens if it is needed.

On the other hand, in my price range - this ASUS P12R-M, P12R-M fits - but I read nothing about it on the internet. Is it good or should I look somewhere on Amazon / eBay? I'm living in Europe, so the market is a bit different than in the US.

And just one more thing during idling, Ryzens or Xeons consume less current, I saw some tests which showed that Ryzens eat more.

 

[mijki]

And one more thing, can you recommend RAM for the Ryzen build?
Thank you very very much.

 

[Etorix]

So if I choose the Ryzen path, should I go with the X470 or the newer B550, X570? I guess it is worth the price because I can use later Ryzens if it is needed.

The choice is yours. These AM4 boards are stuck to Zen3 and earlier anyway.
For RAM, you may check the QVL from AsRockRack for security—but generally any ECC module from Micron/Samsung/SK Hynix should work.

On the other hand, in my price range - this ASUS P12R-M, P12R-M fits - but I read nothing about it on the internet. Is it good or should I look somewhere on Amazon / eBay?

No idea. We tend to avoid latest generation motherboards and CPU for our NAS because new hardware is expensive and a NAS does not require great computing power. Check compatibility carefully if you plan to do transcoding for Plex because with the previous generation this required C246, not C242.

 

[mijki]

@Etorix
So I modified it a bit as we talked, so what do you think about this? Which one could be better?

Part	Intel	Ryzen
Case	Fractal Design Node 804	Fractal Design Node 804
Supply	EVGA SuperNOVA 550 GM SFX+ATX	EVGA SuperNOVA 550 GM SFX+ATX
Motherboard	Supermicro MBD-X11SCH-F	ASRock Rack X470D4U AM4
CPU	Intel Xeon E-2224G or i3-9100	Ryzen 7 PRO 4750G or Ryzen 5 PRO 3400G
RAM	Supermicro 16GB DDR4 2666 (PC4-21300) MEM-VR416LD-EU26) (4 pcs.)	NEMIX RAM 32GB Replacement for Samsung M391A4G43MB1-CTD DDR4-2666 ECC UDIMM 2Rx8 (2 pcs.)
Cache storage	Samsung 970 EVO Plus 250GB M.2 PCIe	Samsung 970 EVO Plus 250GB M.2 PCIe
Boot storage	Kingston A400 2.5 480GB SATA3 (2 pcs., mirrored)	Kingston A400 2.5 480GB SATA3 (2 pcs., mirrored)
Storage	Seagate IronWolf 4TB SATA3 (ST4000VN006) (4pcs.)	Seagate IronWolf 4TB SATA3 (ST4000VN006) (4pcs.)

 

[WI_Hedgehog]

Intel: 4-bit ECC, which is really important given ZedFS chews through RAM like a fat man at an all-you-can-eat fried chicken buffet.

AMD: 1-bit ECC with certain tested hardware configurations. Maybe. See:

Ryzen 3 1200, ECC Memory Check?

I have yet to find a post regarding Ryzen 3 compatibility to ECC memory. I did the dmidecode -t memory test and following data came out: Handle 0x0032, DMI type 17, 40 bytes Memory Device Array Handle: 0x0029 Error Information Handle: 0x0031 Total Width: 128 bits...

www.truenas.com

AMD Ryzen with ECC and 6x M.2 NVMe build

I wouldn't personally run docker/kubernetes off spinning disks - SSD all the way imo My docker/kubernetes is running from 3x M.2 NVMe array (R-Z1) ..

www.truenas.com

Ryzen 5 PRO 4650 Build (new to Truenas)

Hi, I've been running a "nas" with drives attached to a PI4, and I want to finally move to a real solution, however my budget is not unlimited. I've read this forum and research the topic and it seems that a ECC memory is very important, am I wrong? Long story short, at 150EUR I could find the...

www.truenas.com

[SOLVED] Crash after a couple mins of copy: Fatal Trap 9: General Protection Fault in kernal mode

A physical inspection of the modules (especially the pins) and the slots might help. Dust can cause issues too. Reseating the CPU is also an option.

www.truenas.com

 

[Etorix]

Either list looks mostly fine.
Not sure why you list 16 GB for Intel but 32 GB for AMD.
The proper terminology for "read cache" is "L2ARC". Again: L2ARC requires RAM to work. A minimum of 64 GB RAM is recommended to use a L2ARC, and a rule of thumb for sizing is L2ARC= 5*RAM —at the utmost, 10*RAM. If there were no apps and no VMs, 256 GB L2ARC with 32 GB RAM might work, though testing would be required. With 16 GB RAM or with apps/VMs on top of NAS duty, you will almost certainly degrade performance by adding a L2ARC. (If you meant "write cache", you're all wrong! A SLOG is not awrite cache and a 970 Pro is unsuitable as SLOG..)
Mirroring the boot drive is of limited use for home. Better have a single boot device, keep a copy of the configuration file to reinstall if needed, and one or two M.2 drives as a separate pool for apps/VMs.

 

[ChrisRJ]

Not sure why you list 16 GB for Intel but 32 GB for AMD.

The list is 2*32 GB for AMD and 4*16 GB for Intel. But I also stumbled over this

 

[WI_Hedgehog]

Regarding the boot drive:

Go from 1 hdd boot to 2 new ssd's

Hi guys. I have TrueNAS-SCALE-22.12.0 boot from old 320GB hdd and I would like to replace it with 2 new 120GB SSDs in mirror raid. After weeks of setting it up, it would be terrible to lose it. What is the easiest method to achieve this? I am not very familiar with TrueNAS or Linux. Bearly...

www.truenas.com

 

[mijki]

Firstly I want to apologize if my table was not easily readable. Thank you for your help guys!
These 4-bit ECC (or AECC) is for 3rd Gen Xeon Scalable only if I read correctly and they are a bit out of my budget, unfortunately.

About the Cache / Boot drives, so 1 piece of SATA SSD for Boot and 2 pieces of M2 for the Apps/VMs. I got it.

@Etorix may I ask you to tell me more about the "write cache" / SLOG?

Thank you in advance guys :)

 

[Etorix]

Here you go:

Some insights into SLOG/ZIL with ZFS on FreeNAS

What is the ZIL? POSIX provides a facility for the system or an application to make sure that data requested to be written is actually committed to stable storage: a synchronous write request. Upon completion of a sync write request, the underlying filesystem is supposed to guarantee that a...

www.truenas.com

TL;DR A SLOG is only useful for sync writes (NFS, iSCSI, databases, business critical VM). Even with a good SLOG (Optane, Radian RMS-200/300), sync writes are still an order of magnitude slower than async.

Terminology matters. It is best to always refer to 'SLOG' or 'L2ARC' rather than 'cache'.

 

[mijki]

Wow, thank you very much :) You guys helping me a lot :)

And one more thing, TrueNAS should be installed "bare metal" or with a Supervisor like ESXi or Proxmox if I want to run a VM with Windows and a webserver for a webpage? Or the containers of the TrueNAS can do that?

Sorry if this is a stupid question.

 

[Etorix]

It's certainly not a stupid question. Carefully read the resource linked in post #2 before virtualising TrueNAS in ESXi or Proxmox. Doing so may also introduce further hardware requirements with respect to proper PCI pass-through—though your last choice of server motherboards should be fine.
Running TrueNAS bare metal is safer for your data. A container in SCALE, or a jail in CORE, can certainly run a web server. I cannot comment on running Windows as VM in bhyve (CORE) or kvm (SCALE).

 

[WI_Hedgehog]

These 4-bit ECC (or AECC) is for 3rd Gen Xeon Scalable only if I read correctly and they are a bit out of my budget, unfortunately.

3rd generation and later Xenon Scalable processors (4th gen. is current).

New is spendy. Used datacenter servers are often suggested because they usually have a lot of life in them, are affordable, and $500 to your door* gets a server with enough power for 3 high-definition video editors**, or an office of 35 average users, or 125 blonde college students.*** TrueNAS doesn't take that much CPU power, just RAM. The more the better. (ECC RAM. Don't be cheap.)

$700 will get you an absolute powerhouse. Buy new if you want, but 15 minutes after you plug it in it's "used." Just sayin'...

Like @Etorix**** said, the more stuff you run on a server the more chance there is for bad things to happen. How important is your data?

---
*plus drives, but you pay for those regardless.
**or editrixes, depending.
***I'm thinking of the average modeling school, for "general perspective," not because they're my favorite service call.
****Sounds similar to editrix, no?