SOLVED First NAS Build Help

[stavroswtf]

Hello guys.
So I am pretty new to this. I have been reading some of the documentation, the hardware guide and watching some YouTube videos and I have come up with this build here: https://de.pcpartpicker.com/list/yYxYBL.

My main use would be having 5 users accessing small files (under 10mb, mostly under 5mb) through Windows SMB share and running a Windows Server VM 24/7 with VPN accessibility. My internet speed is pretty limited (100mbpps). I was thinking of using 4-6 HDDs (haven't really decided yet) with cloned NVME drives for boot and 2 SATA SSDs for caching.

Is this build overkill? Would caching be redundant for my use case and internet speed? Any useful input welcome.

 

[artlessknave]

insufficient RAM for considering L2ARC (level 2 read cache). there is no disk based write cache. the zfs R/W cache is in RAM already (ARC). because of this, you will generally get the most performance by adding more RAM for ZFS to use.
as that's a workstation motherboard it's not an amazing choice for a server. better than a gaming mobo though.
mobo supports ECC but you appear to have chosen non ECC.
nvme is kind of silly for boot, unless you are using all the sata for data disks. even then, just get an HBA and add 8+ disks. nvme is usually used for VM/app/jail storage.

 

[DigitalMinimalist]

Is this build overkill? Would caching be redundant for my use case and internet speed? Any useful input welcome.

Yes, if you only access via Internet with 100Mbit (=12.5MB/s). I would even suggest a simple mirror - more than enough…
2x16TB instead of 6x4TB RAIDZ2

No caching SSDs and Mirrored OS is nice to have: you can also export your config and can get TrueNAS up and running in approx 30-60Min…

I suppose you chose the C246 with 9100 for „ECC on a budget“?
I would recommend AMD AM4 instead. Asus and Asrock with a Ryzen 5600, or 4650G and ECC RAM (consider 32GB UDIMM modules)

 

[stavroswtf]

insufficient RAM for considering L2ARC (level 2 read cache). there is no disk based write cache. the zfs R/W cache is in RAM already (ARC). because of this, you will generally get the most performance by adding more RAM for ZFS to use.
as that's a workstation motherboard it's not an amazing choice for a server. better than a gaming mobo though.
mobo supports ECC but you appear to have chosen non ECC.
nvme is kind of silly for boot, unless you are using all the sata for data disks. even then, just get an HBA and add 8+ disks. nvme is usually used for VM/app/jail storage.

1. So I guess using those SATA SSDs would be useless. Got it.
2. It says on PCPartPicker that the RAM I chose is ECC and unbuffered. Isn't that true at all?
3. What would you recommend for booting + storing my OS on then?

 

[stavroswtf]

Yes, if you only access via Internet with 100Mbit (=12.5MB/s). I would even suggest a simple mirror - more than enough…
2x16TB instead of 6x4TB RAIDZ2

No caching SSDs and Mirrored OS is nice to have: you can also export your config and can get TrueNAS up and running in approx 30-60Min…

I suppose you chose the C246 with 9100 for „ECC on a budget“?
I would recommend AMD AM4 instead. Asus and Asrock with a Ryzen 5600, or 4650G and ECC RAM (consider 32GB UDIMM modules)

1. I was thinking of adding more drives with less storage for more redundancy (as my files are really important) and faster resilvering time if a drive gets fried. Wouldn't be more ideal than choosing just 2 drives?
2. Are those chips energy efficient at idle? Would I have a problem with ECC being unofficially supported by those AMD chips/mobos?

 

[artlessknave]

2. It says on PCPartPicker that the RAM I chose is ECC and unbuffered. Isn't that true at all?

my mistake, they are indeed ECC.

So I guess using those SATA SSDs would be useless. Got it.

it depends what you use them for. I would boot from sata SSD's and use the nvme drives for fast storage. maybe a metatdata L2ARC, or metadata special vdev (this makes it a permanent part of the pool though)
many people come to zfs and think that "cache" means the same thing as it does in something like unraid. it does not.

unraid implements a read/write cache disk that sends ALL writes there first before putting them to the slow storage disks using a tiered cache system. zfs does not do this; everything is already cached in RAM; zfs will use all unused RAM for ARC, bundling writes into transaction groups that get flushed to disk in predictable intervals as contiguous writes and caching the most frequently used files for quick access.

another common misunderstanding, one that is frustratingly reinforced by IX, is that SLOG is a "write cache", which it outright is not. at all. SLOG functions as a backup of writes in ARC; its only used for sync writes, and will never be read from in normal operation.


if you have GB nic ZFS will likely outperform your network whether its mirrors or raidz. mirrors, however, have better management, as you can add and remove vdevs and the drives IN a vdev. this cannot (yet) be done with raidz

 

[stavroswtf]

I would boot from sata SSD's and use the nvme drives for fast storage. maybe a metatdata L2ARC, or metadata special vdev (this makes it a permanent part of the pool though)

Any documentation/videos on how to accomplish this and maybe something to quantify the gain in performance vs not using those 2 NVMs at all?

 

[DigitalMinimalist]

1. I was thinking of adding more drives with less storage for more redundancy (as my files are really important) and faster resilvering time if a drive gets fried. Wouldn't be more ideal than choosing just 2 drives?

I was contemplating for weeks if I should choose striped mirror or RAIDZ2 for weeks.

RAIDZ2: better resilience, as any 2 disks can fail

Striped mirror: only one disk per mirror can fail, but better speed, resilvering is faster/easier and pool can be expanded in the future.

I chose 4 disks for lower power consumption and price per TB (new). I always prefer to have an automated backup of my important data vs. relying that no highly improbable scenario occurs.

Therefore I‘m currently in the process to build a second TrueNAS Scale backup server with leftover parts / bargains and will do a weekly replication. Server will have 8-12 4TB drives (I bought a few used SAS drives for 10€/pcs) - power consumption irrelevant as it only boots up time controlled for the backup…

As you are in Greece/Europe:

EU - Gigabyte Mainboard MC12-LE0 Re1.0 AMD B550 AM4 Ryzen

https://www.piospartslap.de/Gigabyte-Mainboard-MC12-LE0-Re10-AMD-B550-AM4-Ryzen-5000-4000-3000-Server-Board-NEU-NEW?pio_show=1 53,10€ for people in Germany, with discount code "Advent2023". Other EU states have to pay some shipping costs.

forums.servethehome.com

 

[stavroswtf]

I was contemplating for weeks if I should choose striped mirror or RAIDZ2 for weeks.

RAIDZ2: better resilience, as any 2 disks can fail

Striped mirror: only one disk per mirror can fail, but better speed, resilvering is faster/easier and pool can be expanded in the future.

I chose 4 disks for lower power consumption and price per TB (new). I always prefer to have an automated backup of my important data vs. relying that no highly improbable scenario occurs.

Therefore I‘m currently in the process to build a second TrueNAS Scale backup server with leftover parts / bargains and will do a weekly replication. Server will have 8-12 4TB drives (I bought a few used SAS drives for 10€/pcs) - power consumption irrelevant as it only boots up time controlled for the backup…

As you are in Greece/Europe:

EU - Gigabyte Mainboard MC12-LE0 Re1.0 AMD B550 AM4 Ryzen

https://www.piospartslap.de/Gigabyte-Mainboard-MC12-LE0-Re10-AMD-B550-AM4-Ryzen-5000-4000-3000-Server-Board-NEU-NEW?pio_show=1 53,10€ for people in Germany, with discount code "Advent2023". Other EU states have to pay some shipping costs.

forums.servethehome.com

Thanks for the input man. Will look into this board if I choose to build something big enough to fit a Micro ATX. Any other somewhat comparable ITX boards or isn't it worth considering? Was thinking of 3D Printing a NAS Case + Disk Bay.

 

[DigitalMinimalist]

I have the Gigabyte MC12-LE0 here and it runs nicely with 4650G and 128GB ECC RAM and TrueNAS Scale

17W with one NVME :) And for less than 400€ (got the RAM for 200€ from eBay)
Real steal…

I would always max out RAM first, before even considering L2ARC.
As you are mainly using SMB (async), SLOG isn’t really helping you.
special vDev for Metadata might be an option to make access „snappier“…

When you have multiple HDDs: How much is your gain of going ITX? Minimal, I would say

 

[stavroswtf]

When you have multiple HDDs: How much is your gain of going ITX? Minimal, I would say

This is my dream ITX case right here (https://modcase.com.au/products/nas). It fits like 10 drives while being aesthetic and somewhat compact. That's why I would be really excited for an ITX build. Your build is quite promising though, will have it as an alternative since the motherboard is 200€ cheaper than my Intel one while also being of higher quality.

 

[DigitalMinimalist]

Search View - Inter-Tech Elektronik Handels GmbH

www.inter-tech.de

If you want a cheap aesthetic micro ATX with up to 5 HDDs
I don’t have it, as I went for rackmount

 

[stavroswtf]

Search View - Inter-Tech Elektronik Handels GmbH

www.inter-tech.de

If you want a cheap aesthetic micro ATX with up to 5 HDDs
I don’t have it, as I went for rackmount

That's perfect, thanks a lot man!