Truenas Scale Energy efficient setup & build

[wlknsn]

Hi all,

The goal for my new setup is to have a relatively lean setup in terms of power consumption yet have the flexibility to run a few apps. The system will be used as a file repository, backup manager, media server and 2 vm's. However, the backup and file storage will not be used very regularly (once/twice daily). The apps will probably be used more frequently. What I'm trying to find out is if my selected hardware is "ok" and what config is ideal for this.

System:
CPU: AMD Ryzen 7 5700G (+Noctua NH-L12S cooler)
MB: Asrock B550 Phantom Gaming-ITX/ax (it's going into a Jonsbo N1 case)
RAM: DDR4 2x 32GB (ECC) Samsung M393A4K40DB3-CWE 32
PSU: EVGA SuperNOVA 550 GM, 80+ GOLD 550W PSU (too much?)
HDD: 4x WD RED PLUS 8TB (5400rpm - less energy consuming)

Boot: some small m2 SSD (it seems pretty irrelevant from what I read)
Cache/apps: WD RED SN700 2TB Nvme

From a hardware perspective:
* Isn't that PSU overkill and would that be the leanest option to support all the above?
* The cache/app HD: Will it matter if its 1x M2 nvme, or could I better have an adapter and have two SSD instead which are mirrored? I guess the latter is more secure but is it overkill given the light load?

Presuming the above makes sense, If I:
* try & create a pool for the more commonly used app parts (on the SSD / nvme),
* and separate one for storage and backup (on the HDDs),
=> would this result in the HD's being only spun on demand for instance when files are being access and/or backups are being made? Or is there another better approach for this.

Also, what raid config would you apply on all this or other config that you might be of interest?

Any tips are very welcome!!!!

 

[ChrisRJ]

Why did you decide to go for TrueNAS? This will help us understand your motivation and by that provide better support. If by any chance the reason is this video from LTT (https://www.youtube.com/watch?v=boKmZKTKXHc), it was not good advice. That channel is about entertainment, not teaching technology. And they have history with TrueNAS that is not to their credit.

The bottom line is that getting something to boot and survive a few superficial tests, is not the benchmark for building a NAS. You want to entrust your valuable data to the latter and not play Russian roulette. This sounds harsh, I know. If you want good videos on TrueNAS go to https://www.youtube.com/@LAWRENCESYSTEMS. And, of course, spend time on the forum here.

Please also have a careful look (i.e. read them thoroughly) at the first couple of links from "Recommended readings" in my signature. TrueNAS/ZFS is different in some aspects from the stuff you are familiar with. With little or no background chances are you are going to shot yourself in the foot. And we don't want that .

 

[Etorix]

The "Gaming" motherboard is not optimal… by virtue of being consumer-grade.
It does have an Intel NIC, but this is the i225, which is not the best of the bunch (an i210 would have been safer).
ECC RAM, though whether it does work as intended on non-Pro Ryzen with a gaming motherboard is an open question.
The PSU is adequate, and there's not much to spare here by down sizing anyway—rather look at older second-hand/refurbished motherboard/CPU/RAM for savings.
(The HDDs actually spin at 7200 rpm but WD markets them as "5400rpm-class", whatever that means, with some crippling in firmware. There are no longer any 5400 rpm HDD suitable for a ZFS NAS on the market anyway.)

The boot drive is indeed of little relevance.
A single drive application pool means that you'd have to reinstall (anew or from backup) in the event of failure, which may be acceptable in a home setup. A second drive in mirror would allow for redundancy—and avoid potential headaches.
But "cache"???

For a first attempt, you're not too far. But this can be further improved with some extra reading.

 

[jgreco]

PSU: EVGA SuperNOVA 550 GM, 80+ GOLD 550W PSU (too much?)

Not really.

Proper Power Supply Sizing Guidance

I've seen about 1,000 threads like this one where people decide that they can power a dozen hard drives off a 360 watt supply. DO NOT DO THIS. I've seen another 1,000 threads where people decide to buy the cheapest power supply that they can find. DO NOT DO THIS. Your NAS lives or dies by...

www.truenas.com

 

[WI_Hedgehog]

It seems WD drives with a capacity of 8TB or less and the smaller 128 MB cache actually spin at 5400 RPM. The 256 MB cache drives are 7200.

That looks like a great mainboard for gaming, they probably tweaked it right to the edge of stability for cutting-edge gaming performance. If you want to play games with your data, however, that will possibly have an ending you're less than pleased with.

One error in 10^14 transactions may not seem like much, at least until you're regularly pushing around a bunch of important data and your RAM, data transfers, and hard drives are each somewhere around 1 in 10^14 and you start multiplying the probability an error will happen in one or more systems, then it's game-on.

For reference, Linus of LTT has very smart people working for him, and he's great at marketing, but his level of maturity isn't "mature" (yet). LTT built up a TrueNAS system in their typical fajigger-it fashion and, like often happens with them, it broke (and cost LTT about $15,000 CAD to recover). I don't know how often you've lost data, but if you've been through it enough times you eventually want to take reasonable and sensible precautions up front to avoid losing said data. If I remember that's something like Linus 4th time of experiencing data loss and the event that pushed him into getting a tape backup.

Mind you, we're not telling you that you can't build that system, we're simply saying TrueNAS/ZFS might not be the best thing to put on it. You might want to go with something less resilient and more flexible than TrueNAS given what you're wanting to do, which of course is your choice. The best of both worlds might be to keep your data on a dedicated TrueNAS box and run a Plex server on another box. Yes, that would use more power. However, when you understand how often errors actually occur in a system you might just want to keep your data storage system running lean and clean, and everything else that could go "a little" sideways on a different machine.

 

[NickF]

I wouldn't build a NAS with anything smaller than mATX, you're really limiting yourself with only 1 PCIE slot. You can have low power and realtively small size without going ITX.

 

[Etorix]

Micro-ATX is the right size for a server motherboard with a socketed processor.
Embedded boards, with soldered processors do fine even in mini-ITX size. X10SDV (Xeon D-1500) boards provide 6 SATA ports, which is enough is enough for a small NAS (think Jonsbo N1 or Fractal Design Node 304 cases); ver. 2 boards can bifurcate their x16 slot all the way to x4x4x4x4 for a decent NVMe pool. A2SDi (Atom C3000) boards provide up to 12 SATA on board. Both series have boards with on-board 10 GbE, which keeps the PCIe slot free for either a HBA or extra NVMe drives. Both series make for great, energy-efficient, NAS with little boards (and possibly lots of cheap RDIMM RAM), though the ca. 2 GHz CPU may be limiting for SMB, or if the NAS is further intended to run VMs/apps which require substantial computing power.

 

[wlknsn]

Why did you decide to go for TrueNAS? This will help us understand your motivation and by that provide better support. If by any chance the reason is this video from LTT (https://www.youtube.com/watch?v=boKmZKTKXHc), it was not good advice. That channel is about entertainment, not teaching technology. And they have history with TrueNAS that is not to their credit.

The bottom line is that getting something to boot and survive a few superficial tests, is not the benchmark for building a NAS. You want to entrust your valuable data to the latter and not play Russian roulette. This sounds harsh, I know. If you want good videos on TrueNAS go to https://www.youtube.com/@LAWRENCESYSTEMS. And, of course, spend time on the forum here.

Please also have a careful look (i.e. read them thoroughly) at the first couple of links from "Recommended readings" in my signature. TrueNAS/ZFS is different in some aspects from the stuff you are familiar with. With little or no background chances are you are going to shot yourself in the foot. And we don't want that .

Via friends who have it running. I was planning on buying a new Synology (RS822+) but it's extremely expensive for the hardware you end up getting. Those friends then told me that you might as well look into something like Truenas instead, which I did.

I then stumbled upon that case via LTT which I liked a lot so wanted to go with that. Then there is the video of Wolfgang about energy efficient NAS which also relates to a google sheet that is out there with a number of very energy efficient builds. From this I wanted to go for an Intel CPU with iGPU to support some minimal decoding but turns out that to have ECC & iGPU you could take a good i5 but need a very expensive motherboard to make the ECC work. So then turned to AMD 5700H which had very nice power consumption results vs performance in that list and has GPU & ECC. So that's how I got to this build. Unfortunately, motherboard availability overall is not great so there are not a lot of choices apparently to build something.

Anyway, I bought all the components now, just hope I can set it up in a way that it's not running full power all the time (for which I'm hoping still to get some feedback).

 

[wlknsn]

It seems WD drives with a capacity of 8TB or less and the smaller 128 MB cache actually spin at 5400 RPM. The 256 MB cache drives are 7200.

That looks like a great mainboard for gaming, they probably tweaked it right to the edge of stability for cutting-edge gaming performance. If you want to play games with your data, however, that will possibly have an ending you're less than pleased with.

One error in 10^14 transactions may not seem like much, at least until you're regularly pushing around a bunch of important data and your RAM, data transfers, and hard drives are each somewhere around 1 in 10^14 and you start multiplying the probability an error will happen in one or more systems, then it's game-on.

For reference, Linus of LTT has very smart people working for him, and he's great at marketing, but his level of maturity isn't "mature" (yet). LTT built up a TrueNAS system in their typical fajigger-it fashion and, like often happens with them, it broke (and cost LTT about $15,000 CAD to recover). I don't know how often you've lost data, but if you've been through it enough times you eventually want to take reasonable and sensible precautions up front to avoid losing said data. If I remember that's something like Linus 4th time of experiencing data loss and the event that pushed him into getting a tape backup.

Mind you, we're not telling you that you can't build that system, we're simply saying TrueNAS/ZFS might not be the best thing to put on it. You might want to go with something less resilient and more flexible than TrueNAS given what you're wanting to do, which of course is your choice. The best of both worlds might be to keep your data on a dedicated TrueNAS box and run a Plex server on another box. Yes, that would use more power. However, when you understand how often errors actually occur in a system you might just want to keep your data storage system running lean and clean, and everything else that could go "a little" sideways on a different machine.

Want to step away from Synology and like what I read so far about Truenas Scale. Unraid was also an option but as I know a few people who have Truenas Scale, it's easier to get some feedback.

The motherboard was unfortunately pretty much the only thing that was available to buy if you go for built-in, mini-itx, gpu, ECC support, 4x sata & 2 nvme. The popular Asus alternative is out of stock all over the place. There is the W680 chipset MB's for intel CPU's to support ECC, but are very expensive and seemed overkill for what I'm aiming to do.

I did read stuff about "supermicro" mb's as good solution (including a mini-itx one), but can't seem to buy those anywhere. So if there is a better candidate at some point in time with the combination of above requirements, I'm very much interested.

 

[FrostyCat]

I should begin by telling that all the advice you'll get here is mostly good and well intentioned, but, and there's a big but it can come from a perspective that is not your own.

My advice would be to build your system based on how much you value your data.

If you don't care about data loss that much, or the data is replaceable, server grade builds are going to be overkill, expensive, power hungry and loud. It's only worth going for such a build if your data is irreplaceable and only in the context also having multiple backups stored in different places so where you eventually have a crash and lose everything you can recover.

So, if on a scale from 1 to 10, from least important to most important, your data is below a 6 or 7, build on whatever and just backup the most important stuff, i.e. your documents and photos you can't replace. In this case it doesn't matter if your motherboard is a gaming one, or you don't have ECC RAM, your NIC is a Realtek, etc.

Anything above that and you should start thinking seriously around having solid hardware, spares and redundancy with proper backup plans in place.

 

[wlknsn]

The reason i'm asking the question in the first place is to get other opinions, so no worries there. I take from most feedback that other than the choice of motherboard, all is pretty good.

Not a lot of feedback on the power consumption unfortunately which I was hoping to get more info on certain strategies to keep that to a minimum.

 

[FrostyCat]

Generally server grade is not going to be very energy efficient. My setup goes for about 40W idle which is pretty good for how powerful the system is, you can go lower but you’ll lose some performance.

To give you an idea, my system is based on Ryzen Pro 4750G, 64GB of ECC and 6 x 6TB Ironwolves and 2 SSDs, one for boot, one for apps.

Edit: power consumption is very important for me as energy is expensive here in Australia. I do have solar panels so during the day running my network is pretty much free but not a home battery yet.

 

[NickF]

I mean the build you are planning isn't really minimizing power.

An Atom is what you want if that;'s what you are trying to do. The 5700G alone draws up to 88 Watts under load, whereas the MiniXL+ has a load at 101W for the whole system

 

[FrostyCat]

That’s peak power itsnot relevant as no systems runs at 100% all the time.

I actually monitor my whole “rack” and track and graph power usage in my Home Assistant.

 

[jgreco]

server grade builds are going to be overkill, expensive, power hungry and loud. It's only worth going for such a build if your data is irreplaceable

This is simply false. "Loud" is usually if you've bought datacenter rackmounts, because the fans required to generate a high pressure differential to force air around the drives are naturally doing a lot of work and generating a lot of noise. You can certainly build a NAS that's quiet. Expensive is a load of BS too; the Xeon server boards parallel to a similarly-featured desktop board are generally only 10-15% more, and you gain value by spending that because you don't have to replace the Realtek ethernet with an Intel, and the components are rated for longer life, so it actually works out cheaper in the long run. Power hungry is also silly; the Xeons use about the same amount of power as their consumer equivalents. And the real kicker is that an older generation Xeon is perfectly fine for NAS but prices for used server gear are generally a lot cheaper because the server market is perennially flooded with used gear.

 

[FrostyCat]

I’ll grant you the noise one, I agree there may be ways to have a quiet server grade build.

For everything else, I’d appreciate some numbers as otherwise anything you said is just words. IMO consumer grade stuff is almost as good nowadays as what beard stroking people call server grade but it lacks redundancy and expandability.

That won’t matter much for a home NAS, but going from 40W to 80W will double the energy cost. For me and many others this matters a lot.

 

[jgreco]

For everything else, I’d appreciate some numbers as otherwise anything you said is just words. IMO consumer grade stuff is almost as good nowadays as what beard stroking people call server grade but it lacks redundancy and expandability.

I'm not even clear on what you're saying here. What lacks redundancy and expandability? I can certainly find you both server and desktop grade parts that are either highly expandable or virtually unexpandable. You seem to have some quaint ideas about server gear; is your only experience with server grade gear some unpleasant experiences with Dell or HP rackmount kit you encountered at work or something like that? I just want to understand the context here.

There's been plenty of watts discussion over the years, and if you look back to the Sandy and Ivy days, you'll find some rather specific head-to-head comparisons by me and others. The thing is you have to go apples to apples. For example, you can compare an E3-1230 Xeon Sandy Bridge CPU to whatever the equivalent Sandy Bridge desktop grade part is; they'll be very similar in terms of clock and power consumption, it's mostly that one will have ECC and one won't. You can't compare that same desktop part to an E3-1650 because the 1650 will have two more cores, more memory capability, and more PCIe capability. That'd be an apples-to-oranges comparison, and hell yes, the 1650 will burn, oh, I dunno, 30-50 watts more depending on the features present on the mainboard (which are responsible for some of the extra watt burn).

The "beard stroking server grade" comment is unnecessary; there's a clear delineation just as there is between desktop and gaming boards. Server grade boards are designed to be put into a chassis that will go and rot in a data center somewhere for possibly a decade or more. This typically means they have high quality long-life components, lack frippery such as built-in audio/Wifi/BT/HDMI, and usually offer features like VGA, IPMI, server-grade Intel ethernets, PCIe x8 slots rather than x1/x4, etc. I talk about some of this in my original hardware recommendations guide. Server-grade boards are a very real thing and are targeted at specific types of uses just as you might target a $50 ITX desktop board at a different type of user than a $200 ATX desktop.

 

[NickF]

I pay about $.25/kWh. Let use your numbers

700*.25 is about 175 dollars a year or is about $15 dollars a month.
350*.25 is about 88 dollars a year or about $7.5 a month

Lets imagine for a second that you spent $3,000 on a server which was highly energy efficient (40 Wtts) with all of the latest goodness, or you spent $1500 on a server which was a couple of years old and was decommed from a server farm. In fact, that $1500 server has better specs, but it draws double the power.

These numbers are round to make this easy to demonstrate, but even 72 months in you haven't recouped the investment in newer lower power hard. And 6 years is a pretty good run for a server deployment.
ware

 

[FrostyCat]

I’ll just make a couple of points and then I’ll graciously back out of here as this discussion doesn’t seem to become any more productive and I believe OP got a lot of useful information out of this thread so far.

AMD exists and their stuff is amazing nowadays, I personally don’t even look at Intel anymore. I believe everyone should be aware of this as AMD is a real option and brings awesome stuff to the table.

I personally like to build from scratch instead of buying second had decommissioned server machines as I like the flexibility in choosing a form factor, e.g ITX.

Noise, power and cost are all important, I refuse to run an 80W systems when I can run a 40W system, build cost being equal. Only my data will determine if the system is server or consumer grade and how much redundancy I should aim for, and if critical, the former will not matter as much anymore.

In my particular case, consumer grade is more than enough and I’m happy to stay away from the large, often noisy and power hungry stuff. My NAS sits in an IKEA cubby with room to spare.

Keep up the good work, I’m out.

 

[NickF]

I’ll just make a couple of points and then I’ll graciously back out of here as this discussion doesn’t seem to become any more productive and I believe OP got a lot of useful information out of this thread so far.

AMD exists and their stuff is amazing nowadays, I personally don’t even look at Intel anymore. I believe everyone should be aware of this as AMD is a real option and brings awesome stuff to the table.

I personally like to build from scratch instead of buying second had decommissioned server machines as I like the flexibility in choosing a form factor, e.g ITX.

Noise, power and cost are all important, I refuse to run an 80W systems when I can run a 40W system, build cost being equal. Only my data will determine if the system is server or consumer grade and how much redundancy I should aim for, and if critical, the former will not matter as much anymore.

In my particular case, consumer grade is more than enough and I’m happy to stay away from the large, often noisy and power hungry stuff. My NAS sits in an IKEA cubby with room to spare.

Keep up the good work, I’m out.

I don’t know where any of this is coming from frankly. No one said anything about AMD. Hell, I have two AMD EPYC systems.

The part about your data, on your reliability, that’s a sample size of 1. Good luck gathering meaningful data.

Ignoring TCO is rather silly, putting your head in the sand isn’t really an answer. Servers depreciate in value faster than cars. A 2 or even 5 year old enterprise server is going to be cheaper than a RYO server with current consumer parts. And it will be more reliable.