Hardware recommendations for low-power home server

[treant]

Hi all,
new user here!
I have an Unraid server at home which I have been using to store all my data and provide services for a few years.
It's built around an ancient Gigabyte J3455 mini-itx low-cost motherboard, with a few 3TB drives attached, and an SSD for write caching.
So far it's been serving me very well, but it's showing its age, so I want to upgrade.
The new server should not do anything more than the current one does (fairly well), it should just do it better.
Plus, I'd love to get my hands dirty with ZFS (which I have never tried), so... I want to try out TrueNAS Scale.

I'm very comfortable with Linux machines and with building hardware, but the hardware choice is proving though (yes, of course I read the hardware guide!). I would love any help and suggestion!

The usecases for my server would be:

• media streaming, mostly in-house, with limited transcoding
• nextcloud and file sharing
• torrenting
• other misc services such as teamspeak, authelia, vaultwarden, traccar, home automation and the like. Nothing resource-heavy.
• and of course, storing data safely!

Some key aspects for this build are:

• Form factor should be mini-ITX or microATX, due to limitations in my enclosure (it's actually a modified Ikea cabinet drawer).
• low-power components (power is very expensive where I live). This means:
  • The target power consumption should be <25W at idle. Idle power is the most important metric in this regard since the system is going to stay idle for most of the time.
  • Peak power consumption should be <150W, so I can use a picoPSU.
  • I should probably use as few components as possible.
  • Not sure yet whether it's best to go AMD (something like a 5650G) or Intel (Alder Lake?). I will need some integrated GPU due to media transcoding and encoding needs. AMD seems more competitive in terms of price-to-performance (especially with ECC support), Intel is probably going to use less power at idle.
  • Hard drives should spin down as much as possible. In Unraid, this is accomplished with an SSD write cache and a daily mover script, but from my understanding this is not available in ZFS, right? Thus, I'll probably need an SSD pool for stuff like torrenting and nextcloud, while limiting HD access to media usage and backups (mostly 1-2h a day). Is this correct? I could use a SLOG but from what I read it would probably be overkill for my usecase, right?
• Total storage should be ~12TB. Not sure if I'm going for a 2x12TB mirror or a 3x6TB or 3x8TB raid-Z1. Might decide depending on which offers I find.
• I do not need double redundancy since everything will be remotely backed up nightly.
• I currently have no use for >1Gbps LAN, but that might change in the future. Future-proofness would be awesome, especially since of the limited expandibility of the chosen form factor.
• The conventional wisdom is that for such a small server, I don't need L2ARC. Now that it is persistent (with ZFS 2.0), though... is it a good choice? Thinking of a 1TB NVMe SSD to accelerate frequent access to stored media, for example.
• My last server has been going strong (well, it's a Celeron J3455, not exactly strong...) for 5 years. I want this investment to last at least as long.
• Motherboard choice is very hard, especially due to the form factor and low power requirements. This very useful video links to this german forum post and this google sheet, with some data. A good place to start, but still I would love any suggestion in this regard! The motherboard market is a mess and it's hard to find motherboards that fit my usecase perfectly:
  • low power
  • reasonably priced (in relation to the meager amount of storage I'm putting in)
  • no need for IPMI (and it uses too much power, AFAIK)
  • at least 4 SATA
  • Some M.2, to use for boot, fast pool, SLOG, metadata device or L2ARC if I need them. PCI-ex expandibility to add them later is fine.
  • ECC support
  • doesn't cost as much as the rest of the system combined! For example, the Asrock Rack X570D4U-2L2T/BCM would seem a good choice, but it would be very overbudget (>500€) for what I need.
• If I can, I'm going to use ECC memory, probably DDR4, probably 32GB. The problem is finding a motherboard that supports ECC and all the features I want!

As a complete ZFS and TrueNAS noob, I'd love any pointer in the right direction.

1. Are my goals completely unattainable?
2. Do you have any motherboard suggestion? I can't seem to find a motherboard-CPU combo that is relatively recent, has ECC support and does not cost an arm and a leg.
3. Do I need an SSD pool to act as a write cache so that I can reduce the time the drives are spinning (especially for torrent downloads)? And/or, do you have any pointer to how to mimic Unraid's mover behaviour, which works well in my usecase?
4. Would a large, persistent L2ARC help in reducing the need to spinup the drives?
5. Do I need to shop for used goods? Any good pointer for where to do that in the EU? Is eBay my best bet? Any particular CPU and/or motherboard model that usually pops up as particularly affordable?
6. Anything else that comes to mind?

Thank in advance for any suggestion!

 

[artlessknave]

one way to look at TrueNAS is it's like driving a stick shift. great and powerful when done right, expensive and dangerous when done wrong.

you haven't posted your hardware, which is a forum rule, but here are a few things I see.

no need for IPMI (and it uses too much power, AFAIK)

if 10w is too much power, TrueNAS, for you, might not be.

Hard drives should spin down as much as possible.

truenas, and ZFS, expect the drives to be available all the time. not only does spinning down your drives wear them out and cost a ton of power to spin them up, but ZFS is designed for high availability 24/7 operations and runs things like scrubs regularly to ensure data integrity.

I don't need L2ARC.

if you dont already KNOW you need L2ARC, with usage data and ARC hit rates to justify it, it's very unlikely you need L2ARC

use a SLOG

no. SLOG is not a cache and, much like L2ARC, if you don't already know you need it, you most likely don't. SLOG is for writes that absolutely must be written to non volatile storage (think VMs and databases), instead of RAM, and thus only applies to sync writes, async writes will utterly ignore it.

Do I need an SSD pool to act as a write cache so that I can reduce the time the drives are spinning

no such thing. in ZFS RAM is your closest thing to read/write cache. there is no such thing as keeping the drive not spinning, anything written to the spinning disk pool needs to be written to the spinning disks, the only thing that changes if is its written immediately to ZIL or SLOG then to the disks for synchronous writes or sits in RAM for the few seconds needed to fill a transaction group before being flushed to disk. not even a fusion pool creates a "write cache".

(it's actually a modified Ikea cabinet drawer).

I shudder to imagine how terrible the airflow in that might be....

Would a large, persistent L2ARC help in reducing the need to spinup the drives?

nope. it will simply take more RAM to hold the L2ARC tables. RAM that could be ARC, which is always faster.


from what I am seeing here, you should probably stick with unRAID.

 

[Arwen]

To clarify some of what @artlessknave said, TrueNAS & ZFS were not designed for the lowest end of power consumption.

In theory, you could design a TrueNAS server for very low power during normal usage. However, the twice a month recommended ZFS scrubs would kill a low end CPU, and certainly use up more than expected / desired power for the duration of the scrub. Not to mention the regularly scheduled SMART short and long tests. All of which need the disks spinning.

You might look at a mixed configuration. One miniature server, (like a Raspberry Pi), for the applications, another for un-protected media storage and last a NAS that holds protected backups of your media and applications. The NAS can be powered off except during backups, though again TrueNAS might not be your best option in this scenario.

 

[ChrisRJ]

You might look at a mixed configuration. One miniature server, (like a Raspberry Pi), for the applications, another for un-protected media storage and last a NAS that holds protected backups of your media and applications. The NAS can be powered off except during backups, though again TrueNAS might not be your best option in this scenario.

As an additional idea my configuration consists of 2 machines:

• The main NAS as per my signature
• A small machine with low power consumption that runs ESXi and all the applications I want

For this case, simply putting some cheap SSDs into the small machine and use it as a first storage tier might be an option.

In general, though, I agree with the others in that TrueNAS might not be the best option in this case.

 

[treant]

one way to look at TrueNAS is it's like driving a stick shift. great and powerful when done right, expensive and dangerous when done wrong.

you haven't posted your hardware, which is a forum rule, but here are a few things I see.

if 10w is too much power, TrueNAS, for you, might not be.

truenas, and ZFS, expect the drives to be available all the time. not only does spinning down your drives wear them out and cost a ton of power to spin them up, but ZFS is designed for high availability 24/7 operations and runs things like scrubs regularly to ensure data integrity.

Lucky me I come from a place where driving a stick shift is the norm (and much cheaper than an auto)!

Jokes aside. I have not posted my hardware because, well, I don't have any hardware. The point is that I am planning to buy some!

if 10w is too much power, TrueNAS, for you, might not be.

Any continuous 1W in power consumption is about 5€/year (the same in $, at the current exchange rate). Not much of course, but it adds up!
So no, 10W more is not too much, but in my experience it's easy to add power hungry components if you don't plan to pay a lot of attention to it from the start.

truenas, and ZFS, expect the drives to be available all the time. not only does spinning down your drives wear them out and cost a ton of power to spin them up, but ZFS is designed for high availability 24/7 operations and runs things like scrubs regularly to ensure data integrity.

Then this might be a game stopper for me. However I somewhat disagree about the wear and power considerations: spinning up 2-3 times a day is perfectly acceptable for today's hard drives, and in terms of total power consumption they only take a few W (usually ~10W) for a second or two when spinning up (~0.01Wh). Certainly not enough to compensate for 8 hours of running idle at 2-3W (~20Wh).
I already scrub my BTRFS Unraid pool weekly, so I know already what kind of load that induces.

Good points re ARC, SLOG, etc. My expertise on this is zero so I cherish any help!

I shudder to imagine how terrible the airflow in that might be....

You'd be positively surprised ;)
I've been building and modifying PCs for 20 years and I know a thing or two about physics and aerodynamics. This one has a 20cm noctua fan blowing filtered air directly on top of the motherboard; air then passes through the HDD stack before exiting through slots in the front of the cabinet.
The fan has very low static pressure, and the holes in the cabinet are designed with that in mind, in order to make use of the airflow without impeding it. It's nowhere near a server cabinet, with all components stashed close to one another, which needs a boatload of pressure from high-rpm fans.
It works extremely well (and is totally silent), with the low-end hardware that's currently in it, and I'm very confident that few modifications could easily improve that even more.
The previous iteration of this machine was completely fanless and only used natural convection, inside a custom-built case. That one struggled a bit (hard drives reaching 40°C) when ambient temp exceeded 25°C, so I moved it into the cabinet next to me.

from what I am seeing here, you should probably stick with unRAID.

That is probably my best course of action to be completely fair.
But still, I'd love to try out ZFS!
We use it at work, but it's not in my department so I don't have the opportunity to play with it much.

 

[treant]

To clarify some of what @artlessknave said, TrueNAS & ZFS were not designed for the lowest end of power consumption.

In theory, you could design a TrueNAS server for very low power during normal usage. However, the twice a month recommended ZFS scrubs would kill a low end CPU, and certainly use up more than expected / desired power for the duration of the scrub. Not to mention the regularly scheduled SMART short and long tests. All of which need the disks spinning.

You might look at a mixed configuration. One miniature server, (like a Raspberry Pi), for the applications, another for un-protected media storage and last a NAS that holds protected backups of your media and applications. The NAS can be powered off except during backups, though again TrueNAS might not be your best option in this scenario.

I already do scheduled scrubs and smart tests on my (BTRFS-based Unraid). Either every one or two weeks.
They require about 4h at full speed (which is fine, they run nightly), and they are drive-limited, not CPU-limited, even with my measly J3455.
Is this not true for ZFS? After all all it does it verify the checksum and parity (if raid-Z) of all files, right? The same as BTRFS, in this regard, as far as I understand it.
Overall a few h/week of full-speed spinning still has a limited power consumption impact. This is why I want to target low idle power; the system is idle 95% of the time.

"Hardware-tiering" the services would be a good idea, in fact.
What I don't like about it is that I loved the elegance of having a single machine being responsible of most of the stuff I need (basically everything except the router, which is a separate machine, and an Nvidia Shield for media consumption).

As an additional idea my configuration consists of 2 machines:

• The main NAS as per my signature
• A small machine with low power consumption that runs ESXi and all the applications I want

For this case, simply putting some cheap SSDs into the small machine and use it as a first storage tier might be an option.

In general, though, I agree with the others in that TrueNAS might not be the best option in this case.

Another option could have software data tiering in TrueNAS. AFAIK this is not available, but maybe someone has tried using for example autotier from 45drives?


Anyhow thank you all for all your precious comments!

 

[danb35]

no need for IPMI (and it uses too much power, AFAIK)

IPMI isn't a significant power draw.

For low power draw, the HPE Microservers are pretty good. My Gen10+ idles at about 35W (and that's without spinning down drives, and with a 10G NIC added, as well as the add-on card for iLO (HPE's version of IPMI)); I recall my parents' Gen8 running at about that power consumption as well--in both cases with Xeon CPUs. Or, if you really want to roll it yourself, the Avoton-based boards are pretty good in that regard; both SuperMicro and ASRock had them. Your power budget might be just a little low for TrueNAS, though.

As another alternative, consider:

Linksammlung | Alles ohne Windows

sysops.tv Projekt Beachten Sie unser neues Prospekt, in dem wir versuchen alle Leistungen aufzuzeig...

sysops.tv

 

[Arwen]

Yes, BTRFS scrubs would be similar power usage as ZFS scrubs. Though if I understand some of the issues with BTRFS, it's not doing RAID-5 or -6. So ZFS scrubbing might be reading in more disk blocks, and checking parity. While the UnRAID with BTRFS is not checking the parity, as that is handled by UnRAID's proprietary protection scheme. But, as they say in yank land, power usage would be "in the ball park".


If your work servers that are using ZFS run Solaris as the OS, just note that around 2011 Solaris ZFS & OpenZFS diverged. Similar, but no longer the same. When Oracle bought Sun Microsystems, they stopped open source development of Solaris. Pool compatibility for any feature since then, on either side, makes the pools incompatible.

 

[treant]

IPMI isn't a significant power draw.

For low power draw, the HPE Microservers are pretty good. My Gen10+ idles at about 35W (and that's without spinning down drives, and with a 10G NIC added, as well as the add-on card for iLO (HPE's version of IPMI)); I recall my parents' Gen8 running at about that power consumption as well--in both cases with Xeon CPUs. Or, if you really want to roll it yourself, the Avoton-based boards are pretty good in that regard; both SuperMicro and ASRock had them. Your power budget might be just a little low for TrueNAS, though.

As another alternative, consider:

Linksammlung | Alles ohne Windows

sysops.tv Projekt Beachten Sie unser neues Prospekt, in dem wir versuchen alle Leistungen aufzuzeig...

sysops.tv

Thank you, I'll take a look at the Avoton boards!

Yes, BTRFS scrubs would be similar power usage as ZFS scrubs. Though if I understand some of the issues with BTRFS, it's not doing RAID-5 or -6. So ZFS scrubbing might be reading in more disk blocks, and checking parity. While the UnRAID with BTRFS is not checking the parity, as that is handled by UnRAID's proprietary protection scheme. But, as they say in yank land, power usage would be "in the ball park".


If your work servers that are using ZFS run Solaris as the OS, just note that around 2011 Solaris ZFS & OpenZFS diverged. Similar, but no longer the same. When Oracle bought Sun Microsystems, they stopped open source development of Solaris. Pool compatibility for any feature since then, on either side, makes the pools incompatible.

Yeah we use Debian so OpenZFS. But I don't work on those servers directly so have very little experience with them.