Extrem power efficient FreeNAS

[silencium]

Hi everyone,

here in Germany the bills for energy consumption are piling up and are getting bigger and bigger.
I am a system administrator and need to educate myself from time to time at home. For that reason I have 3 servers at the moment.

1) 1231v3, 32GB DDR3, 1TB SSD, Supermicro X10SLM-+LN4F, SeaSonic Platinum PSU 400W fanless ~45W
2) 1231v3, 32GB DDR3, 1TB SSD, Supermicro X10SL7-F, 1x 10TB WD Red SeaSonic Platinum PSU 400W fanless ~60W
The VM for FreeNAS has 16GB RAM, 6 Cores :D and of course the LSI-Controller for SMART!
3) 2620v4, 72GB DDR4, 1.8TB SAS, ProLiant Gen9 ~85W

Adding two UniFi 24 and an 8 Port Switch + FritzBox! 6590 and you are eating up round about 250W. 24/7 this adds up being roughly 650€ a year! Welcome to Germany :) I think at this point I will have explained why I want to reconstruct my Homelab. CPU is like 95% of the time at around 5% idle?! I barely use CPU-Power but I need the RAM. So in total I need around 128GB Ram. I use VMWare right now and that won't change. Server no. 2 is the one hosting FreeNAS (on 2 SSDs, redundancy).
I do not need more than the one 10TB drive, for now. Maybe I will upgrade to another 10TB drive in the future.

So what I need are 3 systems, which give me the freedom of using basically more RAM than I can right now and that use less power. I hope to get them all down to around 25 - 30W. Do you guys see any possibility there?

The FreeNAS Server is only for me and my wife. I backup the vCenter + 2 Clients onto the 10TB + I have some SMBs. The Server doesn't need to encode or decode stuff. Its basically a fileserver. No Jailes (yet).
What I really not want to miss is something like iLO or IPMI. 64GB RAM would be future proof. 10GBASE-T (is that downwards compatible?) isn't something I need or am looking for -> power consumption. Until now I was looking for a Supermicro X11SSL-CF, because I did not want a dedicated server for FreeNAS. But I think it's time to think different and make it a dedicated server.

At the moment I see an Atom or Xeon D 15xx being the right contestant. The system should also support up to 32GB of Ram.


Maybe you guys can help me with my situation. Is there a solution that would work with ECC REGISTERED RAM? The UDIMM is just so freaking expensive -.-


Cheers
Silencium


PS: Yes, I did read the Hardware Guide and yes I did research the web for solutions. But there hasn't been one that did seem to be right for me.

 

[Arwen]

I've heard of people using Raspberry PIs as NASes, with USB attached disks. They have claimed they are quiet, fast enough, so little energy they can be battery powered, and of course, cheap.

In some ways FreeNAS is not suitable for the bottom of the energy efficiency. (Nor the cheap end.) ZFS works better with a bit of RAM, and reasonable I/O to storage. Other file systems and RAID systems, (Linux BTRFS or EXT4/MD-RAID), work okay with lower end requirements.

All that said, I can't advise on your questions. Just pointing out a few things.

 

[silencium]

@Arwen, thanks for replying to my post.
You might be right, but I only want to lower the consumption by 100 - 150W. I think that it is possible and the Raspberry Pi is way under my expectations!

Right now I am thinking of switching the 6 SAS HDDs in the 3) Server and bringing in just 4 SSDs + the 10TB HDD. For the 10TB I need to get a cage though and some other parts. Also I do need to figure out if that system would be suitable for my scenario. Especially in terms of SATA passthrough. So yeah, that would mean to not switch to a bare metal FreeNAS solution.

 

[Constantin]

You could do a few things to bring down disk power needs by allowing them to spin down. Many NAS' allow this, though with a latency hit for spin-up. Plus, your power supply likely won't be a fan either. Here is my my suggestion:

Build a NAS around a single 10TB drive. Whether it's QNAP, ReadyNAS, etc. they all make capable little boxes that can handle a single drive or two and present them to the user in a usable format. The key thing with single drives hosting all your content is to either have at least 2 backups (then store 1 off-site and rotate) or off-site cloud storage. A good little NAS will make the process relatively painless and reduce your on-site power demands.

For example, I use a small drive to host my music content for a Sonos at home since I don't trust Sonos with access to my main NAS. The hard drive host is a Apple Extreme base station, 6th gen, and it works just fine for that application (burner disk).

FreeNAS really isn't set up for power-savings. The emphasis is on preventing bit-rot (via scrubbing), high performance (no spin-down by default), and a lot of parity work, which in turn necessitates multiple hard drives to make it worthwhile. Low-power incarnations of FreeNAS servers are possible (use 2.5" spinning platters or SSDs), but there are tradeoffs (cost, per TB of data).

If you still want to go the FreeNAS low-power route (i.e. bit rot is important and you don't trust the ongoing btrfs struggles to get that covered), I suggest you look at using 2.5" drives with the AsRock C2750 or C2550 board. Power demands are low, performance is decent, esp. in home use.

 

[rvassar]

While temporarily out in California a few years ago (2013 - 2014), I was renting a house that got the post-Enron 5 tier electrical rate (since discontinued) that topped out at a punishing 55 cents per kWh (0.48 Euro at today's exchange rate). I had picked up a Raspberry Pi before moving, and pressed it into service for some network services. I then picked up a Bananna Pi, a knock off of the Raspberry Pi with additional features. It had 1GbE networking, a dual core CPU (RPi's have caught up here), and a single SATA connector. There was lots of manual configuration, and it wasn't anything to write home about, but it got the bulk storage & MySQL DB jobs done for less than 0.4 kWh per day. When I needed something more, I could turn on an actual PC.

 

[Constantin]

We pay $0.25/kWh in New England, of which about 75% are Transmission and Distribution. I know of another person who used a Pi to run a low-power NAS, for similar reasons to mine. I've been thinking of using one as a IDS but that presumes far more free time to get it set up than I have.

The Enron fiasco in CA was completely avoidable but that's a discussion for another day. Bottom line, don't expect the market to play nice if you give it all the wrong incentives.

 

[danb35]

Low-power NAS? An Odroid HC2 running OMV.

 

[Constantin]

That's a great option in case you're not already running a NAS-capable piece of hardware on your network like a AirPort Extreme.

The beauty of the AirPort Extreme is that you can bus-power 2.5" drives, it allows rudimentary NAS functions like allowable users, etc. Since it all runs on a RISC processor, it's also reasonably energy efficient. Most Mac users will figure out how to use it and the possibility of using it as a TimeMachine target makes it even more attractive. Too bad Apple decided to withdraw from the WiFi market, none of their gear has ever been pwned from an OS POV, AFAIK (KRAK and other issues associated with the WiFi alliance are out of the vendors hands).

 

[rvassar]

Low-power NAS? An Odroid HC2 running OMV.

That's really not much more than the Bananna Pi I was running back in '14. The unaddressed problems being redundancy, no RAID at all, and I'll guess not enough RAM for a proper filesystem like ZFS. With a retail 2-drive QNAP or Synology box, you at least get mirroring, rudimentary snapshots, and cloud backup. I have a small QNAP TS-228 that was unsuited to my needs, but quite efficient. If it had a second network port I might have kept it in service. Since the OP only has the one 10Tb drive, he may find this solution acceptable. If they truly need the capabilities of FreeNAS, then they need to take a fully minimalist approach on the newest generation of hardware they can afford. Buy the low-power CPU, minimal RAM, rig the chassis for as passive a cooling setup as possible.

I will offer a cautionary note on using your WiFi router as a storage device. Once you get to Gigabit fiber speeds for your ISP, your router may be so busy hauling the mail, it starts dropping DHCP renewals!

 

[Constantin]

I will offer a cautionary note on using your WiFi router as a storage device. Once you get to Gigabit fiber speeds for your ISP, your router may be so busy hauling the mail, it starts dropping DHCP renewals!

Agreed. But those kinds of users also typically don't care too much about energy consumption because the cost of running that kind of service costs $$$ where I live and the resultant energy bill will be a rounding error.

That said, all my AP's but the one hosting the burner drive for Sonos are dedicated to being a AP - no routing, etc. An edgerouter takes care of the firewall, DHCP, and so on. I am unlikely to ever saturate that, nor are my kids - they have limited bandwidth per the VLAN I put them on. :)

 

[kdragon75]

You might be right, but I only want to lower the consumption by 100 - 150W

That's more than half!

You could do a few things to bring down disk power needs by allowing them to spin down.

Not with a VMware datastore on it.
If we toss FreeNAS and go with a pure Linux (Debian) you could look at a Pine RockPro64 with the SATA card and NAS case. You could use GEOM or what ever on Linux to do mirroring. I would even be willing to try ZFS with some (lots) manual tuning.
In fact Ive been thinking about doing this myself as a backup target but I'm about to start an AiO FreeNAS/ESXi build that should be under 180 watts. I'll also be able to cut out my big switch saving me another 60-80 watts. My goal is to be under 300 all in. At least until I can upgrade to some newer hardware.

 

[danb35]

The unaddressed problems being redundancy, no RAID at all, and I'll guess not enough RAM for a proper filesystem like ZFS.

Minimal RAM and no redundancy, true, but the HC2 can be made to do ZFS. Still trying to work out the best way to make that happen, though...

 

[otpi]

I have a 5 kW space heater in my basement. I just assume any power spent by my servers is electric-heating with useful calculations before entering the room.

Do you really need the two switches (24p + 8p)? I thought about adding a bigger switch, but because of power consumption (and a reality check) ended up using dumb 8p switches, it's only 4.8 W.

 

[silencium]

Okay folks thanks for all your input. I did not want to disturb the input and stayed quiet for a little while - on purpose :)
@otpi yes, I need two switches. Its because of the floor plan and the fact that I always prefer lan! lan >>> wlan.

I've decided to sell the HP Server. I will switch the "little" 1) Server with the following.
Supermicro X10SDV-TLN4F-O, Xeon-D 1541 and 128 GB RAM. This will give me peace. I read a lot about the Xeon-D 15xx and 21xx. Performance / Power it seems to be the best deal for me. I payed 1000€ for that combination, which is okay. Sold the HP for 1080€. One can complain that the HP was worth way more but that's another story and does not fit into this discussion.

The X10SDV will give me plenty options. I'll be able to use an LSI 92xx and passthrough it as HBA/JBOD, get all the SMART I need.