Help needed for HW Selection for Low Power System [Attention: A longer Post]

[thomas-hn]

Hi,

first a huge sorry for the long text in this post.
Currently I'm trying to find the hardware for my first TrueNAS CORE build, but after reading almost all guides here about TrueNAS as well as searching for hardware and information, I am still struggling with selecting the right parts.
The most important goal for me is data integrity, followed by the power consumption, and performance.

My requirements are:

• The idle power consumption of the system should be at a maximum of around 130 watts (my current system which is Intel Core i5-4690S, 32 GB RAM, Adaptec RAID Controller, and 8x 6TB HDDs consumes this amount of energy and the energy costs in my country are gigantic).
• The system shall serve:
  • as NAS for personal data as well as multimedia data
  • some Jails:
    • Plex
    • Syncthing
    • Nextcloud
    • CUPS
    • UniFi controller
    • FreePBX
    • Gitlab
  • 2 VMs running Win10
  • 1 VM running Docker (webserver for private use, portainer, Zoneminder)
• Currently around 8 TB for personal data
• Currently around 17 TB for multimedia data
• I prefer server-grade hardware (if possible)
• 10 GBit/s Ethernet (SFP+ preferred, otherwise 10GBASE-T)
• IPMI
• Supermicro mainboard preferred
• CPU
  • I am planning a CPU with virtualization support for running the VMs.
  • Furthermore, I was looking for a CPU with iGPU for transcoding support in Plex.
• Chassis:
  • Preferred 19" rack housing or a chassis which can be used as tower and 19" rack
  • Hotswap
  • Disk Failure LEDs
  • The chassis shall be able to keep the disks at around 30°C.

As you can see above I want to virtualize some services. For this my preferred solution would be Proxmox. However, running TrueNAS on top of Proxmox is not recommended and also I don't want to run TrueNAS on top of ESXi, because this makes things much more complicated and increases the risk for data corruption.
This said, I plan to go with the TrueNAS virtualization options (Jails + bhyve). However, in case we get a system that fulfills the above requirements and consumes in idle state (with spinning disks) much less than the above mentioned 130 watts, I would go with a second machine for Proxmox which, maybe, accesses all VM data over the network from the TrueNAS machine.

Regarding to my thread "What's your Idle Power Consumption?" some users here have systems like

• Xeon E3-1230, 16GB ECC RAM, 6x HDD (spinning) running at 65W idle.
• Xeon E5 1620, 64GB ECC RAM, 8x HDD (spinning) running at 85W idle and 110W on medium load.

With these power consumption in mind I want to try to build a similar system and then

1. check the idle and medium load power consumptions
2. if these power consumptions are low enough, I would let run the system as NAS only (without any virtualization) and trying to add a 2nd server for Proxmox which uses the storage on the NAS. However, see my list of questions at the end of this post, please.

Having this said, the requirement for the CPU is that it should be powerful enough for the virtualization, but also has a low idle power consumption if the system has only a small load.

What do you think about the following setup?

• Mainboard
  • Supermicro X11SSH-CTF
    • Is this board fully compatible to TrueNAS/FreeBSD? It seems that some years ago there were some incompatibilities.
• CPU
  • Intel Xeon E3-1275 v6
  • I prefer a CPU which can idle at a very low wattage, but boost up if required.
• CPU-Cooler
  • ???
• RAM
  • 64 GB ECC
  • Exact model ???
• NIC
  • The X11SSH-CTF has onboard 10GBASE-T which seem to consume a lot of energy while being not very flexible.
  • Do you think it is better to take a board with 1000BASE-T instead (and disable it) and adding a separate SFP+ network interface card?
  • I would prefer 10 GBit/s, because 1 GBit/s allowes at a maximum 125 MB/s which is a limitating factor when using two vdevs/pools.
• HBA
  • The X11SSH-CTF has an onboard LSI 3008. Would you go with this or using simple onboard SATA connectors or adding one of the preferred HBA separately?
• Power Supply
  • ???
• Chassis
  • My current favorites are:
    • SuperChassis 743AC-1200B-SQ (Height 45.2 cm; Width 17.8 cm; Depth 64.7 cm; can be used as tower or 19" rack)
    • SuperChassis 745BAC-R1K28B2 (Height 45.2 cm; Width 17.8 cm; Depth 64.7 cm; can be used as tower or 19" rack)
    • SuperChassis 825TQ-R720LPB (Height 8.9 cm; Width 43.7 cm; Depth 64.7 cm; rack only)
    • Fractal Design Define R5 (much cheaper than the SuperChassis, but no Hotswap and no exchangeable drive bays)
  • My main issue with the chassis is that I don't have a 19" rack at the moment. However, it is planned to have one the next years, so I am still thinking about which way to go.
  • Any other recommendations or hints for the chassis?
• HDDs
  • 4x 10TB HDD SATA as RAIDZ2 = 16 TB usable (considering max. 80% fill level)
    • From the available 16TB, around 8 TB will directly be filled with existing data.
  • 4x 16TB HDD SATA as RAIDZ2 = 25 TB usable (considering max. 80% fill level)
    • From the available 25TB, around 17 TB will directly be filled with existing data.
  • The plan is to use two vdevs to allow single storage space enlargement in the future as well as better redundancy compared to using 8 drives in a single vdev.
  • The exact HDD models (CMR) have still to be defined.
  • Do you recommend to mix different makes and models of harddrives per vdev (to avoid that all drives fail together)?
• SSDs
  • 2x Samsung 860 PRO 256 GB (SATA)
    • Are these consumer SSDs fine for a ZFS mirror for the TrueNAS system dataset?
  • Does it make sense to use two similar SSDs or would it be better to use different makes/models/sizes?

Questions:

• Would you go with my approach or would you directly plan to use a real low power system for TrueNAS (e.g. based on an Atom C3558 or Xeon-D) and only use it as pure NAS system plus an additional system for Proxmox?
  
  • My fear is, that if I go directly with a low power Atom system, it still consumes too much power, so that a second system for Proxmox is not feasible and the Atom system is not powerful enough to also run the virtualization parts on it.
  • If directly planning for two systems, can we ensure that the systems really met the energy consumptions (could a TrueNAs system be low poer enough)?
  • If going with two systems what are your CPU and mainboard recommendations?
  • Could an Atom based system provide the full speed of two pools which are located on 2 vdevs with 4 drives in RAIDZ2 via a 10 GBit/s network connection? For my understanding the drives should deliver at a maximum 2x ~150 MB/s = 300 MB/s. Is this possible with a low power Atom? What performance can be reached with an Atom based TrueNAS system?
• Two of the above mentioned Supermicro chassis can be used as tower or 19" rack housing. When changing the usage type, the drive bays will change from horizontally to vertically. Do you see any problems for the harddrives if they will be changed from horizontally operation to vertically operation later, after some time of operation? A few years ago, a golden rule was to use drives always in the same position as they were used before. Is this still a valid rule for newer drives?
• In case Plex will be used on the TrueNAS system, it could be that I want to run a TV card for Plex. Does someone know if there are drivers for a Hauppauge WinTV-Quad HD (maybe there is a small chance that someone knows, even if this here is not a Plex, TV, Multimedia forum)?
• If we absolutely cannot meet the overall 130 watts for the TrueNAS and Proxmox system, what do you think could be a realistic power consumption for such a low power setup consisting of two machines?
• In case that two separate machines are absolutely not possible with around 130 watt. Is my hardware proposal from above ok for a TrueNAS system which shall also run the virtualization parts?

Hopefully, all your your comments, recommendations, experiences, etc. make my understanding better about which hardware is sufficient and help me with the selection of sufficient hardware.

Thanks a lot in advance,

Thomas

 

[thomas-hn]

Just pushing the thread as there was no response for a day.

 

[Inxsible]

• The idle power consumption of the system should be at a maximum of around 130 watts (my current system which is Intel Core i5-4690S, 32 GB RAM, Adaptec RAID Controller, and 8x 6TB HDDs consumes this amount of energy and the energy costs in my country are gigantic).

That's very difficult to gauge without actually having the components and a kil-a-watt and measuring everything.

The system shall serve:

• as NAS for personal data as well as multimedia data
• some Jails:
  • Plex
  • Syncthing
  • Nextcloud
  • CUPS
  • UniFi controller
  • FreePBX
  • Gitlab
• 2 VMs running Win10
• 1 VM running Docker (webserver for private use, portainer, Zoneminder)

You would be better served with a Proxmox installation to do all that. Bhyve is decent but cannot compete against Proxmox when it comes to Virtualization. You could choose to have TrueNAS as a VM under Proxmox -- provided you can pass through the disks to TrueNAS directly. Not sure where you read that running TrueNAS in a Proxmox VM is not recommended. Many users do that. If you are so concerned, run them both in separate machines.

Furthermore, I was looking for a CPU with iGPU for transcoding support in Plex.

Do you really transcode streams that much? How many streams do you run simultaneously? If not, you would not gain much by getting an iGPU integration.

Would you go with my approach or would you directly plan to use a real low power system for TrueNAS (e.g. based on an Atom C3558 or Xeon-D) and only use it as pure NAS system plus an additional system for Proxmox?

I ran FreeNAS for a long time on a Pentium G3450 and 8GB of RAM. Did I transcode? No, not more than 1 stream. I never bothered to measure the idle wattage, but the max TDP on that cpu was 53W, IIRC. Wattage consumption can depend on a number of factors in your setup and even on how clean the supply to your home is. Trying to achieve a specific wattage consumption is a wild goose chase. You'd end up fretting about that more than enjoying the build process and using the server.

2x Samsung 860 PRO 256 GB (SATA)

Sure if you want to spend more money. I'd just buy the cheapest 2 SSDs of 128GB each. Kingston A400 128GB for $20 a pop would do the job just fine. You don't need 256GB or even 128GB. I am running TrueNAS off of a 8GB Sandisk Cruzer USB2 stick for the past 6 years . Yeah yeah, I know.

If directly planning for two systems, can we ensure that the systems really met the energy consumptions (could a TrueNAs system be low poer enough)?

Yes, yes it can. See my comment about running TrueNAS on a Pentium G3450. But then you can't have it do everything under the sun.

A few years ago, a golden rule was to use drives always in the same position as they were used before. Is this still a valid rule for newer drives?

Nah. Just don't change their orientation while they are actually running.

In case Plex will be used on the TrueNAS system, it could be that I want to run a TV card for Plex. Does someone know if there are drivers for a Hauppauge WinTV-Quad HD (

You are all over the place with this. Choose a low-power system or a massive system that can do everything. You CAN'T have both in the same machine.

If we absolutely cannot meet the overall 130 watts for the TrueNAS and Proxmox system,

I can tell you that you will require much more than 130W for 8 drives + a whole bunch of components at least during bootup. If 130W of idle wattage makes you feel fuzzy inside -- then I can do better. Don't build or use a server. That's 0W right there. Can't beat that.

And what good is idle wattage? I don't want my server to consume 130W and do nothing. I'd rather it do what I need/want it to do without worrying about how much electricity consumption is needed for it. IMHO, You are looking at it the wrong way. You need to decide what you want the machine to do for your current+5 years needs. If the build satisfies that condition & stays within your budget, everything else will fall in place.
If the consumption is more than 130W -- then well, you needed it. If it's less than 130W, well then you saved a whole lot more because it satisfies your needs and saves you money (compared to what you thought you'd be paying)

I won't comment much on your actual hardware, because I don't think you have figured out what you want the server for and the components list is probably going to change drastically by the time you get there. I will say though, that you should keep away from onboard SAS controllers or 10G NICs because they can be easily added on via PCIe (what else are you going to put on those PCIe ports in a server). The advantage being that they can all be individually upgraded if you ever come to that instead of trashing the whole motherboard and starting from scratch if one of those onboard components were to fail.