CPU recommendations

[steveroch-rs]

Hello,

reading various official hardware sizing guides and user builds I find it hard to decide how much CPU horsepower I'll need for my use-case.

I'm planning to build a FreeNAS box that uses very little power <80W when idling with 12-16 drives but has high performance and throughput when needed for Plex, Backup and file transfer.

So what so I plan to do with it?
1. PLEX for 3-6 users with trandcoding FullHD movies
2. downloading with sonarr, radarr, lidarr
3. iSCSI storage for proxmox
4. Backup server
5. Storage for Home surveillance
6. Database server

All in all it will be the central data storage for everything in my network.

I thought about an 8c/16t Intel Xeon nur maybe something less powerful is already sufficient.

I'm looking foward to your suggestions and feel freue to ask further questions. :)

 

[ornias]

reading various official hardware sizing guides and user builds I find it hard to decide how much CPU horsepower I'll need for my use-case.

You would be amazed how much info isn't in any guide or build... :(

I'm planning to build a FreeNAS box that uses very little power <80W when idling with 12-16 drives but has high performance and throughput when needed for Plex, Backup and file transfer.

Well I LOVE power efficiency builds :D
So here we go:

- Platform with highest idle efficency (for the whole system!): Intel (without a doubt), without disks, pcie-cards and peripherals 5W idle is duable.
- Motherboard manufacturer with highest power efficiency (and preferably server grade): Fujitsu, now rebranded Kontron
- Disks type: If you can manage to get the disk to actually spin down 3.5" is doable, but for power efficiency I would advice 2.5" drives. In both cases the biggest drives you can find, funny enough power consumption does not increase with disk size liniar
-Home Surveillance, okey your disks are not going to spin down much, period... so your idle is basically "Idle with disks". In that case I would advice 2.5"drives, because you are hard pressed in doing 80W idle with 3.5" disk spinning
- Database server: More vdevs = more iops.
- Plex: Gen 4-7 Quicksync hardware acceleration is already possible with FreeNAS 11.3 ( https://github.com/kern2011/Freenas-Quicksync), Quicksync for Gen 8-9 will be comming with TrueNAS core. This means HUGE efficiency improvement and (more importantly) less CPU load!

Actually reading it we both have the same requirements...
I'm currently saving for the following setup:
- Fujitsu D3643-H
- Intel 6c or 8c (depending on price development) with iGPU for quicksync
- 4x16GB ram (if ECC, max 6c cpu due to quicksync req.)
- 10Gbe card
- either a good sas card (enough ports) or a cheaper sas card and a expander(backplane)
- Load of disks: thinking about 12-18 2.5" drives + 3 SSD's for Special allocation classes (coming in TrueNAS core) and 2 SSD's for FreeNAS.
- A good PSU with high idle efficiency...


Above build is designed with idle and load power efficiency in mind :)
Hope this helps you out a little!

 

[steveroch-rs]

- Disks type: If you can manage to get the disk to actually spin down 3.5" is doable, but for power efficiency I would advice 2.5" drives. In both cases the biggest drives you can find, funny enough power consumption does not increase with disk size liniar
-Home Surveillance, okey your disks are not going to spin down much, period... so your idle is basically "Idle with disks". In that case I would advice 2.5"drives, because you are hard pressed in doing 80W idle with 3.5" disk spinning
- Database server: More vdevs = more iops.

Thanks for the fast reply.
The home surveillance thing is more of a maybe than a definitely. I was thinking about either WD Red Pros or Seagate Ironwolf Pros, but leaning towards WD. What are your thoughts on that?
How is disk spin-down handled in FreeNAS? Is it per vdev or per drive accessed?
I thought about 6-8TB drives because I only """""need""""" 40-60TB total storage.
For vdevs I was planning to do 3 or 4 RAIDZ1s and just have 2 drives laying around in case one fails. What do you think about that?

The Xeon 2146G looks pretty good to me but what mainboard I should use is the question.
The Fujitsu one you posted does not have ECC support nor is it C236 chipset.
Maybe we can do further resarch together. I found the Supermicro X11 boards to be pretty interesting.

 

[ornias]

I'll keep it short-ish this time:

- Supermicro are good boards, but not the most power efficient...
- My "biggest drives you can find" was missing some nuance... What I mean to say: "get decent size disks" meaning "not 1TB disks" 6-8TB is perfectly reasonable
-Maybe or not, surveillance does mean your disks are constantly getting data writen, which changes the chances of spin down to about 0. Which means your "idle" would include spinning disks, which is significant.
- In short (and yes this is not 100% accurate*, but lets call it 95%) FreeNAS writes to all VDEVS in a pool at once and with raidz it writes to all disks in a vdev at once. So either all disks are idle (and thus might(!) spin down) or all stay active and keep running.

So simply put: If you keep writing constantly, the disks never idle and thus never reach a chance to spin down. Regardless of how spindown itself is handled... (For spindown, a disk needs to be idle. If no disk ever idles, it wouldn't spin down regardless of the spindown setting).

anyhow:
Spindown is per drive setting... But keep in mind that all drives are mostly writen at once* and the times for spindown (regardless of how low its set) only starts counting down when a drive is on standby.

WD Red Pros or Seagate Ironwolf Pros are good drives, but afaik write power consumption is about 5-6W* PER DRIVE. So with 16 drives that's already 80watt... It's a topic not discussed much, but disks are actually quite a heavy power sink even at low load...

* conditions do apply

 

[steveroch-rs]

I have done some further research and did some honest calculations.
The need for Space I have currently won't ever exceed 10TB so I thought about getting 2 4-drive RAIDZ1 vdevs resulting in a total capacity of 24TiB (with 20% free space in mind). Maybe I'll just do 1 vdev resulting in 12TiB. I can still expand later on.
But 16 6TB drives would add to about €3.8k just for the drives :OOOO for space I won't ever need.

I also won't be doing any surveillance with it and use a seperate system to record that data when I need it.
So I can say my drives will spin down most of the time which is great.

In Germany there is this awesome shop selling used but well kept server hardware from data-centers for very low prices. I think I'll get my hands on one of those HP DL380p systems. without disks they should idle at ~80W maybe even less. And they are dual socket machines so I won't have to worry about CPU horsepower or RAM expandability.

I have to thank you for your kind and detailed support but for me that solution is cheaper than building one of my own. Also the RAM and HDD upgrades are way cheaper than buying they would be when building my own.

 

[ornias]

Ohh I forgot something:
Don't use raidz1

Not (only) because you might expect another complete disk failure, but because of dataloss due to on-disk data corruption.
Simply put: If you have RAIDZ1 and a disk goes down, there is a chance some of the data on the remaining disk is broken and there wouldn't be a way to repair it. Said chance is higher than that of total disk failure, which is already significant with current-day disk sizes.

I would advice just using 1 RAIDZ2 vdev with 6 drives, that leaves you the option to upgrade to 2 or even 3 vdevs in the future.
Your workload is also not very io-heavy, which helps when just doing 1 vdev.

I also won't be doing any surveillance with it and use a separate system to record that data when I need it.

You could use the same system, but just do 2 pools... In that case one pool could spin down while the other keeps running to store surveillance.;)

In Germany there is this awesome shop selling used but well kept server hardware from data-centers for very low prices. I think I'll get my hands on one of those HP DL380p systems. without disks they should idle at ~80W maybe even less. And they are dual socket machines so I won't have to worry about CPU horsepower or RAM expandability.

I wouldn't find 80W without disks acceptable personally, but to each their own :)

I have to thank you for your kind and detailed support but for me that solution is cheaper than building one of my own. Also the RAM and HDD upgrades are way cheaper than buying they would be when building my own.

HDD upgrades are generally cheaper when building your own...
RAM is another story, DDR3 is indeed cheaper than building a new DDR4 system ;)