TrueNAS Scale Build with ECC for up to 4x 3.5 + NVMe

[pixelwave]

I am currently looking to retire my Synology DS918+ for a custom build TrueNAS Scale System. My current spec list looks something like this:

• CPU: Intel Core i3-9100F (no iGPU)
• Mainboard: ASUS P11C-I/NGFF2280 (ASPEED AST2500 Onboard)
• RAM: Samsung 16GB DDR4 ECC (M391A1K43BB2-CTD > 2666@2400)
• Case: Fractal Design Node 304

Power I still have a ATX supply laying around:

• Power: ASUS ROG Strix 750G

With already existing storage I plan to continue to use (for now):

• HDD (RAIDZ1): 4x Western Digital WD Red 3TB, SATA 6Gb/s (WD30EFRX)
  • as main storage
• NVMe (Single): 1x Samsung SSD 950 PRO 512GB, M.2
  • for docker, vm

Currently that totals for around 500€.

For boot / OS I would additionally opt for:

• SATADom(Single or Mirror): 1-2x Innodisk SATADOM-SL 3ME3 V2 32GB

In the future I would like to upgrade 10Gbe or maybe a cheap Mellanox ConnectX-3 card (40Gbe). For that setup is an idle power consumption below 30-35W realistic?

Alternative would be a Dell PowerEdge T40 (with Xeon E-2224G + 16GB) which is comparably cheap currently (also around 500€) but the case is quite cramped and does not fit 4 3.5 HDD easily and requires additional NVMe PCIe card because the port on the mainboard is locked.

 

[QonoS]

30-35W is probably unrealistic.

PSUs are typically most power efficient with load between 50-80% . Less load means less efficiency as a general rule of thumb.

example scenario: 2 equally good PSUs, one 300W and another 750W ; 30W idle load ; 100W full load
idle : 30 / 300W => 10% load ||| 30 / 750W => 4% load
full : 100 / 300W => 33% load ||| 100 / 750W => 13% load

So that PSU is probably too big.

 

[pixelwave]

What would be a good match for that config?

 

[Etorix]

The hardware list looks generally sound, but with VMs you may need more than just 16 GB RAM.
What's not sound is the power expectation, especially if 30-35 W also includes a 10 GbE NIC. Maybe an Atom C3000 motherboard would come closer to that, but that'll cost over 500 E for the motherboard alone—and there are still the drives.

 

[QonoS]

Generally PSUs having a "80 PLUS" certification is good starting point. Gold, Platinum, Titanium are the highest ratings.

Here is their official list : https://www.clearesult.com/80plus/manufacturers/115V-Internal

Even found a Cybenetics report on the ASUS ROG Strix 750W: https://gzhls.at/blob/ldb/0/8/3/d/bb01d0e8eaa14b321bcc7cce655ae11a5ce6.pdf
As you see there below 50W efficiency is not good.

 

[pixelwave]

Generally PSUs having a "80 PLUS" certification is good starting point. Gold, Platinum, Titanium are the highest ratings.

Here is their official list : https://www.clearesult.com/80plus/manufacturers/115V-Internal

Even found 80PLUS report on the ASUS ROG Strix 750W: https://gzhls.at/blob/ldb/0/8/3/d/bb01d0e8eaa14b321bcc7cce655ae11a5ce6.pdf
As you see there below 50W efficiency is not good.

I did ... for comparison hook up my FSP150-50GBA (150W) which draws 32W idle ... compared to the ASUS ROG Strix 750W which sits at 30W idle on my old Intel Core i3-2120T with ASUS P8H67-I Deluxe. But maybe the FSP150 is crap ... :)

 

[pixelwave]

The hardware list looks generally sound, but with VMs you may need more than just 16 GB RAM.
What's not sound is the power expectation, especially if 30-35 W also includes a 10 GbE NIC. Maybe an Atom C3000 motherboard would come closer to that, but that'll cost over 500 E for the motherboard alone—and there are still the drives.

Yes I also looked at the Supermicro A2SDI-H-TF retail. Intel Atom C3758 and 2x 10Gbe sound nice at 25W TDP. But price for MB + CPU is 700+€.

I can get a new Intel Xeon E-2314 on a ASUS P12R-M/10G-2T for about 500€. Would be maybe more future proof with Socket1200 and PCIe4.

 

[QonoS]

I did ... for comparison hook up my FSP150-50GBA (150W) which draws 32W idle ... compared to the ASUS ROG Strix 750W which sits at 30W idle on my old Intel Core i3-2120T with ASUS P8H67-I Deluxe. But maybe the FSP150 is crap ... :)

Interesting, well then maybe it no bad after all. Having it tested is always best. Generally this seems ASUS is quite good. :)

 

[pixelwave]

So after digging more and more into the rabbithole i am currently torn between the current bad availbility / high prices of the Atom C3758 Supermicro chipsets and the Xeon-D chipsets seeming a bit outdated as well (Skylake being the newest one in the Supermicro lineup).

Would it make sense to wait a bit for newer cpu/boards? What comes next after Broadwell in the dense, Low-power Xeon camp with a max TDP of 35W, 10Gbe and ideally PCIe4 ... or even 5? I assume Xeon D-17xx series ...?

 

[Etorix]

The next step is Xeon D-2100, with more compute power for running more VMs, higher TDP… and WAYYY higher prices.
Xeon D-1500 is not yet outdated… if you can find an opportunity.

You have done some research and have a rather good list. A NAS does not need the latest and greatest, so I'd suggest to go for the i3-9100F in Node 304—or some prebuilt Dell, Lenovo or HP Microserver if you find one with the right amount of bays at the right price point.

 

[pixelwave]

Just to update I settled for a Ryzen 5000 system with ECC and 6x NVMe support >>>

AMD Ryzen Pro with ECC Build Review

 

[MrGuvernment]

Just to update I settled for a Ryzen 5000 system with ECC and 6x NVMe support >>>

AMD Ryzen Pro with ECC Build Review

Nice, i just saw this and was going to say, the WD Reds are SMR drives, so would want to avoid those!

List of known SMR drives

Hard drives that write data in overlapping, "shingled" tracks, have greater areal density than ones that do not. For cost and capacity reasons, manufacturers are increasingly moving to SMR, Shingled Magnetic Recording. SMR is a form of PMR...

www.truenas.com

 

[pixelwave]

Nice, i just saw this and was going to say, the WD Reds are SMR drives, so would want to avoid those!

List of known SMR drives

Hard drives that write data in overlapping, "shingled" tracks, have greater areal density than ones that do not. For cost and capacity reasons, manufacturers are increasingly moving to SMR, Shingled Magnetic Recording. SMR is a form of PMR...

www.truenas.com

Which ones? I only use CMR HDDs or NVMe SSDs ....

 

[MrGuvernment]

ya, your right, my brain was miss-reading the model you had posted above WD30EFRX , being the CRM drives!

 

[pixelwave]

Yeah.. I did read about the SMR issues so steered clear of those drives. Only question .. because I selected 4KN native drives (WD520) ..if down the road I can mix them with 512 Emulation ones?

 

[MrGuvernment]

Yeah.. I did read about the SMR issues so steered clear of those drives. Only question .. because I selected 4KN native drives (WD520) ..if down the road I can mix them with 512 Emulation ones?

Good news is your not the first to ask!

Pool with 512b and 4k sector drives

So i read this: By default, FreeNAS will treat all disks as 4K sector (“Advanced Format”) disks. This is a future-proof setting that allows AF disks to later be used as replacement drives for older, legacy 512 byte sector drives without compromising performance. The administrator can optionally...

www.truenas.com

 

[pixelwave]

Good news is your not the first to ask!

Pool with 512b and 4k sector drives

So i read this: By default, FreeNAS will treat all disks as 4K sector (“Advanced Format”) disks. This is a future-proof setting that allows AF disks to later be used as replacement drives for older, legacy 512 byte sector drives without compromising performance. The administrator can optionally...

www.truenas.com

I sure like that:

This is a future-proof setting that allows AF disks to later be used as replacement drives for older, legacy 512 byte sector drives without compromising performance.