Truenas Consumer AMD Build with ECC

[alexnetdoc]

Hi all,

I was searching for the build that would be power efficient and would have ECC support. in my country Asrock rack costs 300+ Eur only for motherboard, for a simple NAS solution it is overkill and expensive. So I ordered some parts from the local supplier with the possibility to return if this setup won't work, but to my surprise it actually worked!

Motherboard: B450M Pro4-F R2.0 - I was refering to this MB because Asrock states on their website that G series CPU's supports ECC with Pro series
*For Ryzen Series APUs (Picasso, Cezanne, Renoir and Raven Ridge), ECC is only supported with PRO CPUs.
CPU: AMD Ryzen 3 PRO 3200G
Memory: Micron 16GB Unbuffered ECC MTA9ASF2G72AZ-3G2R
PSU: Seasonic FOCUS PX PSU 550 W, 120 mm ----> Replaced with Pico PSU 150W
HDD: Seagate Ironwoolf 8TB x2
SSD: Samsung EVO 870 500GB x2
M.2 SSD: Samsung evo 970 Evo plus 250GB

Power consumption: 30 watts with all hardware installed idle
Memtest:

Truenas Scale screenshot:

I will be ordering more Micron ECC RAM from the supplier, hoping to reach 32GB of RAM for my home use.

Hope it helps to find also for others the consumer builds!

 

[alexnetdoc]

Question: it has a Realtek LAN. I have an Intel LAN card laying around

Does Truenas Scale still has issues with Realtek LAN cards?

 

[sretalla]

Does Truenas Scale still has issues with Realtek LAN cards?

Probably. Driver support for those outside of Windows is poor in general. Linux won't be much better than FreeBSD.

 

[alexnetdoc]

I have noticed that AMD CPU's have a very big drawback.
in Truenas Scale with my previous CPU i3-6100 I saw package states used up to C8
in AMD Pro 3200G I can see only C2 package state :(

in overall the power consumption from my previous Intel build is 10 more watts, but AMD adds ECC RAM. So I probably must look at it as better life :D

 

[alexnetdoc]

Update: after a few hours of work the server froze. Nothing in the logs... ping was lost, keyboard did not react. Only hard reboot did put the server back online.

I am starting to think that it's either the C states or Realtek NIC.

 

[sretalla]

For stability, I have noticed that most AMD builds need to disable Cool 'n' quiet and C states or live with regular manual reboots.

Realtek NICs can't be trusted in general.

That's a noble idea you have, but maybe not the best platform to work with to get it.

 

[Etorix]

I was searching for the build that would be power efficient and would have ECC support. in my country Asrock rack costs 300+ Eur only for motherboard, for a simple NAS solution it is overkill and expensive. So I ordered some parts from the local supplier with the possibility to return if this setup won't work, but to my surprise it actually worked!

Why wouldn't it? With an Intel NIC and CnC disabled, you'll probably be fine.

Otherwise, if you're within the return window—or more likely for other users with similar requirements for a cheap low power NAS—I stumbled yesterday on this eBay ad for a X10SDV-6C-TLN4F-B at 199,-E (I have no relationship to the seller):

SUPERMICRO X10SDV-6C-TLN4F-B, MOTHERBOARD, NO COOLING, NO SHIELD | eBay

SUPERMICRO X10SDV-6C-TLN4F-B, MOTHERBOARD, NO COOLING, NO SHIELD | Informatica en netwerken, Computer: onderdelen, Moederborden | eBay!

www.ebay.nl

The lack of heatsink is annoying since this will be a hard to find part (either for a genuine Supermicro heatsink or a compatible part from AliExpress), but these boards make lovely little NAS and it would be hard to argue against the price.

 

[alexnetdoc]

Yeah, that's very expensive :). Hopefully My needs will grow enough so that I will make a proper dedicated server for truenas (even for home).
Meanwhile I have found the reason for my freezes. It seems that I did a little bit of too much undervolting to reduce power. Increasing the RAM back from 1.1v to 1.2v which is advertised by Micron stopped the freezing so far. Also I have adjusted a bit higher voltage on the CPU core from my undervolting (I reduced from Auto to 1.025 > now went with 1.050)

So far so good

What I did appreciate very much and give soo much kudos to Truenas devs is that even if you change the motherboard and CPU with RAM, you just plug your drives and everything works as it has never changed. That's just god blessed features, thanks!

 

[alexnetdoc]

after few days I can confirm that:
* The system is running stable with Intel NIC
* if I switch to the realtek NIC the system does not freeze, it's just looses the connection at some point and there's nothing I can do except to reload the driver
* Global C states must be disabled as if you do enable it, it gives SOOOO many errors of going to disabled state messages
----------------------

With an Intel NIC and CnC disabled, you'll probably be fine.

So with this comment I think it's a fundamental answer for AMD builds. Thank you, but I had also to test myself, so invested money not only into the parts, but also into the experience

Now I have tested the chinese MB with REG ECC Approach, Intel boards and AMD boards. One last boss to handle is a server motherboard :D

 

[Etorix]

A server motherboard is just a motherboard made to server requirements. Nothing special when building a system around it.
Except for IPMI: Instead of plugging a monitor and keyboard for setting BIOS and installing an OS, you just plug a RJ45 cable into the management port and do the rest from a browser window on your regular desktop (or from the couch, using your laptop ).

 

[alexnetdoc]

A server motherboard is just a motherboard made to server requirements. Nothing special when building a system around it.
Except for IPMI: Instead of plugging a monitor and keyboard for setting BIOS and installing an OS, you just plug a RJ45 cable into the management port and do the rest from a browser window on your regular desktop (or from the couch, using your laptop ).

Thanks, I am actually waiting until Raspberry pi's prices goes down , then I can build a piKVM and can use any motherboard I want.
My idea is always not stick myself to one brand / hardware type / roadmap. I think taking approach from different angles makes the whole infrastructure more flexible as the situations may vary, so you have to be prepared for everything. Ofc. it's easy to say > spend 1500 euros for server hardware, but that's not the only thing to have :)

I think that the most successfull approach to low power systems happened with 6-8 gen Intel hardware. my testing bench draws 6 watts with i3-6100, 2 SSD's and 32GB of RAM, intel gigabit NIC. that's just crazy. I am a bit afraid that intel is just adding more chips and plates on 12-13th gen and it just increases the power usage with no benefit for the consumer (only gamers perhaps).

BTW: Does truenas support ARM cpu's?

 

[ChrisRJ]

Ofc. it's easy to say > spend 1500 euros for server hardware, but that's not the only thing to have :)

While prices seem to have gone up, used server components are what I prefer to refer people to. As an example, I paid about 200 Euros for motherboard, CPU, and RAM of my NAS (see signature). It idles at around 80 W and goes up to 120 W under load, which rarely happens.

BTW: Does truenas support ARM cpu's?

No.

 

[alexnetdoc]

While prices seem to have gone up, used server components are what I prefer to refer people to. As an example, I paid about 200 Euros for motherboard, CPU, and RAM of my NAS (see signature). It idles at around 80 W and goes up to 120 W under load, which rarely happens.


No.

Thank you for the feedback! For me though 80 Watts idle is VERY high! My whole home infrastructure currently (2 proxmox servers HA (HP Elitedesk), Synology Nas backup, TPlink POE switch 16 ports, APC Smart UPS, 3 cameras & 3 access points & a truenas eats 95 watts total :) )...

Even though I have an electrical plant at the roof and pay 22 eur for home heating, hot water / electric bills a month, I still feel that my server as it's mostly idle should not eat a lot... and I am sorry that it's a bit of a pain in the butt topic for someone, but for me it's important

 

[alexnetdoc]

The more I dig, the more I start to regret going AMD.

As I use Emby for my family trips & TV shows, it hits the CPU load so bad on 3200G Pro for straight 10-15 mins (which hits the temps of the CPU for 50 degrees C and 100% usage with 70 watts of use!). The movie is 1080p 10mbs stream in settings, then it lowers it down to 5-10% cpu load (hits similiar as intel does)... where my trusty i3-6100 was just chilling in the corner. I assume this is the quicksync technology? Is it so that Emby / Truenas does not have the AMD iGPU support for faster transcoding?

 

[alexnetdoc]

it seems that I was wrong. Intel iGPU and quicksync is not that great as I was thinking at first
I have put both 7 and 8th gen MB's and cpu's for truenas and they are experiencing the same behavior transcoding the same video. So it's not the CPU, Ryzen does a good job...

Continuing the research

 

[alexnetdoc]

update:
The system have been stable as hell! All the issues are resolved.
As for the CPU usage and Emby:
* it seems that I have mistakenly have HEVC videos from my Go pro, which my browsers did not understand properly and forced Emby to transcode
* after changing to H.264 with handbrake help I was able to make Emby do directplay to both Macbook air m1 and Windows laptop with no CPU usage! WIN

I can now start recommending AMD setups with consumer builds, just make sure it has Intel NIC is used instead of Realtek and Chill and Cool (Global C state) disabled!

 

[asdffdsa1122]

to my surprise it actually worked!

@alexnetdoc, hello, thanks much for the info, looks like a good fit for a cheapish NAS with ECC.
one concern from the manual
it looks like the board handle only 4 storage devices at the same time.
"4 x SATA3 6.0 Gb/s Connectors, support RAID (RAID 0,RAID 1 and RAID 10), NCQ, AHCI and Hot Plug*
* M2_2 and SATA3_3 share lanes. If either one of them is in use, the other one will be disabled."

yet your system has 5 storage devices used at the same time
2HDD+2SSD+1M.2

please, is that something you or anybody can help explain to me, i am so confused?

i need a system that can handle 5+ devices, 4HDD + something to boot from(HDD or SDD or M.2)

 

[Patrick M. Hausen]

@asdffdsa1122 it looks to me as if the motherboard supports both MSATA or NVMe devices in that M.2 slot. If you are using an NVMe SSD you can probably use all 4 SATA ports. This is the case with nearly all Supermicro boards I have in use. I would write to ASRock or the seller's support just to be sure in your position.