TrueNAS-12.0-U6.1 Constant Ransom Reboots (truenas.local had an unscheduled system reboot. The operating system successfully came back online)

[SKova]

I have had my TrueNAS Core system running for a few months and have had random reboots with this message:

truenas.local had an unscheduled system reboot. The operating system successfully came back online​

The frequency of these events varies, some days very few and some days more:

• 7 on 2021-12-11
• 4 on 2021-12-10
• 4 on 2021-12-09
• 6 on 2021-12-08
• etc.

I have read several threads with a similar or same message and have not been able to find any associated cause for this behavior. The times that they occur are not the same and there does not seem to be any activity that would cause it. I know that there is something triggering this, as there is always a root cause for things. This is my first TrueNAS system and I would like some guideance on how to troubleshoot for the root cause of this. Is there some debug logs or something that can be enabled to help locate this?

My system information is:
TrueNAS-12.0-U6.1 Core
Gigabyte B450 Aorus M
AMD Ryzen 5 1600
16GB G.SKILL Aegis RAM
Crucial P2 250GB NVMe M.2 SSD (Boot)
4 HGST HMS5C4040BLE640 4TB HDD

Thank you,
Stephen

 

[NugentS]

What PSU are you using?
I would also suggest running a memtest on the system for a day to see if anything shows

 

[SKova]

What PSU are you using?
I would also suggest running a memtest on the system for a day to see if anything shows

The PSU is a GAMEMAX VP-600-RGB, which I have used on several machines and they are really nice. I like the idea of the memtest, I will create a boot of that and run it. Thank you for the suggestion.

 

[NugentS]

Hmmm, GameMax, not exactly a high tier PSU, and the RGB doesn't exactly recommend it for 24*7 use either. The good news is that (at least in theory) its rated high enough based on the 600W maximum. I love how on the spec sheet the highest billing goes to the RGB
If the memory doesn't throw an error I would look at the PSU - its a cheap consumer PSU

However you might find something in /var/log to help out.

 

[Alecmascot]

Have the Ryzen-appropriate BIOS work-arounds been enabled (disable Cool-n-Quiet, disable C6 state)?

 

[LarsR]

Have the Ryzen-appropriate BIOS work-arounds been enabled (disable Cool-n-Quiet, disable C6 state)?

You've beaten me to it xD

I recently switched from first gen ryzen to third gen and with third gen those settings dont need to be disabled anymore.
So it seems it's really only the first gen that need those tweaks

 

[SKova]

Have the Ryzen-appropriate BIOS work-arounds been enabled (disable Cool-n-Quiet, disable C6 state)?

Thank you for that information. I just did those and will see if there are any changes.

 

[SKova]

Update: After many reboots and capturing the errors in a picture (I cannot find anything in a log), I may have solved this. I added a PCIe Intel Network card to the system and moved the connection to that. I was not able to disable the on-board Realtek NIC, although there is nothing connected to it. The system has run for over 24 hours with no reboots or errors. I will update this thread in a couple of days after observing and all of the details of what was used and how everything is configured.

 

[kn51]

Have the same issue, though not as often. I pretty much narrowed it down to the realtek as well looking through log and dump files.

 

[NugentS]

How did I not pick that up?
I always blame the realtek crap
:-(

 

[pschatz100]

As the Gigabyte B450 Aorus M is a gaming motherboard, it has lots of features that are not supported by TrueNAS. I would go into the bios and disable everything that is not being used. That would include the advanced power settings, sound, support for LED strips, etc. If you cannot disable the realtek NIC in the bios then make certain you disable all of the advanced networking features.

 

[SKova]

Have the same issue, though not as often. I pretty much narrowed it down to the realtek as well looking through log and dump files.

Here is what I switched to: https://smile.amazon.com/gp/product/B09D3JL14S/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1

 

[SKova]

As the Gigabyte B450 Aorus M is a gaming motherboard, it has lots of features that are not supported by TrueNAS. I would go into the bios and disable everything that is not being used. That would include the advanced power settings, sound, support for LED strips, etc. If you cannot disable the realtek NIC in the bios then make certain you disable all of the advanced networking features.

All of that was done before changing to the new NIC. It has been close to 42 hours and no reboots. It has been perfect since then.

 

[pschatz100]

Well, it is well known that Realtek NIC implementations can be the cause of significant headaches. That is why Intel Nic's are recommended.