SOLVED upgraded to faster CPU, TrueNAS keeps hanging, restarting

[digity]

I upgraded the CPU in my TrueNAS Core 12 u8.1 server from a XEON E5-2620 v2 CPU (2.1 GHz, 6c/12t) to a XEON E5-2690 v2 CPU (3.0 GHz, 10c/20t) in hopes of getting better throughput from my 4x 1.6TB U.2 NVMe stripe pool (for VM storage, via 40 Gb NICs). The problem is TrueNAS now hangs for about 30 seconds and then restarts itself whenever a certain amount of data is written to the NVMe pool. To verify this I ran dd to benchmark and TrueNAS hangs almost immediately after a couple of GBs are written. If I dd on a rust/HDD pool TrueNAS runs fine, no hanging. If I pop the E5-2620 v2 CPU back in TrueNAS runs fine again - no hanging & restarting on a load or when benchmarking with dd on the NVMe pool. As another test, I stress tested the E5-2690 v2 CPU for 10+ minutes under an Ubuntu installation and run just fine.

Any ideas why TrueNAS Core is hanging, crashing with the E5-2690 v2 CPU??


P.S. - NVMe HBA is a Linkreal 4 Port PCIe 3.0 x16 to U.2 NVMe SSD Adapter with SFF-8643 Connector (PLX8747 Chipset). Mobo is a ASUS P9X79 Deluxe with 32 GB RAM (non-ECC).

P.P.S - I checked the log for clues (/var/log/messages), but it does not show an entry for the crash time - it just shows the log since the current boot/uptime.

P.P.P.S. - I'm getting only 1000 to 1200 MB/s reads and writes over both the network (CrystalDiskMark on Windows 10) and locally (dd on TrueNAS) with the E5-2620 v2 CPU, in case you're wondering. Hence why I'm trying to upgrade to the E5-2690 v2 CPU.

 

[digity]

So I upgraded to TrueNAS Scale thinking it was a Core/FreeBSD problem, but Scale hangs too when there's a load on the NVMe pool using the 2690 v2 CPU. Any other ideas?

 

[sretalla]

BIOS settings, particularly sleep states... disable them.

 

[digity]

Welp! This is embarrassing - the culprit was power and the SAS/SATA HBA card I had added during the CPU upgrade. I had a cheapo 485 watt power supply in there and it simple gives up during the new power spike of the newer 130 watt CPU on load (the old one was 80 watt CPU). It spikes up to 350 watts. Out of curiosity, I removed the SAS HBA and the outgoing PSU just barely handles the power spike with an I/O load on the NVMe pool. I swapped that PSU out for a Corsair 650 watt and everything runs fine with the SAS HBA and the new CPU installed.

Hope that helps someone.