FreeNAS Crashing Often

Bashed · May 2, 2023

Hello community,

I need your help troubleshooting why my FreeNAS system is crashing often. I have to constantly power cycle it. Not sure what's causing it, not quite comprehending what the error messages below mean. Any help is appreciated.

Version: FreeNAS-11.2-U8
Processor: Intel(R) Xeon(R) CPU E5-2430L 0 @ 2.00GHz (24 cores)
Memory: 32 GiB
Disks: 16TB Total

Error: /var/log/messages

Code:

Apr 16 00:14:39 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:39 nas MCA: CPU 16 COR (24228) OVER MS channel 2 memory error
Apr 16 00:14:39 nas MCA: Address 0x5cf6c4ac0
Apr 16 00:14:39 nas MCA: Misc 0x122100000000208c
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 5, Status 0xcc1cd34000010092
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (29517) OVER RD channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x4d862eb40
Apr 16 00:14:40 nas MCA: Misc 0x21425c3c86
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 10, Status 0xcc17df90000800c2
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (24446) OVER MS channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5cfaa7240
Apr 16 00:14:40 nas MCA: Misc 0x122100000000208c
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 5, Status 0xcc1cc20000010092
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (29448) OVER RD channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5de2986c0
Apr 16 00:14:40 nas MCA: Misc 0x214a00e086
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 10, Status 0xcc17c050000800c2
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (24321) OVER MS channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5cfe94a40
Apr 16 00:14:40 nas MCA: Misc 0x122100000000208c
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 5, Status 0xcc1ca30000010092
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (29324) OVER RD channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x4d862eb40
Apr 16 00:14:40 nas MCA: Misc 0x21425c3c86
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 10, Status 0xcc179a90000800c2
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (24170) OVER MS channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5d0275f40
Apr 16 00:14:40 nas MCA: Misc 0x122100000000208c
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 5, Status 0xcc1d844000010092
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (30225) OVER RD channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x4d862eb40
Apr 16 00:14:40 nas MCA: Misc 0x21425c3c86
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 10, Status 0xcc180e10000800c2
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (24632) OVER MS channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5d06665c0
Apr 16 00:14:40 nas MCA: Misc 0x122100000000208c
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 5, Status 0xcc1da40000010092
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (30352) OVER RD channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x4d862eb40
Apr 16 00:14:40 nas MCA: Misc 0x21425c3c86
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 10, Status 0xcc183110000800c2
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (24772) OVER MS channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5d0a64ec0
Apr 16 00:14:40 nas MCA: Misc 0x122100000000208c
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 5, Status 0xcc1d6d0000010092
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (30132) OVER RD channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x4d862eb40
Apr 16 00:14:40 nas MCA: Misc 0x21425c3c86
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 10, Status 0xcc17ded0000800c2
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (24443) OVER MS channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5d0e55240
Apr 16 00:14:40 nas MCA: Misc 0x122100000000208c
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 5, Status 0xcc1e69c000010092
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (31143) OVER RD channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x4d862eb40
Apr 16 00:14:40 nas MCA: Misc 0x21425c3c86
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 10, Status 0xcc189650000800c2
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (25177) OVER MS channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5d1254440
Apr 16 00:14:40 nas MCA: Misc 0x122100000000208c
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 5, Status 0xcc1e02c000010092
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (30731) OVER RD channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x4d862eb40
Apr 16 00:14:40 nas MCA: Misc 0x21425c3c86
Apr 16 00:14:40 nas MCA: Unable to allocate space for an event.
Apr 16 00:14:40 nas MCA: Bank 10, Status 0xcc17e750000800c2
Apr 16 00:14:40 nas MCA: Global Cap 0x0000000001000c12, Status 0x0000000000000000
Apr 16 00:14:40 nas MCA: Vendor "GenuineIntel", ID 0x206d7, APIC ID 36
Apr 16 00:14:40 nas MCA: CPU 16 COR (24477) OVER MS channel 2 memory error
Apr 16 00:14:40 nas MCA: Address 0x5d1636040
Apr 16 00:14:40 nas MCA: Misc 0x122100000000208c
May  2 22:12:45 nas syslog-ng[2842]: syslog-ng starting up; version='3.20.1'
May  2 22:12:45 nas Copyright (c) 1992-2018 The FreeBSD Project.
May  2 22:12:45 nas Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
May  2 22:12:45 nas     The Regents of the University of California. All rights reserved.
May  2 22:12:45 nas FreeBSD is a registered trademark of The FreeBSD Foundation.
May  2 22:12:45 nas FreeBSD 11.2-STABLE #0 r325575+4710c8b6420(HEAD): Fri Feb 14 13:59:19 UTC 2020
May  2 22:12:45 nas root@tnbuild02.tn.ixsystems.com:/freenas-releng/freenas/_BE/objs/freenas-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64
May  2 22:12:45 nas FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)
May  2 22:12:45 nas VT(vga): resolution 640x480
May  2 22:12:45 nas CPU: Intel(R) Xeon(R) CPU E5-2430L 0 @ 2.00GHz (2000.04-MHz K8-class CPU)

jgreco · May 2, 2023

You are experiencing a high rate of memory errors. You should test each bank of memory to identify which one is causing the problem, and replace any faulty DIMM's.

An E5-2430L is over a decade old, and it is not uncommon for these to develop various faults. You may wish to pull the CPU's and DIMM's and use some contact cleaner on all the contacts as well.

Important Announcement for the TrueNAS Community.

FreeNAS Crashing Often

Bashed

Cadet

jgreco

Resident Grinch

Similar threads