FreeNas Randomly Rebooting

[CypSteel]

Version: FreeNAS-9.2.1.2-RELEASE-x64 (002022c)

My server bounces periodically with or without load. I have checked the messages file but it shows nothing:

Code:

Oct 30 05:50:12 cypnas1 kernel: em0: promiscuous mode enabled
Oct 30 05:50:12 cypnas1 kernel: bridge0: link state changed to UP
Oct 30 05:50:12 cypnas1 kernel: epair0a: Ethernet address: 02:17:87:00:07:0a
Oct 30 05:50:12 cypnas1 kernel: epair0b: Ethernet address: 02:17:87:00:08:0b
Oct 30 05:50:12 cypnas1 kernel: epair0a: link state changed to UP
Oct 30 05:50:12 cypnas1 kernel: epair0b: link state changed to UP
Oct 30 05:50:12 cypnas1 kernel: em0: link state changed to DOWN
Oct 30 05:50:12 cypnas1 kernel: epair0a: promiscuous mode enabled
Oct 30 05:50:16 cypnas1 kernel: em0: link state changed to UP
Oct 30 10:33:30 cypnas1 syslogd: kernel boot file is /boot/kernel/kernel
Oct 30 10:33:30 cypnas1 kernel: Copyright (c) 1992-2013 The FreeBSD Project.
Oct 30 10:33:30 cypnas1 kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Oct 30 10:33:30 cypnas1 kernel: The Regents of the University of California. All rights reserved.
Oct 30 10:33:30 cypnas1 kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
Oct 30 10:33:30 cypnas1 kernel: FreeBSD 9.2-RELEASE-p3 #0 r262572+825d682: Sat Mar  1 10:02:19 PST 2014
Oct 30 10:33:30 cypnas1 kernel: root@build.ixsystems.com:/home/jkh/checkout/freenas/os-base/amd64/tank/home/jkh/checkout/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64
Oct 30 10:33:30 cypnas1 kernel: gcc version 4.2.1 20070831 patched [FreeBSD]
Oct 30 10:33:30 cypnas1 kernel: CPU: Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (3399.08-MHz K8-class CPU)
Oct 30 10:33:30 cypnas1 kernel: Origin = "GenuineIntel"  Id = 0x306c3  Family = 0x6  Model = 0x3c  Stepping = 3
Oct 30 10:33:30 cypnas1 kernel: Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Oct 30 10:33:30 cypnas1 kernel: Features2=0x7fdafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,<b11>,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
Oct 30 10:33:30 cypnas1 kernel: AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>

So in this example the server rebooted about 10:33:30 with no warning or error logs. I plan to run memtest as indicated from other posts, is there anything else I can try?

 

[L]

Is that 5 hours between the errors and boot? Did it dump core? check in .systems/core

 

[CypSteel]

Thanks for your reply Linda!!! I don't think the errors are related, they are just from the last boot up sequence at 5:50. There is no corresponding workload or timeframe between boots. Today boot times were: 04:55:55, 05:50:00, 10:33:30. There was literally nothing going on or any users during that time.

I don't have a /system, /systems, .system, or .systems. That must be the problem! There is no system! ***sorry bad joke***

Fun Fact after doing some research in the old message logs: It doesn't look like it rebooted at all in the month of September.

This leads me to believe maybe 2 things might be contributing factors:

1. I upgraded my Plex Media Server jail to a new version sometime early-mid october.
2. I had a loss of power and the FreeNAS server went down hard twice in early october. This has since been remediated with a UPS. If there was a corruption of some sort, would I know about it?

Thanks again for any insight.

 

[L]

the .system data set will be somewhere in your pool, I didn't put in the whole path name cuz i don't know your poolname.. I expect that you will find a hardware anomoly.

also i would run a scrub and see if there is anything in the pool causing the problem

 

[CypSteel]

Thanks again Linda. Cores directory is empty. I will initiate e a scrub.

 

[L]

in advanced settings make sure traceback is turned on. also enable auto upload of kernel.

 

[L]

Actually I am now hooked on figuring this out.. Can you save out debug and send it to me?? linda@kateley.com

 

[CypSteel]

Both the traceback and autoupload was already enabled. Scrub in progress. Debug file headed your way. Thanks again!

 

[cyberjock]

The debug really should be attached to the forums for others (like myself) to look at.

You provided ZERO of the requested info per the forum rules, so please read the forum rules and post your info...

Don't be surprised if the first answer is "upgrade". 9.2.1.2 is like 8 months old and should NOT be around any more.