Freenas locking up every few weeks

[nvader]

My x64 freenas 8.3.0 seems to be locking up every few weeks. I have to go and hit the reset button on the machine.
when i look at the logs in /var/logs they all seem to be new from the reboot
how i can track down what is causing the problem?

 

[William Grzybowski]

First of all you need to state your build.

Have you checked the console? Is there a kernel panic message?

 

[nvader]

Thank you for the reply.
I did state it in the 1st line
x64 version 8.3.0
Apologies if that wasn't as specific as it needed to be.
will this be more helpful?
FreeBSD 8.3-RELEASE-p5 #2 r244158M: Wed Dec 12 10:04:42 PST 2012

every few weeks at random, it will just hard freeze and i have to press the reset button.
when I do, all logs are re-created and no history is retained.
so if there is some type of kernel panic message, I can't see it once the system restarts.
my system is in a good case with plenty of ventilation. So i don't think it's a temp issue.

any ideas? how can I make my logs persist past a reboot?

 

[William Grzybowski]

When I asked for you build, I meant your full hardware specs in addition to software version.

Yeah, I get that it freezes in a few weeks, you said that in the first post.
Like a said, for kernel panic message you must check the console, prior to reboot.

 

[nvader]

Sorry for the confusion. My system specs are below.
Unfortunately my system is hard locked, so i can't see any messages in the console.
I see the ASCII freeBsd devil guy and that' sit.. nothing else and it's totally frozen.

Build FreeNAS-8.3.0-RELEASE-p1-x64 (r12825)
Platform Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
Memory 8175MB

Mar 3 10:37:51 freenas newsyslog[1806]: logfile first created
Mar 3 10:37:51 freenas syslogd: kernel boot file is /boot/kernel/kernel
Mar 3 10:37:51 freenas kernel: Copyright (c) 1992-2012 The FreeBSD Project.
Mar 3 10:37:51 freenas kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Mar 3 10:37:51 freenas kernel: The Regents of the University of California. All rights reserved.
Mar 3 10:37:51 freenas kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
Mar 3 10:37:51 freenas kernel: FreeBSD 8.3-RELEASE-p5 #2 r244158M: Wed Dec 12 10:04:42 PST 2012
Mar 3 10:37:51 freenas kernel: root@build.ixsystems.com:/home/jpaetzel/8.3.0/os-base/amd64/usr/home/jpaetzel/8.3.0/FreeBSD/src/sys/FREENAS.amd64 amd64
Mar 3 10:37:51 freenas kernel: Timecounter "i8254" frequency 1193182 Hz quality 0
Mar 3 10:37:51 freenas kernel: CPU: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz (2999.67-MHz K8-class CPU)
Mar 3 10:37:51 freenas kernel: Origin = "GenuineIntel" Id = 0x1067a Family = 6 Model = 17 Stepping = 10
Mar 3 10:37:51 freenas kernel: Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Mar 3 10:37:51 freenas kernel: Features2=0x408e3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE>
Mar 3 10:37:51 freenas kernel: AMD Features=0x20100800<SYSCALL,NX,LM>
Mar 3 10:37:51 freenas kernel: AMD Features2=0x1<LAHF>
Mar 3 10:37:51 freenas kernel: TSC: P-state invariant
Mar 3 10:37:51 freenas kernel: real memory = 8589934592 (8192 MB)
Mar 3 10:37:51 freenas kernel: avail memory = 8257622016 (7875 MB)
Mar 3 10:37:51 freenas kernel: ACPI APIC Table: <GBT GBTUACPI>
Mar 3 10:37:51 freenas kernel: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
Mar 3 10:37:51 freenas kernel: FreeBSD/SMP: 1 package(s) x 2 core(s)
Mar 3 10:37:51 freenas kernel: cpu0 (BSP): APIC ID: 0
Mar 3 10:37:51 freenas kernel: cpu1 (AP): APIC ID: 1
Mar 3 10:37:51 freenas kernel: WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
Mar 3 10:37:51 freenas kernel: ioapic0: Changing APIC ID to 2
Mar 3 10:37:51 freenas kernel: ioapic0 <Version 2.0> irqs 0-23 on motherboard
Mar 3 10:37:51 freenas kernel: kbd1 at kbdmux0
Mar 3 10:37:51 freenas kernel: hpt27xx: RocketRAID 27xx controller driver v1.0 (Dec 12 2012 10:04:31)
Mar 3 10:37:51 freenas kernel: cryptosoft0: <software crypto> on motherboard
Mar 3 10:37:51 freenas kernel: aesni0: No AESNI support.
Mar 3 10:37:51 freenas kernel: acpi0: <GBT GBTUACPI> on motherboard
Mar 3 10:37:51 freenas kernel: acpi0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: acpi0: Power Button (fixed)
Mar 3 10:37:51 freenas kernel: acpi0: reservation of 0, a0000 (3) failed
Mar 3 10:37:51 freenas kernel: acpi0: reservation of 100000, dfde0000 (3) failed
Mar 3 10:37:51 freenas kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
Mar 3 10:37:51 freenas kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
Mar 3 10:37:51 freenas kernel: cpu0: <ACPI CPU> on acpi0
Mar 3 10:37:51 freenas kernel: ACPI Warning: Incorrect checksum in table [TAMG] - 0xB0, should be 0xAF (20101013/tbutils-354)
Mar 3 10:37:51 freenas kernel: cpu1: <ACPI CPU> on acpi0
Mar 3 10:37:51 freenas kernel: acpi_button0: <Power Button> on acpi0
Mar 3 10:37:51 freenas kernel: pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
Mar 3 10:37:51 freenas kernel: pci0: <ACPI PCI bus> on pcib0
Mar 3 10:37:51 freenas kernel: pcib1: <PCI-PCI bridge> irq 16 at device 1.0 on pci0
Mar 3 10:37:51 freenas kernel: pci1: <PCI bus> on pcib1
Mar 3 10:37:51 freenas kernel: vgapci0: <VGA-compatible display> port 0xee00-0xeeff mem 0xe0000000-0xefffffff,0xfddc0000-0xfdddffff irq 16 at device 0.0 on pci1
Mar 3 10:37:51 freenas kernel: pci1: <multimedia, HDA> at device 0.1 (no driver attached)
Mar 3 10:37:51 freenas kernel: uhci0: <Intel 82801JI (ICH10) USB controller USB-D> port 0xff00-0xff1f irq 16 at device 26.0 on pci0
Mar 3 10:37:51 freenas kernel: uhci0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: usbus0: <Intel 82801JI (ICH10) USB controller USB-D> on uhci0
Mar 3 10:37:51 freenas kernel: uhci1: <Intel 82801JI (ICH10) USB controller USB-E> port 0xfe00-0xfe1f irq 21 at device 26.1 on pci0
Mar 3 10:37:51 freenas kernel: uhci1: [ITHREAD]
Mar 3 10:37:51 freenas kernel: usbus1: <Intel 82801JI (ICH10) USB controller USB-E> on uhci1
Mar 3 10:37:51 freenas kernel: uhci2: <Intel 82801JI (ICH10) USB controller USB-F> port 0xfd00-0xfd1f irq 18 at device 26.2 on pci0
Mar 3 10:37:51 freenas kernel: uhci2: [ITHREAD]
Mar 3 10:37:51 freenas kernel: usbus2: <Intel 82801JI (ICH10) USB controller USB-F> on uhci2
Mar 3 10:37:51 freenas kernel: ehci0: <Intel 82801JI (ICH10) USB 2.0 controller USB-B> mem 0xfdfff000-0xfdfff3ff irq 18 at device 26.7 on pci0
Mar 3 10:37:51 freenas kernel: ehci0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: usbus3: EHCI version 1.0
Mar 3 10:37:51 freenas kernel: usbus3: <Intel 82801JI (ICH10) USB 2.0 controller USB-B> on ehci0
Mar 3 10:37:51 freenas kernel: pci0: <multimedia, HDA> at device 27.0 (no driver attached)
Mar 3 10:37:51 freenas kernel: pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
Mar 3 10:37:51 freenas kernel: pci2: <ACPI PCI bus> on pcib2
Mar 3 10:37:51 freenas kernel: pcib3: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
Mar 3 10:37:51 freenas kernel: pci3: <ACPI PCI bus> on pcib3
Mar 3 10:37:51 freenas kernel: ahci0: <JMicron JMB363 AHCI SATA controller> mem 0xfdbfe000-0xfdbfffff irq 19 at device 0.0 on pci3
Mar 3 10:37:51 freenas kernel: ahci0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ahci0: AHCI v1.00 with 2 3Gbps ports, Port Multiplier supported
Mar 3 10:37:51 freenas kernel: ahcich0: <AHCI channel> at channel 0 on ahci0
Mar 3 10:37:51 freenas kernel: ahcich0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ahcich1: <AHCI channel> at channel 1 on ahci0
Mar 3 10:37:51 freenas kernel: ahcich1: [ITHREAD]
Mar 3 10:37:51 freenas kernel: atapci0: <JMicron JMB363 UDMA133 controller> port 0xdf00-0xdf07,0xde00-0xde03,0xdd00-0xdd07,0xdc00-0xdc03,0xdb00-0xdb0f irq 16 at device 0.1 on pci3
Mar 3 10:37:51 freenas kernel: atapci0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ata2: <ATA channel> at channel 0 on atapci0
Mar 3 10:37:51 freenas kernel: ata2: [ITHREAD]
Mar 3 10:37:51 freenas kernel: pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
Mar 3 10:37:51 freenas kernel: pci4: <ACPI PCI bus> on pcib4
Mar 3 10:37:51 freenas kernel: re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F PCIe Gigabit Ethernet> port 0xce00-0xceff mem 0xfdeff000-0xfdefffff,0xfdee0000-0xfdeeffff irq 16 at device 0.0 on pci4
Mar 3 10:37:51 freenas kernel: re0: Using 1 MSI-X message
Mar 3 10:37:51 freenas kernel: re0: Chip rev. 0x3c000000
Mar 3 10:37:51 freenas kernel: re0: MAC rev. 0x00400000
Mar 3 10:37:51 freenas kernel: miibus0: <MII bus> on re0
Mar 3 10:37:51 freenas kernel: rgephy0: <RTL8169S/8110S/8211B media interface> PHY 1 on miibus0
Mar 3 10:37:51 freenas kernel: rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
Mar 3 10:37:51 freenas kernel: re0: Ethernet address: 00:1f:d0:9c:63:9f
Mar 3 10:37:51 freenas kernel: re0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: uhci3: <Intel 82801JI (ICH10) USB controller USB-A> port 0xfc00-0xfc1f irq 23 at device 29.0 on pci0
Mar 3 10:37:51 freenas kernel: uhci3: [ITHREAD]
Mar 3 10:37:51 freenas kernel: usbus4: <Intel 82801JI (ICH10) USB controller USB-A> on uhci3
Mar 3 10:37:51 freenas kernel: uhci4: <Intel 82801JI (ICH10) USB controller USB-B> port 0xfb00-0xfb1f irq 19 at device 29.1 on pci0
Mar 3 10:37:51 freenas kernel: uhci4: [ITHREAD]
Mar 3 10:37:51 freenas kernel: usbus5: <Intel 82801JI (ICH10) USB controller USB-B> on uhci4
Mar 3 10:37:51 freenas kernel: uhci5: <Intel 82801JI (ICH10) USB controller USB-C> port 0xfa00-0xfa1f irq 18 at device 29.2 on pci0
Mar 3 10:37:51 freenas kernel: uhci5: [ITHREAD]
Mar 3 10:37:51 freenas kernel: usbus6: <Intel 82801JI (ICH10) USB controller USB-C> on uhci5
Mar 3 10:37:51 freenas kernel: ehci1: <Intel 82801JI (ICH10) USB 2.0 controller USB-A> mem 0xfdffe000-0xfdffe3ff irq 23 at device 29.7 on pci0
Mar 3 10:37:51 freenas kernel: ehci1: [ITHREAD]
Mar 3 10:37:51 freenas kernel: usbus7: EHCI version 1.0
Mar 3 10:37:51 freenas kernel: usbus7: <Intel 82801JI (ICH10) USB 2.0 controller USB-A> on ehci1
Mar 3 10:37:51 freenas kernel: pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
Mar 3 10:37:51 freenas kernel: pci5: <ACPI PCI bus> on pcib5
Mar 3 10:37:51 freenas kernel: fwohci0: <Texas Instruments TSB43AB23> mem 0xfdcff000-0xfdcff7ff,0xfdcf8000-0xfdcfbfff irq 23 at device 7.0 on pci5
Mar 3 10:37:51 freenas kernel: fwohci0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: fwohci0: OHCI version 1.10 (ROM=0)
Mar 3 10:37:51 freenas kernel: fwohci0: No. of Isochronous channels is 4.
Mar 3 10:37:51 freenas kernel: fwohci0: EUI64 00:7b:69:5a:00:00:1f:d0
Mar 3 10:37:51 freenas kernel: fwohci0: Phy 1394a available S400, 3 ports.
Mar 3 10:37:51 freenas kernel: fwohci0: Link S400, max_rec 2048 bytes.
Mar 3 10:37:51 freenas kernel: firewire0: <IEEE1394(FireWire) bus> on fwohci0
Mar 3 10:37:51 freenas kernel: sbp0: <SBP-2/SCSI over FireWire> on firewire0
Mar 3 10:37:51 freenas kernel: fwohci0: Initiate bus reset
Mar 3 10:37:51 freenas kernel: fwohci0: fwohci_intr_core: BUS reset
Mar 3 10:37:51 freenas kernel: fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, CYCLEMASTER mode
Mar 3 10:37:51 freenas kernel: isab0: <PCI-ISA bridge> at device 31.0 on pci0
Mar 3 10:37:51 freenas kernel: isa0: <ISA bus> on isab0
Mar 3 10:37:51 freenas kernel: ahci1: <Intel ICH10 AHCI SATA controller> port 0xf900-0xf907,0xf800-0xf803,0xf700-0xf707,0xf600-0xf603,0xf500-0xf51f mem 0xfdffd000-0xfdffd7ff irq 19 at device 31.2 on pci0
Mar 3 10:37:51 freenas kernel: ahci1: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ahci1: AHCI v1.20 with 6 3Gbps ports, Port Multiplier supported
Mar 3 10:37:51 freenas kernel: ahcich2: <AHCI channel> at channel 0 on ahci1
Mar 3 10:37:51 freenas kernel: ahcich2: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ahcich3: <AHCI channel> at channel 1 on ahci1
Mar 3 10:37:51 freenas kernel: ahcich3: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ahcich4: <AHCI channel> at channel 2 on ahci1
Mar 3 10:37:51 freenas kernel: ahcich4: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ahcich5: <AHCI channel> at channel 3 on ahci1
Mar 3 10:37:51 freenas kernel: ahcich5: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ahcich6: <AHCI channel> at channel 4 on ahci1
Mar 3 10:37:51 freenas kernel: ahcich6: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ahcich7: <AHCI channel> at channel 5 on ahci1
Mar 3 10:37:51 freenas kernel: ahcich7: [ITHREAD]
Mar 3 10:37:51 freenas kernel: pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
Mar 3 10:37:51 freenas kernel: acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0
Mar 3 10:37:51 freenas kernel: Timecounter "HPET" frequency 14318180 Hz quality 900
Mar 3 10:37:51 freenas kernel: atrtc0: <AT realtime clock> port 0x70-0x73 on acpi0
Mar 3 10:37:51 freenas kernel: fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
Mar 3 10:37:51 freenas kernel: fdc0: [FILTER]
Mar 3 10:37:51 freenas kernel: uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
Mar 3 10:37:51 freenas kernel: uart0: [FILTER]
Mar 3 10:37:51 freenas kernel: ppc0: <Parallel port> port 0x378-0x37f irq 7 on acpi0
Mar 3 10:37:51 freenas kernel: ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
Mar 3 10:37:51 freenas kernel: ppc0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: ppbus0: <Parallel port bus> on ppc0
Mar 3 10:37:51 freenas kernel: lpt0: <Printer> on ppbus0
Mar 3 10:37:51 freenas kernel: lpt0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: lpt0: Interrupt-driven port
Mar 3 10:37:51 freenas kernel: orm0: <ISA Option ROM> at iomem 0xd3000-0xd5fff on isa0
Mar 3 10:37:51 freenas kernel: sc0: <System console> at flags 0x100 on isa0
Mar 3 10:37:51 freenas kernel: sc0: VGA <16 virtual consoles, flags=0x300>
Mar 3 10:37:51 freenas kernel: vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Mar 3 10:37:51 freenas kernel: atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
Mar 3 10:37:51 freenas kernel: atkbd0: <AT Keyboard> irq 1 on atkbdc0
Mar 3 10:37:51 freenas kernel: kbd0 at atkbd0
Mar 3 10:37:51 freenas kernel: atkbd0: [GIANT-LOCKED]
Mar 3 10:37:51 freenas kernel: atkbd0: [ITHREAD]
Mar 3 10:37:51 freenas kernel: acpi_perf0: <ACPI CPU Frequency Control> on cpu0
Mar 3 10:37:51 freenas kernel: coretemp0: <CPU On-Die Thermal Sensors> on cpu0
Mar 3 10:37:51 freenas kernel: p4tcc0: <CPU Frequency Thermal Control> on cpu0
Mar 3 10:37:51 freenas kernel: coretemp1: <CPU On-Die Thermal Sensors> on cpu1
Mar 3 10:37:51 freenas kernel: est1: <Enhanced SpeedStep Frequency Control> on cpu1
Mar 3 10:37:51 freenas kernel: est: CPU supports Enhanced Speedstep, but is not recognized.
Mar 3 10:37:51 freenas kernel: est: cpu_vendor GenuineIntel, msr 616092606000926
Mar 3 10:37:51 freenas kernel: device_attach: est1 attach returned 6
Mar 3 10:37:51 freenas kernel: p4tcc1: <CPU Frequency Thermal Control> on cpu1
Mar 3 10:37:51 freenas kernel: fuse4bsd: version 0.3.9-pre1, FUSE ABI 7.8
Mar 3 10:37:51 freenas kernel: Timecounters tick every 1.000 msec
Mar 3 10:37:51 freenas kernel: hpt27xx: no controller detected.
Mar 3 10:37:51 freenas kernel: firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me)
Mar 3 10:37:51 freenas kernel: firewire0: bus manager 0
Mar 3 10:37:51 freenas kernel: usbus0: 12Mbps Full Speed USB v1.0
Mar 3 10:37:51 freenas kernel: usbus1: 12Mbps Full Speed USB v1.0
Mar 3 10:37:51 freenas kernel: usbus2: 12Mbps Full Speed USB v1.0
Mar 3 10:37:51 freenas kernel: usbus3: 480Mbps High Speed USB v2.0
Mar 3 10:37:51 freenas kernel: usbus4: 12Mbps Full Speed USB v1.0
Mar 3 10:37:51 freenas kernel: usbus5: 12Mbps Full Speed USB v1.0
Mar 3 10:37:51 freenas kernel: usbus6: 12Mbps Full Speed USB v1.0
Mar 3 10:37:51 freenas kernel: usbus7: 480Mbps High Speed USB v2.0
Mar 3 10:37:51 freenas kernel: ugen0.1: <Intel> at usbus0
Mar 3 10:37:51 freenas kernel: uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
Mar 3 10:37:51 freenas kernel: ugen1.1: <Intel> at usbus1
Mar 3 10:37:51 freenas kernel: uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
Mar 3 10:37:51 freenas kernel: ugen2.1: <Intel> at usbus2
Mar 3 10:37:51 freenas kernel: uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
Mar 3 10:37:51 freenas kernel: ugen3.1: <Intel> at usbus3
Mar 3 10:37:51 freenas kernel: uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
Mar 3 10:37:51 freenas kernel: ugen4.1: <Intel> at usbus4
Mar 3 10:37:51 freenas kernel: uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
Mar 3 10:37:51 freenas kernel: ugen5.1: <Intel> at usbus5
Mar 3 10:37:51 freenas kernel: uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
Mar 3 10:37:51 freenas kernel: ugen6.1: <Intel> at usbus6
Mar 3 10:37:51 freenas kernel: uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
Mar 3 10:37:51 freenas kernel: ugen7.1: <Intel> at usbus7
Mar 3 10:37:51 freenas kernel: uhub7: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7
Mar 3 10:37:51 freenas kernel: uhub0: 2 ports with 2 removable, self powered
Mar 3 10:37:51 freenas kernel: uhub1: 2 ports with 2 removable, self powered
Mar 3 10:37:51 freenas kernel: uhub2: 2 ports with 2 removable, self powered
Mar 3 10:37:51 freenas kernel: uhub4: 2 ports with 2 removable, self powered
Mar 3 10:37:51 freenas kernel: uhub5: 2 ports with 2 removable, self powered
Mar 3 10:37:51 freenas kernel: uhub6: 2 ports with 2 removable, self powered
Mar 3 10:37:51 freenas kernel: uhub3: 6 ports with 6 removable, self powered
Mar 3 10:37:51 freenas kernel: uhub7: 6 ports with 6 removable, self powered
Mar 3 10:37:51 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 3 10:37:51 freenas kernel: ugen7.2: <SanDisk> at usbus7
Mar 3 10:37:51 freenas kernel: umass0: <SanDisk Firebird USB Flash Drive, class 0/0, rev 2.00/1.26, addr 2> on usbus7
Mar 3 10:37:51 freenas kernel: ada0 at ahcich2 bus 0 scbus4 target 0 lun 0
Mar 3 10:37:51 freenas kernel: ada0: <ST3000DM001-9YN166 CC4H> ATA-8 SATA 3.x device
Mar 3 10:37:51 freenas kernel: ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Mar 3 10:37:51 freenas kernel: ada0: Command Queueing enabled
Mar 3 10:37:51 freenas kernel: ada0: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
Mar 3 10:37:51 freenas kernel: ada1 at ahcich3 bus 0 scbus5 target 0 lun 0
Mar 3 10:37:51 freenas kernel: ada1: <ST3000DM001-1CH166 CC43> ATA-8 SATA 3.x device
Mar 3 10:37:51 freenas kernel: ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Mar 3 10:37:51 freenas kernel: ada1: Command Queueing enabled
Mar 3 10:37:51 freenas kernel: ada1: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
Mar 3 10:37:51 freenas kernel: ada2 at ahcich4 bus 0 scbus6 target 0 lun 0
Mar 3 10:37:51 freenas kernel: ada2: <ST3000DM001-1CH166 CC43> ATA-8 SATA 3.x device
Mar 3 10:37:51 freenas kernel: ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Mar 3 10:37:51 freenas kernel: ada2: Command Queueing enabled
Mar 3 10:37:51 freenas kernel: ada2: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
Mar 3 10:37:51 freenas kernel: ada3 at ahcich5 bus 0 scbus7 target 0 lun 0
Mar 3 10:37:51 freenas kernel: ada3: <ST3000DM001-1CH166 CC43> ATA-8 SATA 3.x device
Mar 3 10:37:51 freenas kernel: ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Mar 3 10:37:51 freenas kernel: ada3: Command Queueing enabled
Mar 3 10:37:51 freenas kernel: ada3: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
Mar 3 10:37:51 freenas kernel: ada4 at ahcich6 bus 0 scbus8 target 0 lun 0
Mar 3 10:37:51 freenas kernel: ada4: <ST3000DM001-1CH166 CC43> ATA-8 SATA 3.x device
Mar 3 10:37:51 freenas kernel: ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Mar 3 10:37:51 freenas kernel: ada4: Command Queueing enabled
Mar 3 10:37:51 freenas kernel: ada4: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
Mar 3 10:37:51 freenas kernel: ada5 at ahcich7 bus 0 scbus9 target 0 lun 0
Mar 3 10:37:51 freenas kernel: ada5: <Hitachi HDS723030ALA640 MKAOA390> ATA-8 SATA 3.x device
Mar 3 10:37:51 freenas kernel: ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Mar 3 10:37:51 freenas kernel: ada5: Command Queueing enabled
Mar 3 10:37:51 freenas kernel: ada5: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
Mar 3 10:37:51 freenas kernel: da0 at umass-sim0 bus 0 scbus10 target 0 lun 0
Mar 3 10:37:51 freenas kernel: da0: <SanDisk Cruzer Fit 1.26> Removable Direct Access SCSI-5 device
Mar 3 10:37:51 freenas kernel: da0: 40.000MB/s transfers
Mar 3 10:37:51 freenas kernel: da0: 3819MB (7821312 512 byte sectors: 255H 63S/T 486C)
Mar 3 10:37:51 freenas kernel: SMP: AP CPU #1 Launched!
Mar 3 10:37:51 freenas kernel: GEOM: da0s1: geometry does not match label (16h,63s != 255h,63s).
Mar 3 10:37:51 freenas kernel: Trying to mount root from ufs:/dev/ufs/FreeNASs1a
Mar 3 10:37:51 freenas kernel: WARNING: /data was not properly dismounted
Mar 3 10:37:51 freenas kernel: ZFS filesystem version 5
Mar 3 10:37:51 freenas kernel: ZFS storage pool version 28
Mar 3 10:37:51 freenas root: /etc/rc: WARNING: failed precmd routine for vmware_guestd
Mar 3 10:37:51 freenas ntpd[2046]: ntpd 4.2.4p5-a (1)
Mar 3 10:37:57 freenas avahi-daemon[2500]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
Mar 3 10:38:01 freenas ntpd[2047]: time reset +0.549012 s
Mar 3 10:53:12 freenas ntpd[2047]: kernel time sync status change 2001
Mar 3 18:08:04 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 3 18:08:05 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 3 23:27:03 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 3 23:27:04 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 02:23:19 freenas upsd[2114]: Data for UPS [ups] is stale - check driver
Mar 4 02:23:19 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 02:23:20 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 02:23:22 freenas upsd[2114]: UPS [ups] data is no longer stale
Mar 4 06:01:32 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 06:01:33 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 08:46:38 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 08:46:39 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 14:21:45 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 14:21:46 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 14:48:10 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 14:48:11 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 17:51:34 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 17:51:35 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 20:31:10 freenas upsd[2114]: Data for UPS [ups] is stale - check driver
Mar 4 20:31:11 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 20:31:12 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 20:31:13 freenas upsd[2114]: UPS [ups] data is no longer stale

 

[William Grzybowski]

What FreeBSD devil guy? Do you have screensaver enabled?

 

[nvader]

Yes I did have screen saver enabled. I will turn it off in hopes that I can see whatever last console message maybe on the screen.
Should I be looking for a way to enable crash dumps?

 

[William Grzybowski]

No, just disabling screensaver should be enough to see it.

 

[nvader]

Had another hard freeze tonight.. nothing on the console screen other than this msg (with today's[SUP][/SUP] date of course)
What now?

Mar 4 06:01:32 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 06:01:33 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 08:46:38 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 08:46:39 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 14:21:45 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 14:21:46 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 14:48:10 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 14:48:11 freenas kernel: ugen2.2: <CPS> at usbus2
Mar 4 17:51:34 freenas kernel: ugen2.2: <CPS> at usbus2 (disconnected)
Mar 4 17:51:35 freenas kernel: ugen2.2: <CPS> at usbus2

 

[cyberjock]

That looks like some USB device is misbehaving. It may be your boot drive, it may not. Note that some onboard devices are actually USB devices so you may not recognize it.

 

[nvader]

That is actually my UPS.
It constantly does that.
I have tested it's functionality and it works (as far as pulling the power and freebsd shutting down like it's supposed to)
but that console message does persist.

Not to be disagreeable. but if it does that every day all day for weeks, do you think that could be the problem? doesn't seem likely to me. (else it would freeze during the day)

I'm hopeful 8.3.1 final will be released soon and I'll give that a go. and see if maybe that solves it..
thx for your replies.

 

[William Grzybowski]

Hard freeze like that sounds like hardware issue, maybe power supply.

If it is freezing at dawn it is when the system is under load (zpool scrub, maintenance tasks, etc)

 

[joeschmuck]

My first question is: How long has the system been running and has it ever just run without the failure you're experiencing? You really need to give us the entire picture so we could help you out or you will end up never getting good help. Say you are a mechanic and I called you up and said my car won't start and that was it. Don't you think you would want a bit more information to help diagnose the problem. It could be the idiot never put the keys in the car or it's out of gas.

I wouldn't be expecting 8.3.1 to magically fix anything. It might but you shouldn't have this problem with the current release.

The error you have is something I've seen before but I doubt it's causing the problem. If you look at the timestamps it's really not that often and it lasts for 1 second. You may need to try a different driver to get rid of it completely or live with it.

Typically when the system crashes it will leave some data on the console screen but in your case it doesn't appear to be doing that.

So you didn't finish specifying your system, one important thing is your hard drive arrangement. Looks like you have six hard drives and one flash drive. How are your hard drives arranged (RAID?) I know my system recently was hanging periodically when I added a 2TB UFS drive. I removed it and the system hasn't crashed yet, time will tell if that was the cause or not.

Also, you might want to do a scrub on your pool since it's crashed often just to ensure your data is in-tact.

Things to look at are:
1) Always check the console for an error message when the system freezes and if you cannot write down the information, take a photo.
2) Is there any consistency to when the failure occurs, like only in the evenings or only during a backup event?
3) If you think it might be event driven, try to recreate the event you suspect, like a backup for example.
4) Have you made any changes since the system was last working fine? This could be making a BIOS change to adding RAM or network card, maybe a FreeNAS setting change.

 

[nvader]

The system did freeze before I added the UPS. It appears to happen at random, but it's far apart.. i.e. it happens every 2 weeks or so it seems. but doesn't seem consistent or event driven.
As for my hard drive set up
I'm running 5x 3TB drives (seagate) in a ZFS Raid Z1 set up. I have 6th 3tb drive (hitachi) set up as a hot spare.
I'm leaning towards hardware myself. I recently picked up a power supply tester from newegg, so i'll give it a go and see what i can find out.

What would you suggest for mem/cpu testing?

Thanks

 

[joeschmuck]

I agree, it's likely hardware.

CPU Testing: Prime 95 but do not run it much longer than 8 hours as it will put a hurtin' on your CPU. Read the docs with it as you want to run all the threads/cores at the same time. Do a search on Prime 95 and you will find forums where people tell you how to set it up. And do not just let this run unsupervised, check in periodically and this is a serious stress test on your CPU.

RAM Testing: Memtest86+ and run it for three complete passes. I usually run this on overnight.

One thing you will hear on this forum a lot is to ensure your BIOS is up to date. And I preach that you need to have a quality power supply and do not skimp on a unit. Little power glitches will screw with you and you may never find them.

So let's say your RAM test fails, well what can you do about it? One thing is to see if there is a lower speed supported by the RAM and drop it. So I have 1600 RAM but it was unreliable in my NAS so I dropped it to 1333 speed. I could have tweaked on the voltage to make it work but honestly I was not gaining much in performance, I'm quite limited by my 1Gb/sec network connection. If your CPU locks up, drop that speed a little as well and retest. As I said, there are forums on testing your equipment, mostly for overclocking but the information is there, you should use it.

Good Luck

 

[nvader]

I just wanted to follow up on this. when the MB/RAM etc was in another system as my HTPC, i would get random reboots. I ran a mem tester and it seemed it was failing on my ram.
the bios settings where set to "auto" so i looked up the specs /speed on the ram and set it to that specifically. no more reboots. when i upgraded the HPTC hardware and put the older stuff in my server to be new NAS, i didn't change the bios settings, because they were set to spec. as a result of the random freezes. I just set everything back to auto and have had no more freezes. my uptime is over 2 months without issue... i have no idea why it works one way in 1 setup but not the other. . but if it's working i'm happy.

 

[joeschmuck]

Good to know and thanks for following up on that. If you haven't run Memtest, I still recommend you do run it for 3 complete passes. It can run from a bootable CD so it doesn't mess with your system software at all.

-Cheers