Jail Processes Locking Up

Status
Not open for further replies.

thavinci

Cadet
Joined
Oct 10, 2014
Messages
4
I have a raidz2 pool consisting of 6 Drives.

On one of the datasets on this raidz2 i have a standard jail with a few applications installed.

Recently processes within this jail hangs and does not respond to kill -9
The only resolution is to reboot the host which also doesn't succeed. Only hard reboot will work.

Even a ls sometimes hangs and cannot be killed.

Im using version 9.2.1.7...

Also it seems these processes are in state "ZFS"


Code:
procstat -kka|grep zfs
8465 101116 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
21340 100739 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
21340 101059 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0x62b vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 vputx+0x201 zfs_rename_unlock+0x55 zfs_freebsd_rename+0x6fb VOP_RENAME_APV+0x84 kern_renameat+0x4cd amd64_syscall+0x5ea Xfast_syscall+0xf7
21745 101112 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
21994 101647 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
22291 101649 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
22416 101086 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
22508 100165 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
22559 101345 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
22580 101111 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
22594 101107 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
23840 101343 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
23978 101658 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
24035 101302 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
24040 101641 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
24068 101652 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
28706 101642 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
28842 100357 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
28902 101646 sawmill          -                mi_switch+0x194 sleepq_wait+0x42 __lockmgr_args+0xea0 vop_stdlock+0x39 VOP_LOCK1_APV+0x84 _vn_lock+0x47 zfs_lookup+0x335 zfs_freebsd_lookup+0x75 VOP_CACHEDLOOKUP_APV+0x6e vfs_cache_lookup+0xf8 VOP_LOOKUP_APV+0x6e lookup+0x3f8 namei+0x4f7 kern_statat_vnhook+0xb6 kern_statat+0x15 sys_stat+0x2a amd64_syscall+0x5ea Xfast_syscall+0xf7
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
Can you post the "debug" file from your server and your hardware models and such?
 

thavinci

Cadet
Joined
Oct 10, 2014
Messages
4
Can you post the "debug" file from your server and your hardware models and such?

Ok I upgraded from FreeNAS-9.2.1.7-RELEASE-x64 to FreeNAS-9.2.1.8-RELEASE-x64 hoping would fix issue, hasn't.
System is
Manufacturer: Supermicro
Product Name: X9DRD-7LN4F(-JBOD)/X9DRD-EF
 

Attachments

  • debug-antacid-20141014210535.tgz
    382.6 KB · Views: 259

thavinci

Cadet
Joined
Oct 10, 2014
Messages
4
I would like to append this with more info about my system:

Code:
hostname kernel log messages:
> Copyright (c) 1992-2013 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>     The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 9.2-RELEASE-p12 #0 r262572+b043649: Sun Sep 28 23:03:31 PDT 2014
>   
> root@build3.ixsystems.com:/tank/home/jkh/build/921/freenas/os-base/amd
> 64/fusion/jkh/921/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64 gcc
> version 4.2.1 20070831 patched [FreeBSD]
> CPU: Intel(R) Xeon(R) CPU E5-2609 v2 @ 2.50GHz (2500.06-MHz K8-class CPU)
>   Origin = "GenuineIntel"  Id = 0x306e4  Family = 0x6  Model = 0x3e  Stepping = 4
>   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>   Features2=0x7fbee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
>   AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
>   AMD Features2=0x1<LAHF>
>   Standard Extended Features=0x281<GSFSBASE,SMEP,ENHMOVSB>
>   TSC: P-state invariant, performance statistics real memory  =
> 17179869184 (16384 MB) avail memory = 16448192512 (15686 MB) Event
> timer "LAPIC" quality 600 ACPI APIC Table: < >
> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
> FreeBSD/SMP: 1 package(s) x 4 core(s)
>  cpu0 (BSP): APIC ID:  0
>  cpu1 (AP): APIC ID:  2
>  cpu2 (AP): APIC ID:  4
>  cpu3 (AP): APIC ID:  6
> WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> ioapic1 <Version 2.0> irqs 24-47 on motherboard
> kbd1 at kbdmux0
> cryptosoft0: <software crypto> on motherboard
> aesni0: <AES-CBC,AES-XTS> on motherboard
> padlock0: No ACE support.
> acpi0: <SUPERM SMCI--MB> on motherboard
> acpi0: Power Button (fixed)
> acpi0: reservation of 0, a0000 (3) failed
> cpu0: <ACPI CPU> on acpi0
> cpu1: <ACPI CPU> on acpi0
> cpu2: <ACPI CPU> on acpi0
> cpu3: <ACPI CPU> on acpi0
> attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254"
> frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182
> Hz quality 100
> atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 Event timer
> "RTC" frequency 32768 Hz quality 0
> hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on
> acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer
> "HPET" frequency 14318180 Hz quality 550 Event timer "HPET1" frequency
> 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz
> quality 440 Event timer "HPET3" frequency 14318180 Hz quality 440
> Event timer "HPET4" frequency 14318180 Hz quality 440 Timecounter
> "ACPI-fast" frequency 3579545 Hz quality 900
> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> pcib1: <ACPI PCI-PCI bridge> irq 26 at device 1.0 on pci0
> pci1: <ACPI PCI bus> on pcib1
> isp0: <Qlogic ISP 2432 PCI FC-AL Adapter> port 0xe000-0xe0ff mem
> 0xfbd40000-0xfbd43fff irq 26 at device 0.0 on pci1
> isp0: setting role to 0x2
> pcib2: <ACPI PCI-PCI bridge> irq 32 at device 2.0 on pci0
> pci2: <ACPI PCI bus> on pcib2
> isp1: <Qlogic ISP 2432 PCI FC-AL Adapter> port 0xd000-0xd0ff mem
> 0xfbc40000-0xfbc43fff irq 32 at device 0.0 on pci2
> isp1: setting role to 0x2
> pcib3: <ACPI PCI-PCI bridge> irq 32 at device 2.2 on pci0
> pci3: <ACPI PCI bus> on pcib3
> mps0: <LSI SAS2308> port 0xc000-0xc0ff mem
> 0xfba40000-0xfba4ffff,0xfba00000-0xfba3ffff irq 34 at device 0.0 on
> pci3
> mps0: Firmware: 19.00.00.00, Driver: 16.00.00.00-fbsd
> mps0: IOCCapabilities:
> 5285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostD
> isc>
> pcib4: <ACPI PCI-PCI bridge> irq 40 at device 3.0 on pci0
> pci4: <ACPI PCI bus> on pcib4
> pcib5: <ACPI PCI-PCI bridge> irq 40 at device 3.2 on pci0
> pci5: <ACPI PCI bus> on pcib5
> igb0: <Intel(R) PRO/1000 Network Connection version - 2.4.0> port
> 0xb060-0xb07f mem 0xfbb60000-0xfbb7ffff,0xfbb8c000-0xfbb8ffff irq 42
> at device 0.0 on pci5
> igb0: Using MSIX interrupts with 5 vectors
> igb0: Ethernet address: 0c:c4:7a:12:7f:64
> igb0: Bound queue 0 to cpu 0
> igb0: Bound queue 1 to cpu 1
> igb0: Bound queue 2 to cpu 2
> igb0: Bound queue 3 to cpu 3
> igb1: <Intel(R) PRO/1000 Network Connection version - 2.4.0> port
> 0xb040-0xb05f mem 0xfbb40000-0xfbb5ffff,0xfbb88000-0xfbb8bfff irq 45
> at device 0.1 on pci5
> igb1: Using MSIX interrupts with 5 vectors
> igb1: Ethernet address: 0c:c4:7a:12:7f:65
> igb1: Bound queue 0 to cpu 0
> igb1: Bound queue 1 to cpu 1
> igb1: Bound queue 2 to cpu 2
> igb1: Bound queue 3 to cpu 3
> igb2: <Intel(R) PRO/1000 Network Connection version - 2.4.0> port
> 0xb020-0xb03f mem 0xfbb20000-0xfbb3ffff,0xfbb84000-0xfbb87fff irq 44
> at device 0.2 on pci5
> igb2: Using MSIX interrupts with 5 vectors
> igb2: Ethernet address: 0c:c4:7a:12:7f:66
> igb2: Bound queue 0 to cpu 0
> igb2: Bound queue 1 to cpu 1
> igb2: Bound queue 2 to cpu 2
> igb2: Bound queue 3 to cpu 3
> igb3: <Intel(R) PRO/1000 Network Connection version - 2.4.0> port
> 0xb000-0xb01f mem 0xfbb00000-0xfbb1ffff,0xfbb80000-0xfbb83fff irq 46
> at device 0.3 on pci5
> igb3: Using MSIX interrupts with 5 vectors
> igb3: Ethernet address: 0c:c4:7a:12:7f:67
> igb3: Bound queue 0 to cpu 0
> igb3: Bound queue 1 to cpu 1
> igb3: Bound queue 2 to cpu 2
> igb3: Bound queue 3 to cpu 3
> pci0: <base peripheral> at device 4.0 (no driver attached)
> pci0: <base peripheral> at device 4.1 (no driver attached)
> pci0: <base peripheral> at device 4.2 (no driver attached)
> pci0: <base peripheral> at device 4.3 (no driver attached)
> pci0: <base peripheral> at device 4.4 (no driver attached)
> pci0: <base peripheral> at device 4.5 (no driver attached)
> pci0: <base peripheral> at device 4.6 (no driver attached)
> pci0: <base peripheral> at device 4.7 (no driver attached)
> pci0: <base peripheral> at device 5.0 (no driver attached)
> pci0: <base peripheral> at device 5.2 (no driver attached)
> pcib6: <ACPI PCI-PCI bridge> irq 16 at device 17.0 on pci0
> pci7: <ACPI PCI bus> on pcib6
> pci0: <simple comms> at device 22.0 (no driver attached)
> pci0: <simple comms> at device 22.1 (no driver attached)
> ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfbe23000-0xfbe233ff
> irq 16 at device 26.0 on pci0
> usbus0: EHCI version 1.0
> usbus0 on ehci0
> ehci1: <EHCI (generic) USB 2.0 controller> mem 0xfbe22000-0xfbe223ff
> irq 23 at device 29.0 on pci0
> usbus1: EHCI version 1.0
> usbus1 on ehci1
> pcib7: <ACPI PCI-PCI bridge> at device 30.0 on pci0
> pci8: <ACPI PCI bus> on pcib7
> vgapci0: <VGA-compatible display> mem
> 0xf9000000-0xf9ffffff,0xfb800000-0xfb803fff,0xfb000000-0xfb7fffff irq
> 22 at device 1.0 on pci8
> isab0: <PCI-ISA bridge> at device 31.0 on pci0
> isa0: <ISA bus> on isab0
> ahci0: <Intel Patsburg AHCI SATA controller> port
> 0xf050-0xf057,0xf040-0xf043,0xf030-0xf037,0xf020-0xf023,0xf000-0xf01f
> mem 0xfbe21000-0xfbe217ff irq 18 at device 31.2 on pci0
> ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
> ahcich0: <AHCI channel> at channel 0 on ahci0
> ahcich1: <AHCI channel> at channel 1 on ahci0
> ahcich2: <AHCI channel> at channel 2 on ahci0
> ahcich3: <AHCI channel> at channel 3 on ahci0
> ahcich4: <AHCI channel> at channel 4 on ahci0
> ahcich5: <AHCI channel> at channel 5 on ahci0
> pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
> pci0: <dasp> at device 31.6 (no driver attached)
> pcib8: <ACPI Host-PCI bridge> on acpi0
> pci255: <ACPI PCI bus> on pcib8
> pci255: <base peripheral> at device 8.0 (no driver attached)
> pci255: <base peripheral> at device 9.0 (no driver attached)
> pci255: <base peripheral> at device 10.0 (no driver attached)
> pci255: <base peripheral> at device 10.1 (no driver attached)
> pci255: <base peripheral> at device 10.2 (no driver attached)
> pci255: <base peripheral> at device 10.3 (no driver attached)
> pci255: <base peripheral> at device 11.0 (no driver attached)
> pci255: <base peripheral> at device 11.3 (no driver attached)
> pci255: <base peripheral> at device 12.0 (no driver attached)
> pci255: <base peripheral> at device 12.1 (no driver attached)
> pci255: <base peripheral> at device 13.0 (no driver attached)
> pci255: <base peripheral> at device 13.1 (no driver attached)
> pci255: <base peripheral> at device 14.0 (no driver attached)
> pci255: <dasp> at device 14.1 (no driver attached)
> pci255: <base peripheral> at device 15.0 (no driver attached)
> pci255: <base peripheral> at device 15.1 (no driver attached)
> pci255: <base peripheral> at device 15.2 (no driver attached)
> pci255: <base peripheral> at device 15.3 (no driver attached)
> pci255: <base peripheral> at device 15.4 (no driver attached)
> pci255: <base peripheral> at device 15.5 (no driver attached)
> pci255: <base peripheral> at device 16.0 (no driver attached)
> pci255: <base peripheral> at device 16.1 (no driver attached)
> pci255: <base peripheral> at device 16.2 (no driver attached)
> pci255: <base peripheral> at device 16.3 (no driver attached)
> pci255: <base peripheral> at device 16.4 (no driver attached)
> pci255: <base peripheral> at device 16.5 (no driver attached)
> pci255: <base peripheral> at device 16.6 (no driver attached)
> pci255: <base peripheral> at device 16.7 (no driver attached)
> pci255: <base peripheral> at device 19.0 (no driver attached)
> pci255: <dasp> at device 19.1 (no driver attached)
> pci255: <base peripheral> at device 19.4 (no driver attached)
> pci255: <dasp> at device 19.5 (no driver attached)
> pci255: <base peripheral> at device 22.0 (no driver attached)
> pci255: <base peripheral> at device 22.1 (no driver attached)
> pci255: <base peripheral> at device 22.2 (no driver attached)
> acpi_button0: <Power Button> on acpi0
> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on
> acpi0
> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
> ichwd0 on isa0
> ichwd0: ICH WDT present but disabled in BIOS or hardware
> device_attach: ichwd0 attach returned 6
> wbwd0: <Winbond 83627DHG-P> at port 0x2e-0x2f on isa0
> ichwd0 at port 0x430-0x437,0x460-0x47f on isa0
> ichwd0: ICH WDT present but disabled in BIOS or hardware
> device_attach: ichwd0 attach returned 6
> orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xce800-0xcf7ff on
> isa0
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on
> isa0
> coretemp0: <CPU On-Die Thermal Sensors> on cpu0
> est0: <Enhanced SpeedStep Frequency Control> on cpu0
> p4tcc0: <CPU Frequency Thermal Control> on cpu0
> coretemp1: <CPU On-Die Thermal Sensors> on cpu1
> est1: <Enhanced SpeedStep Frequency Control> on cpu1
> p4tcc1: <CPU Frequency Thermal Control> on cpu1
> coretemp2: <CPU On-Die Thermal Sensors> on cpu2
> est2: <Enhanced SpeedStep Frequency Control> on cpu2
> p4tcc2: <CPU Frequency Thermal Control> on cpu2
> coretemp3: <CPU On-Die Thermal Sensors> on cpu3
> est3: <Enhanced SpeedStep Frequency Control> on cpu3
> p4tcc3: <CPU Frequency Thermal Control> on cpu3 Timecounters tick
> every 1.000 msec
> ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to
> accept, logging disabled DUMMYNET 0xfffffe0003efc000 with IPv6
> initialized (100409) load_dn_sched dn_sched RR loaded load_dn_sched
> dn_sched WF2Q+ loaded load_dn_sched dn_sched FIFO loaded load_dn_sched
> dn_sched PRIO loaded load_dn_sched dn_sched QFQ loaded
> usbus0: 480Mbps High Speed USB v2.0
> usbus1: 480Mbps High Speed USB v2.0
> ugen0.1: <Intel> at usbus0
> uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on
> usbus0
> ugen1.1: <Intel> at usbus1
> uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on
> usbus1
> uhub0: 2 ports with 2 removable, self powered
> uhub1: 2 ports with 2 removable, self powered
> ugen0.2: <vendor 0x8087> at usbus0
> uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr
> 2> on usbus0
> ugen1.2: <vendor 0x8087> at usbus1
> uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr
> 2> on usbus1
> uhub2: 6 ports with 6 removable, self powered
> uhub3: 8 ports with 8 removable, self powered
> ugen0.3: <Winbond Electronics Corp> at usbus0
> ums0: <Winbond Electronics Corp Hermon USB hidmouse Device, class 0/0,
> rev 1.10/0.01, addr 3> on usbus0
> ums0: 3 buttons and [Z] coordinates ID=0
> ukbd0: <Winbond Electronics Corp Hermon USB hidmouse Device, class
> 0/0, rev 1.10/0.01, addr 3> on usbus0
> kbd0 at ukbd0
> ugen1.3: <Kingston> at usbus1
> umass0: <Kingston DataTraveler 3.0, class 0/0, rev 2.10/1.00, addr 3>
> on usbus1
> umass0:  SCSI over Bulk-Only; quirks = 0x0100
> umass0:10:0:-1: Attached to scbus10
> ses0 at mps0 bus 0 scbus2 target 18 lun 0
> ses0: <LSI SAS2X36 0e12> Fixed Enclosure Services SCSI-5 device
> ses0: Serial Number
> ses0: 600.000MB/s transfers
> ses0: Command Queueing enabled
> ses0: SCSI-3 ENC Device
> da0 at mps0 bus 0 scbus2 target 8 lun 0
> da0: <ATA ST3000VN000-1H41 SC44> Fixed Direct Access SCSI-6 device
> da0: Serial Number             S300HJDT
> da0: 600.000MB/s transfers
> da0: Command Queueing enabled
> da0: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
> da1 at mps0 bus 0 scbus2 target 9 lun 0
> da1: <ATA ST3000VN000-1H41 SC44> Fixed Direct Access SCSI-6 device
> da1: Serial Number             S300HSX3
> da1: 600.000MB/s transfers
> da1: Command Queueing enabled
> da1: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
> da2 at mps0 bus 0 scbus2 target 10 lun 0
> da2: <ATA ST3000VN000-1H41 SC44> Fixed Direct Access SCSI-6 device
> da2: Serial Number             S300JVQZ
> da2: 600.000MB/s transfers
> da2: Command Queueing enabled
> da2: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
> da3 at mps0 bus 0 scbus2 target 11 lun 0
> da3: <ATA ST3000VN000-1H41 SC44> Fixed Direct Access SCSI-6 device
> da3: Serial Number             S300GQMR
> da3: 600.000MB/s transfers
> da3: Command Queueing enabled
> da3: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
> da4 at mps0 bus 0 scbus2 target 12 lun 0
> da4: <ATA ST3000VN000-1H41 SC44> Fixed Direct Access SCSI-6 device
> da4: Serial Number             S300HT27
> da4: 600.000MB/s transfers
> da4: Command Queueing enabled
> da4: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
> da5 at mps0 bus 0 scbus2 target 13 lun 0
> da5: <ATA ST3000VN000-1H41 SC44> Fixed Direct Access SCSI-6 device
> da5: Serial Number             S300HSY9
> da5: 600.000MB/s transfers
> da5: Command Queueing enabled
> da5: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
> da6 at mps0 bus 0 scbus2 target 14 lun 0
> da6: <ATA ST1000NC000-1CX1 CE02> Fixed Direct Access SCSI-6 device
> da6: Serial Number             Z1D8MB8R
> da6: 600.000MB/s transfers
> da6: Command Queueing enabled
> da6: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
> da7 at mps0 bus 0 scbus2 target 15 lun 0
> da7: <SEAGATE ST3146855SS 0002> Fixed Direct Access SCSI-5 device
> da7: Serial Number 3LN0K2A4000097389SNZ
> da7: 300.000MB/s transfers
> da7: Command Queueing enabled
> da7: 140014MB (286749488 512 byte sectors: 255H 63S/T 17849C)
> da10 at umass-sim0 bus 0 scbus10 target 0 lun 0
> da10: <Kingston DataTraveler 3.0 PMAP> Removable Direct Access SCSI-6
> device
> da10: Serial Number 08606E6D414CBE50A70C0A46
> da10: 40.000MB/s transfers
> da10: 30000MB (61440000 512 byte sectors: 255H 63S/T 3824C)
> da10: quirks=0x2<NO_6_BYTE>
> da8 at mps0 bus 0 scbus2 target 16 lun 0
> da8: <ATA ST3000NM0033-9ZM SN03> Fixed Direct Access SCSI-6 device
> da8: Serial Number             Z1Y19779
> da8: 600.000MB/s transfers
> da8: Command Queueing enabled
> da8: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
> da9 at mps0 bus 0 scbus2 target 17 lun 0
> da9: <ATA ST3000NM0033-9ZM SN03> Fixed Direct Access SCSI-6 device
> da9: Serial Number             Z1Y19VL1
> da9: 600.000MB/s transfers
> da9: Command Queueing enabled
> da9: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
> SMP: AP CPU #2 Launched!
> SMP: AP CPU #1 Launched!
> SMP: AP CPU #3 Launched!
> Timecounter "TSC-low" frequency 1250029312 Hz quality 1000 Trying to
> mount root from ufs:/dev/ufs/FreeNASs2a [ro]...
> WARNING: /data was not properly dismounted
> GEOM_RAID5: Module loaded, version 1.1.20130907.44 (rev 5c6d2a159411)
> ZFS filesystem version: 5 ZFS storage pool version: features support
> (5000)
> GEOM_ELI: Device da0p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> GEOM_ELI: Device da1p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> GEOM_ELI: Device da2p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> GEOM_ELI: Device da4p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> GEOM_ELI: Device da3p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> GEOM_ELI: Device da5p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> GEOM_ELI: Device da8p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> GEOM_ELI: Device da7p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> GEOM_ELI: Device da9p1.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> vboxdrv: fAsync=0 offMin=0x35b offMax=0x107a
> bridge0: Ethernet address: 02:43:b5:9d:4d:00
> igb0: promiscuous mode enabled
> bridge0: link state changed to UP
> epair0a: Ethernet address: 02:d6:a9:00:0a:0a
> epair0b: Ethernet address: 02:d6:a9:00:0b:0b
> epair0a: link state changed to UP
> epair0b: link state changed to UP
> igb0: link state changed to DOWN
> epair0a: promiscuous mode enabled
> igb0: link state changed to UP
> ng_ether_ifnet_arrival_event: can't re-name node epair0b
> epair1a: Ethernet address: 02:dc:86:00:0b:0a
> epair1b: Ethernet address: 02:dc:86:00:0c:0b
> epair1a: link state changed to UP
> epair1b: link state changed to UP
> epair1a: promiscuous mode enabled
> ng_ether_ifnet_arrival_event: can't re-name node epair1b


I have been forced to move the Directory that contained the sawmill Database's to a UFS volume dedicated to avoid these lockups as it's a live system.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
I don't have an answer. Moving to UFS seems to be a step backwards to me. I'd take the output from your initial post and the debug file you posted and create a ticket at bugs.freenas.org. Then a developer might be able to tell you what is going on.

I'm not sure what your jail is doing, but is it possible its using lots of RAM and 16GB of RAM isn't going to be enough? You said "database" which makes me think mysql or equivalent, and those can be RAM hungry with small databases.
 

thavinci

Cadet
Joined
Oct 10, 2014
Messages
4
Unfortunately UFS is a huge step up for us as it gaurentees server availability.
The database in question is not mysql or any other standard database rather a flat file that the log processing tool sawmill utilizes so resources required is minimal.

I will have a look at logging it, thank you....
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
Honestly, I'd argue that UFS isn't a huge step and doesn't guarantee server availability.

With large UFS setups, on a loss of power or other unclean dismounting of UFS a fsck is required. This can take hours (or days) depending on your setup. There is no alternative except to let it take its course.
 
Status
Not open for further replies.
Top