Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.3-STABLE #0 41eb257(9.10.2-STABLE): Mon Mar 6 17:05:05 UTC 2017 root@gauntlet:/freenas-9.10-releng/_BE/objs/freenas-9.10-releng/_BE/os/sys/FreeNAS.amd64-DEBUG amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 WARNING: WITNESS option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU E5-2609 v3 @ 1.90GHz (1900.04-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x37ab XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 36507222016 (34816 MB) avail memory = 33123807232 (31589 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 6 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 cpu4 (AP): APIC ID: 8 cpu5 (AP): APIC ID: 10 random: initialized WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard module_register_init: MOD_LOAD (vesa, 0xffffffff80eaf250, 0) error 19 kbd1 at kbdmux0 aesni0: on motherboard padlock0: No ACE support. cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 atrtc0: port 0x70-0x71,0x74-0x77 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 550 Event timer "HPET1" frequency 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz quality 440 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: on acpi0 pci255: on pcib0 pci255: at device 11.1 (no driver attached) pci255: at device 11.2 (no driver attached) pci255: at device 16.1 (no driver attached) pci255: at device 16.6 (no driver attached) pci255: at device 18.1 (no driver attached) pci255: at device 18.5 (no driver attached) pcib1: port 0xcf8-0xcff on acpi0 pci0: on pcib1 pcib2: irq 26 at device 1.0 on pci0 pci1: on pcib2 pcib3: irq 32 at device 2.0 on pci0 pci2: on pcib3 pcib4: irq 32 at device 2.2 on pci0 pci3: on pcib4 mpr0: port 0xe000-0xe0ff mem 0xfb240000-0xfb24ffff,0xfb200000-0xfb23ffff irq 34 at device 0.0 on pci3 mpr0: IOCFacts : MsgVersion: 0x205 HeaderVersion: 0x2a00 IOCNumber: 0 IOCExceptions: 0x0 MaxChainDepth: 128 NumberOfPorts: 1 RequestCredit: 3072 ProductID: 0x2221 IOCRequestFrameSize: 32 MaxInitiators: 1 MaxTargets: 256 MaxSasExpanders: 48 MaxEnclosures: 16 HighPriorityCredit: 128 MaxReplyDescriptorPostQueueDepth: 65504 ReplyFrameSize: 32 MaxVolumes: 0 MaxDevHandle: 313 MaxPersistentEntries: 128 mpr0: Firmware: 12.00.02.00, Driver: 13.01.00.00-fbsd mpr0: IOCCapabilities: 7a85c pcib5: irq 40 at device 3.0 on pci0 pci4: on pcib5 em0: port 0xd020-0xd03f mem 0xfb3a0000-0xfb3bffff,0xfb380000-0xfb39ffff irq 40 at device 0.0 on pci4 em0: Using an MSI interrupt em0: Ethernet address: 00:15:17:1e:15:44 em1: port 0xd000-0xd01f mem 0xfb340000-0xfb35ffff,0xfb320000-0xfb33ffff irq 44 at device 0.1 on pci4 em1: Using an MSI interrupt em1: Ethernet address: 00:15:17:1e:15:45 pcib6: irq 40 at device 3.2 on pci0 pci5: on pcib6 ix0: port 0xc020-0xc03f mem 0xfbc00000-0xfbdfffff,0xfbe04000-0xfbe07fff irq 42 at device 0.0 on pci5 ix0: Using MSIX interrupts with 7 vectors ix0: Ethernet address: 0c:c4:7a:de:8d:32 ix0: PCI Express Bus: Speed 5.0GT/s Width x8 ix1: port 0xc000-0xc01f mem 0xfba00000-0xfbbfffff,0xfbe00000-0xfbe03fff irq 45 at device 0.1 on pci5 ix1: Using MSIX interrupts with 7 vectors ix1: Ethernet address: 0c:c4:7a:de:8d:33 ix1: PCI Express Bus: Speed 5.0GT/s Width x8 pci0: at device 17.0 (no driver attached) ahci0: port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb438000-0xfb4387ff irq 16 at device 17.4 on pci0 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 xhci0: mem 0xfb400000-0xfb40ffff irq 19 at device 20.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) ehci0: mem 0xfb434000-0xfb4343ff irq 18 at device 26.0 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci0 pcib7: irq 16 at device 28.0 on pci0 pci6: on pcib7 pcib8: irq 16 at device 28.4 on pci0 pci7: on pcib8 pcib9: at device 0.0 on pci7 pci8: on pcib9 vgapci0: port 0xb000-0xb07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 16 at device 0.0 on pci8 vgapci0: Boot video device ehci1: mem 0xfb433000-0xfb4333ff irq 18 at device 29.0 on pci0 usbus2: EHCI version 1.0 usbus2 on ehci1 isab0: at device 31.0 on pci0 isa0: on isab0 ahci1: port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xfb432000-0xfb4327ff irq 16 at device 31.2 on pci0 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich5: at channel 1 on ahci1 ahcich6: at channel 2 on ahci1 ahcich7: at channel 3 on ahci1 ahcich8: at channel 4 on ahci1 ahcich9: at channel 5 on ahci1 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 ichwd0: on isa0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc87ff,0xd1000-0xd1fff on isa0 sc0: at flags 0x100 on isa0 sc0: CGA <16 virtual consoles, flags=0x300> vga0: at port 0x3d0-0x3db iomem 0xb8000-0xbffff on isa0 coretemp0: on cpu0 est0: on cpu0 coretemp1: on cpu1 est1: on cpu1 coretemp2: on cpu2 est2: on cpu2 coretemp3: on cpu3 est3: on cpu3 coretemp4: on cpu4 est4: on cpu4 coretemp5: on cpu5 est5: on cpu5 random: unblocking device. usbus0: 5.0Gbps Super Speed USB v3.0 usbus1: 480Mbps High Speed USB v2.0 usbus2: 480Mbps High Speed USB v2.0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled lock order reversal: 1st 0xfffff80008728460 CAM device lock (CAM device lock) @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2441 2nd 0xfffffe0000a441d0 MPR lock (MPR lock) @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2625 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff8215d4b0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffffff8215d560 witness_checkorder() at witness_checkorder+0xe2b/frame 0xffffffff8215d5f0 __mtx_lock_flags() at __mtx_lock_flags+0xa8/frame 0xffffffff8215d640 xpt_action_default() at xpt_action_default+0xc0f/frame 0xffffffff8215d690 xptsetasyncbusfunc() at xptsetasyncbusfunc+0x10b/frame 0xffffffff8215d8c0 xptbustraverse() at xptbustraverse+0xa1/frame 0xffffffff8215d900 xpt_register_async() at xpt_register_async+0x248/frame 0xffffffff8215da40 ctlfeperiphinit() at ctlfeperiphinit+0x19/frame 0xffffffff8215da50 periphdriver_init() at periphdriver_init+0x4e/frame 0xffffffff8215da70 xpt_config() at xpt_config+0xe9/frame 0xffffffff8215daa0 run_interrupt_driven_config_hooks() at run_interrupt_driven_config_hooks+0x7c/frame 0xffffffff8215dac0 boot_run_interrupt_driven_config_hooks() at boot_run_interrupt_driven_config_hooks+0x20/frame 0xffffffff8215db50 mi_startup() at mi_startup+0x108/frame 0xffffffff8215db70 btext() at btext+0x2c ugen1.1: at usbus1 uhub0: on usbus1 ugen0.1: <0x8086> at usbus0 uhub1: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 ugen2.1: at usbus2 uhub2: on usbus2 uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub1: 21 ports with 21 removable, self powered ugen0.2: at usbus0 uhub3: on usbus0 mpr0: SAS Address for SATA device = 371a4445818e7538 ugen1.2: at usbus1 uhub4: on usbus1 ugen2.2: at usbus2 uhub5: on usbus2 mpr0: SAS Address for SATA device = 371a444581907c44 mpr0: SAS Address for SATA device = 371a444581929147 mpr0: SAS Address for SATA device = 371a444581929035 mpr0: SAS Address for SATA device = 371a44458190863a mpr0: SAS Address for SATA device = 371a44458192905a mpr0: SAS Address for SATA device = 371a4445818a843a mpr0: SAS Address for SATA device = 371a444581829034 mpr0: SAS Address for SATA device = 371a444581908538 mpr0: SAS Address for SATA device = 371a44458192915a mpr0: SAS Address for SATA device = 371a444581928f5d uhub3: 4 ports with 3 removable, self powered mpr0: SAS Address for SATA device = 371a44458182914e mpr0: SAS Address from SAS device page0 = 500304801eabe280 mpr0: SAS Address from SATA device = 371a4445818e7538 mpr0: Found device <81,End Device> <12.0Gbps> handle<0x000a> enclosureHandle<0x0002> slot 0 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe281 mpr0: SAS Address from SATA device = 371a444581907c44 mpr0: Found device <81,End Device> <12.0Gbps> handle<0x000b> enclosureHandle<0x0002> slot 1 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe282 mpr0: SAS Address from SATA device = 371a444581929147 mpr0: Found device <81,End Device> <12.0Gbps> handle<0x000c> enclosureHandle<0x0002> slot 2 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe283 mpr0: SAS Address from SATA device = 371a444581929035 mpr0: Found device <81,End Device> <12.0Gbps> handle<0x000d> enclosureHandle<0x0002> slot 3 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe284 mpr0: SAS Address from SATA device = 371a44458190863a mpr0: Found device <81,End Device> <12.0Gbps> handle<0x000e> enclosureHandle<0x0002> slot 4 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe285 mpr0: SAS Address from SATA device = 371a44458192905a mpr0: Found device <81,End Device> <12.0Gbps> handle<0x000f> enclosureHandle<0x0002> slot 5 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe286 mpr0: SAS Address from SATA device = 371a4445818a843a mpr0: Found device <81,End Device> <12.0Gbps> handle<0x0010> enclosureHandle<0x0002> slot 6 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe287 mpr0: SAS Address from SATA device = 371a444581829034 mpr0: Found device <81,End Device> <12.0Gbps> handle<0x0011> enclosureHandle<0x0002> slot 7 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe288 mpr0: SAS Address from SATA device = 371a444581908538 mpr0: Found device <81,End Device> <12.0Gbps> handle<0x0012> enclosureHandle<0x0002> slot 8 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe289 mpr0: SAS Address from SATA device = 371a44458192915a mpr0: Found device <81,End Device> <12.0Gbps> handle<0x0013> enclosureHandle<0x0002> slot 9 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe28a mpr0: SAS Address from SATA device = 371a444581928f5d mpr0: Found device <81,End Device> <12.0Gbps> handle<0x0014> enclosureHandle<0x0002> slot 10 mpr0: At enclosure level 0 and connector name ( ) uhub5: 8 ports with 8 removable, self powered mpr0: SAS Address from SAS device page0 = 500304801eabe28b ugen0.3: at usbus0 ukbd0: on usbus0 kbd0 at ukbd0 mprsas_get_sata_identify: error reading SATA PASSTHRU; iocstatus = 0x804b mpr0: Sleeping 3 seconds after SATA ID error to wait for spinup mpr0: SAS Address from SATA device = 371a44458182914e mpr0: Found device <81,End Device> <12.0Gbps> handle<0x0015> enclosureHandle<0x0002> slot 11 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801eabe2bd mpr0: Found device <4411,End Device> <12.0Gbps> handle<0x0017> enclosureHandle<0x0002> slot 24 mpr0: At enclosure level 0 and connector name ( ) mpr0: SAS Address from SAS device page0 = 500304801f0ebe3d mpr0: Found device <4411,End Device> <12.0Gbps> handle<0x0018> enclosureHandle<0x0003> slot 12 mpr0: At enclosure level 1 and connector name ( ) uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa40 malloc() at malloc+0x192/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 uma_zalloc_arg: zone "64"cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at with the followingxpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fae460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a40 malloc() at malloc+0x192/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da8e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa10 malloc() at malloc+0x192/frame 0xfffffe09f90daa60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 uhub_attach: port 6 power on or off failed, USB_ERR_TIMEOUT probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fae460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d58e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a10 uhub4: 6 ports with 6 removable, self powered KDB: stack backtrace: malloc() at db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0malloc+0x192/frame 0xff9f90da6f0 fffe09f90d5a60 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 sbuf_newbuf() at sbuf_newbuf+0xa6/fkdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampolrame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fae460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 ine() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fae460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fae460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a40 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa40 malloc() at malloc+0x192/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 malloc() at malloc+0x192/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da8e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa10 malloc() at malloc+0x192/frame 0xfffffe09f90daa60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90daa90 sbuf_new() at fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d58e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a10 malloc() at malloc+0x192/frame 0xfffffe09f90d5a60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 sbuf_new+0x7b/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devsbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 ctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59d0 uma_zalloc_arg() at --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a40 malloc() at malloc+0x192/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d58e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a10 malloc() at malloc+0x192/frame 0xfffffe09f90d5a60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5a90 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa40 malloc() at malloc+0x192/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da8e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa10 malloc() at malloc+0x192/frame 0xfffffe09f90daa60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5ac0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64"sbuf_put_byte() at with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa40 malloc() at malloc+0x192/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da8e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa10 malloc() at malloc+0x192/frame 0xfffffe09f90daa60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a40 malloc() at malloc+0x192/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5830 fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d58e0 kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a10 malloc() at malloc+0x192/frame 0xfffffe09f90d5a60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64"xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa40 malloc() at malloc+0x192/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da8e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa10 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- malloc() at uma_zalloc_arg: zone "64"malloc+0x192/frame 0xfffffe09f90daa60 with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a40 malloc() at malloc+0x192/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d58e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a10 malloc() at malloc+0x192/frame 0xfffffe09f90d5a60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the followingmalloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa40 malloc() at malloc+0x192/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da8e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa10 malloc() at malloc+0x192/frame 0xfffffe09f90daa60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a40 malloc() at xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_tmalloc+0x192/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- rampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d58e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a10 malloc() at malloc+0x192/frame 0xfffffe09f90d5a60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa40 malloc() at malloc+0x192/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da8e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da9a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90daa10 malloc() at malloc+0x192/frame 0xfffffe09f90daa60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90daa90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90daac0 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 devctl_notify+0x43/frame 0xfffffe09f90d6000 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_couma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 nf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90da6f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90da7a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90da860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90da8d0 malloc() at malloc+0x192/frame 0xfffffe09f90da920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90da960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90da980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90daa90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90daac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90dab40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90dafd0 devctl_notify() at xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5860 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5910 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59d0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a40 malloc() at malloc+0x192/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 devctl_notify+0x43/frame 0xfffffe09f90db000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90db7c0 probedone() at probedone+0xca9/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5830 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d58e0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d59a0 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5a10 malloc() at malloc+0x192/frame 0xfffffe09f90d5a60 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5a90 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "32" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "128" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5270 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d56f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d57a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5860 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d58d0 malloc() at malloc+0x192/frame 0xfffffe09f90d5920 sbuf_extend() at sbuf_extend+0x72/frame 0xfffffe09f90d5960 sbuf_put_byte() at sbuf_put_byte+0xc9/frame 0xfffffe09f90d5980 kvprintf() at kvprintf+0x101a/frame 0xfffffe09f90d5a90 sbuf_vprintf() at sbuf_vprintf+0x54/frame 0xfffffe09f90d5ac0 g_conf_printf_escaped() at g_conf_printf_escaped+0x67/frame 0xfffffe09f90d5b40 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d5fd0 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6000 cam_periph_error() at cam_periph_error+0x686/frame 0xfffffe09f90d67c0 probedone() at probedone+0xca9/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- ses0 at mpr0 bus 0 scbus0 target 32 lun 0 ses0: Fixed Enclosure Services SPC-3 SCSI device ses0: Serial Number ses0: 1200.000MB/s transfers ses0: Command Queueing enabled ses0: SCSI-3 ENC Device ses1 at mpr0 bus 0 scbus0 target 63 lun 0 ses1: Fixed Enclosure Services SPC-3 SCSI device ses1: Serial Number ses1: 1200.000MB/s transfers ses1: Command Queueing enabled ses1: SCSI-3 ENC Device uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 da0 at mpr0 bus 0 scbus0 target 8 lun 0 da0: Fixed Direct Access SPC-4 SCSI device da0: Serial Number Z840MR25 da0: 1200.000MB/s transfers da0: Command Queueing enabled da0: 7630885MB (15628053168 512 byte sectors) disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5bd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5db0 malloc() at malloc+0x192/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5d80 malloc() at malloc+0x192/frame 0xfffffe09f90d5dd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- da1 at mpr0 bus 0 scbus0 target 9 lun 0 da1: Fixed Direct Access SPC-4 SCSI device da1: Serial Number Z840MT9A da1: 1200.000MB/s transfers da1: Command Queueing enabled devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 da1: 7630885MB (15628053168 512 byte sectors) xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90dabd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dadb0 malloc() at malloc+0x192/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the followingxptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90daba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dad80 malloc() at malloc+0x192/frame 0xfffffe09f90dadd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 da2 at mpr0 bus 0 scbus0 target 10 lun 0 da2: Fixed Direct Access SPC-4 SCSI device da2: Serial Number Z840MVND da2: 1200.000MB/s transfers da2: Command Queueing enabled da2: 7630885MB (15628053168 512 byte sectors) g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5bd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5db0 malloc() at malloc+0x192/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5e30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_lockedevctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5d80 malloc() at malloc+0x192/frame 0xfffffe09f90d5dd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at d() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 da3 at mpr0 bus 0 scbus0 target 11 lun 0 da3: Fixed Direct Access SPC-4 SCSI device da3: Serial Number Z840MVM2 da3: 1200.000MB/s transfers da3: Command Queueing enabled da3: 7630885MB (15628053168 512 byte sectors) disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90dabd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dadb0 malloc() at malloc+0x192/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90daba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dad80 malloc() at malloc+0x192/frame 0xfffffe09f90dadd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffda4 at mpr0 bus 0 scbus0 target 12 lun 0 da4: Fixed Direct Access SPC-4 SCSI device da4: Serial Number Z840MTC7 da4: 1200.000MB/s transfers da4: Command Queueing enabled da4: 7630885MB (15628053168 512 byte sectors) ffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5bd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5db0 malloc() at malloc+0x192/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5d80 malloc() at malloc+0x192/frame 0xfffffe09f90d5dd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f9da5 at mpr0 bus 0 scbus0 target 13 lun 0 da5: Fixed Direct Access SPC-4 SCSI device da5: Serial Number Z840MVMW da5: 1200.000MB/s transfers da5: Command Queueing enabled da5: 7630885MB (15628053168 512 byte sectors) 0fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xffuma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90dabd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dadb0 malloc() at malloc+0x192/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 fffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90daba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dad80 malloc() at malloc+0x192/frame 0xfffffe09f90dadd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 da6 at mpr0 bus 0 scbus0 target 14 lun 0 da6: Fixed Direct Access SPC-4 SCSI device da6: Serial Number Z840MNA7 da6: 1200.000MB/s transfers da6: Command Queueing enabled da6: 7630885MB (15628053168 512 byte sectors) fork_exit() at KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevifork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5bd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5db0 malloc() at malloc+0x192/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at cetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_inidisk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5d80 malloc() at malloc+0x192/frame 0xfffffe09f90d5dd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notifyt() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at +0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 da7 at mpr0 bus 0 scbus0 target 15 lun 0 da7: Fixed Direct Access SPC-4 SCSI device da7: Serial Number Z840MFM1 da7: 1200.000MB/s transfersKDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 da7: Command Queueing enabled da7: 7630885MB (15628053168 512 byte sectors) daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90dabd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dadb0 malloc() at malloc+0x192/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90daba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dad80 malloc() at malloc+0x192/frame 0xfffffe09f90dadd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90dae00 da8 at mpr0 bus 0 scbus0 target 16 lun 0 da8: Fixed Direct Access SPC-4 SCSI device da8: Serial Number Z840MTB5 da8: 1200.000MB/s transfers da8: Command Queueing enabled da8: 7630885MB (15628053168 512 byte sectors) disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() asbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at t xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5bd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5db0 malloc() at malloc+0x192/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5d80 malloc() at malloc+0x192/frame 0xfffffe09f90d5dd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5e30 da9 at mpr0 bus 0 scbus0 target 17 lun 0 da9: Fixed Direct Access SPC-4 SCSI device da9: Serial Number Z840MVNW da9: 1200.000MB/s transfers da9: Command Queueing enabled da9: 7630885MB (15628053168 512 byte sectors) g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90dabd0 kdb_backtrace() at uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_cokdb_backtrace+0x39/frame 0xfffffe09f90dac80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dadb0 malloc() at malloc+0x192/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 nf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90daba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dad80 malloc() at malloc+0x192/frame 0xfffffe09f90dadd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 da10 at mpr0 bus 0 scbus0 target 18 lun 0 da10: Fixed Direct Access SPC-4 SCSI device da10: Serial Number Z840MVLZ da10: 1200.000MB/s transfers da10: Command Queueing enabled da10: 7630885MB (15628053168 512 byte sectors) KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd620 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6d0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd790 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd800 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5bd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5db0 malloc() at malloc+0x192/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 malloc() at malloc+0x192/frame 0xfffffe09f90fd850 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d5c50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d5d10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d5d80 malloc() at malloc+0x192/frame 0xfffffe09f90d5dd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d5e00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d5e30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d5eb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6430 dadone() at dadone+0x30e/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/scsi/scsi_da.c:2244 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90fd5f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90fd6a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90fd760 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90fd7d0 malloc() at malloc+0x192/frame 0xfffffe09f90fd820 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90fd850 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90fd880 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90fd900 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90fdd90 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90fddc0 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90fde80 daregister() at daregister+0x5c0/frame 0xfffffe09f90fe100 cam_periph_alloc() at cam_periph_alloc+0x5a1/frame 0xfffffe09f90fe1e0 daasync() at daasync+0x290/frame 0xfffffe09f90fe250 xptsetasyncfunc() at xptsetasyncfunc+0x11a/frame 0xfffffe09f90fe770 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90fe7c0 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90fe800 xptbustraverse() at xptbustraverse+0xa1/frame 0xfffffe09f90fe840 xpt_register_async() at xpt_register_async+0x216/frame 0xfffffe09f90fe980 dainit() at dainit+0x19/frame 0xfffffe09f90fe990 periphdriver_init() at periphdriver_init+0x4e/frame 0xfffffe09f90fe9b0 xpt_finishconfig_task() at xpt_finishconfig_task+0x16/frame 0xfffffe09f90fe9e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe09f90fea40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe09f90fea70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90feab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90feab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- ada0 at ahcich4 bus 0 scbus5 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number S33MNB0H581014W ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada0: Command Queueing enabled ada0: 114473MB (234441648 512 byte sectors) ada0: Previously was known as ad12 ada1 at ahcich5 bus 0 scbus6 target 0 lun 0 ada1: ACS-2 ATA SATA 3.x device ada1: Serial Number S33MNB0H581027W ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada1: Command Queueing enabled ada1: 114473MB (234441648 512 byte sectors) ada1: Previously was known as ad14 ses0: da0,pass0: Element descriptor: 'Slot00' ses0: da0,pass0: SAS Device Slot Element: 1 Phys at Slot 0 uma_zalloc_arg: zone "64"ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe280 with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 da11 at mpr0 bus 0 scbus0 target 19 lun 0 da11: Fixed Direct Access SPC-4 SCSI device da11: Serial Number Z840MFNK da11: 1200.000MB/s transfers da11: Command Queueing enabled da11: 7630885MB (15628053168 512 byte sectors) xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc5c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- ses0: da1,pass1: Element descriptor: 'Slot01' ses0: da1,pass1: SAS Device Slot Element: 1 Phys at Slot 1 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe281 ses0: da2,pass2: Element descriptor: 'Slot02' ses0: da2,pass2: SAS Device Slot Element: 1 Phys at Slot 2 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe282 ses0: da3,pass3: Element descriptor: 'Slot03' ses0: da3,pass3: SAS Device Slot Element: 1 Phys at Slot 3 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe283 ses0: da4,pass4: Element descriptor: 'Slot04' ses0: da4,pass4: SAS Device Slot Element: 1 Phys at Slot 4 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe284 ses0: da5,pass5: Element descriptor: 'Slot05' ses0: da5,pass5: SAS Device Slot Element: 1 Phys at Slot 5 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe285 ses0: da6,pass6: Element descriptor: 'Slot06' ses0: da6,pass6: SAS Device Slot Element: 1 Phys at Slot 6 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe286 ses0: da7,pass7: Element descriptor: 'Slot07' ses0: da7,pass7: SAS Device Slot Element: 1 Phys at Slot 7 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe287 uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90dabd0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac80 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad40 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dadb0 malloc() at malloc+0x192/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc4c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:5259 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90daba0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90dac50 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90dad10 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90dad80 malloc() at malloc+0x192/frame 0xfffffe09f90dadd0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90dae00 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90dae30 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90daeb0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90db340 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90db370 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90db430 dadone() at dadone+0x30e/frame 0xfffffe09f90db9e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90dba20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90dba70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90dbab0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90dbab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc6460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc1c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fa0c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc2c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- ses0: da8,pass8: Element descriptor: 'Slot08' ses0: da8,pass8: SAS Device Slot Element: 1 Phys at Slot 8 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe288 ses0: da9,pass9: Element descriptor: 'Slot09' ses0: da9,pass9: SAS Device Slot Element: 1 Phys at Slot 9 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe289 ses0: da10,pass10: Element descriptor: 'Slot10' ses0: da10,pass10: SAS Device Slot Element: 1 Phys at Slot 10 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe28a ses0: da11,pass11: Element descriptor: 'Slot11' ses0: da11,pass11: SAS Device Slot Element: 1 Phys at Slot 11 ses0: phy 0: SATA device ses0: phy 0: parent 500304801eabe2bf addr 500304801eabe28b uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fbe460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb8c60) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fb7460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rspSMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #4 Launched! = 0, rbp = 0 --- uma_zalloc_arg: zone "64" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5ff0 Timecounter "TSC" frequency 1900037168 Hz quality 1000 WARNING: WITNESS option enabled, expect reduced performance. kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d60a0 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6160 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61d0 malloc() at malloc+0x192/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x61/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- uma_zalloc_arg: zone "16" with the following non-sleepable locks held: exclusive sleep mutex CAM device lock (CAM device lock) r = 0 (0xfffff80028fc0460) locked @ /freenas-9.10-releng/_BE/os/sys/cam/cam_xpt.c:2160 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe09f90d5fc0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe09f90d6070 witness_warn() at witness_warn+0x4b5/frame 0xfffffe09f90d6130 uma_zalloc_arg() at uma_zalloc_arg+0x3b/frame 0xfffffe09f90d61a0 malloc() at malloc+0x192/frame 0xfffffe09f90d61f0 sbuf_newbuf() at sbuf_newbuf+0xa6/frame 0xfffffe09f90d6220 sbuf_new() at sbuf_new+0x7b/frame 0xfffffe09f90d6250 g_conf_printf_escaped() at g_conf_printf_escaped+0x37/frame 0xfffffe09f90d62d0 devctl_notify_params() at devctl_notify_params+0xe9/frame 0xfffffe09f90d6760 devctl_notify() at devctl_notify+0x43/frame 0xfffffe09f90d6790 disk_attr_changed() at disk_attr_changed+0x95/frame 0xfffffe09f90d6850 daasync() at daasync+0x174/frame 0xfffffe09f90d68c0 xpt_async_process_dev() at xpt_async_process_dev+0x1b9/frame 0xfffffe09f90d6910 xptdevicetraverse() at xptdevicetraverse+0xce/frame 0xfffffe09f90d6960 xpttargettraverse() at xpttargettraverse+0x9f/frame 0xfffffe09f90d69a0 xpt_async_process() at xpt_async_process+0x112/frame 0xfffffe09f90d69e0 xpt_done_process() at xpt_done_process+0x3a0/frame 0xfffffe09f90d6a20 xpt_done_td() at xpt_done_td+0x136/frame 0xfffffe09f90d6a70 fork_exit() at fork_exit+0x84/frame 0xfffffe09f90d6ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe09f90d6ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Trying to mount root from zfs:freenas-boot/ROOT/9.10.2-U2 []... GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7) warning: KLD '/boot/kernel-debug/dtraceall.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/dtrace.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/dtmalloc.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/dtnfscl.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/fbt.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/fasttrap.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/lockstat.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/sdt.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/systrace.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/systrace_freebsd32.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/profile.ko' is newer than the linker.hints file lock order reversal: 1st 0xfffff80059f26068 tmpfs (tmpfs) @ /freenas-9.10-releng/_BE/os/sys/kern/vfs_mount.c:849 2nd 0xfffff80059f25b78 zfs (zfs) @ /freenas-9.10-releng/_BE/os/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1816 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0a8d66d2e0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0a8d66d390 witness_checkorder() at witness_checkorder+0xe2b/frame 0xfffffe0a8d66d420 __lockmgr_args() at __lockmgr_args+0x9ea/frame 0xfffffe0a8d66d560 vop_stdlock() at vop_stdlock+0x3c/frame 0xfffffe0a8d66d580 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xfc/frame 0xfffffe0a8d66d5b0 _vn_lock() at _vn_lock+0xd2/frame 0xfffffe0a8d66d620 zfs_root() at zfs_root+0xd7/frame 0xfffffe0a8d66d660 vfs_donmount() at vfs_donmount+0x138f/frame 0xfffffe0a8d66d960 sys_nmount() at sys_nmount+0x71/frame 0xfffffe0a8d66d9a0 amd64_syscall() at amd64_syscall+0x2c4/frame 0xfffffe0a8d66dab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0a8d66dab0 --- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0x80190521a, rsp = 0x7fffffffce18, rbp = 0x7fffffffce70 --- warning: KLD '/boot/kernel-debug/smbus.ko' is newer than the linker.hints file ipmi0: port 0xca2,0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi ipmi0: IPMI device rev. 1, firmware rev. 3.40, version 2.0 ipmi0: Number of channels 2 ipmi0: Attached watchdog hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 IAP/8/48/0x3ff IAF/3/48/0x67 warning: KLD '/boot/kernel-debug/t3_tom.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/toecore.ko' is newer than the linker.hints file warning: KLD '/boot/kernel-debug/t4_tom.ko' is newer than the linker.hints file em0: link state changed to UP vlan30: link state changed to UP em1: link state changed to UP warning: KLD '/boot/kernel-debug/ums.ko' is newer than the linker.hints file ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 warning: KLD '/boot/kernel-debug/ng_ether.ko' is newer than the linker.hints file GEOM_ELI: Device da0p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da1p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da2p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da3p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da4p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da5p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da6p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da7p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da8p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da9p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da10p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device da11p1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware WARNING: 172.30.30.4 (iqn.1994-05.com.unitrends:88df16cabe6f): no ping reply (NOP-Out) after 5 seconds; dropping connection