HPE D3600/D3700 Maximum Daisy Chain

Status
Not open for further replies.

Sybari

Cadet
Joined
May 28, 2017
Messages
2
Hello All,

Long time lurker of the forums, but first time poster (I think?).

I have a new build I'm working on at the moment, and am hoping the community can help me understand whether my problem is related to the hardware, or if it's something else.

We've got an HP DL360 Gen9 as the server hardware. Dual Proc, 192G of RAM, and 2 500GB SAS disks in RAID 1 attached to a P440ar RAID controller for the OS. FreeNAS 11.1-U5 is installed on this server.

We then have 5 HP DAS devices. One D3700 2.5" 25-bay DAS, followed by 4 D3600 3.5" 12-bay DAS devices. These interconnect to the DL360 Gen9 via a PCI H241 HBA.

We see all of the drives in HP's storage view of iLO, we don't see any issues with cabling whatsoever, two of the drives are showing SMART issues in iLO and will be replaced during the week.

When we go to create our first storage pool, we notice that only drives from the first three daisy chained DAS arrays are visible. dmesg doesn't seem to even detect any drives past the first 3 DAS arrays.

Output of camcontrol devlist:

Code:
root@nas02:~ # camcontrol devlist
<HP RAID 1(1+0) OK>				at scbus0 target 0 lun 0 (pass0,da0)
<HP MO0800JDVEV HPD2>			  at scbus3 target 16 lun 0 (pass1,da1)
<HP MO0800JDVEV HPD2>			  at scbus3 target 17 lun 0 (pass2,da2)
<HP MO0800JEFPB HPD3>			  at scbus3 target 18 lun 0 (pass3,da3)
<HP MO0800JEFPB HPD3>			  at scbus3 target 19 lun 0 (pass4,da4)
<HP MO0800JEFPB HPD3>			  at scbus3 target 20 lun 0 (pass5,da5)
<HP MO0800JEFPB HPD3>			  at scbus3 target 21 lun 0 (pass6,da6)
<HP MO0800JEFPB HPD3>			  at scbus3 target 22 lun 0 (pass7,da7)
<HP MO0800JEFPB HPD3>			  at scbus3 target 23 lun 0 (pass8,da8)
<HP MB6000JEFND HPD9>			  at scbus3 target 41 lun 0 (pass9,da9)
<HP MB6000JEFND HPD9>			  at scbus3 target 42 lun 0 (pass10,da10)
<HP MB6000JEFND HPD9>			  at scbus3 target 43 lun 0 (pass11,da11)
<HP MB6000JEFND HPD9>			  at scbus3 target 44 lun 0 (pass12,da12)
<HP MB6000JEFND HPD9>			  at scbus3 target 45 lun 0 (pass13,da13)
<HP MB6000JEFND HPD9>			  at scbus3 target 46 lun 0 (pass14,da14)
<HP MB6000JEFND HPD9>			  at scbus3 target 47 lun 0 (pass15,da15)
<HP MB6000JEFND HPD9>			  at scbus3 target 48 lun 0 (pass16,da16)
<HP MB6000JEFND HPD9>			  at scbus3 target 49 lun 0 (pass17,da17)
<HP MB6000JEFND HPD9>			  at scbus3 target 53 lun 0 (pass18,da18)
<HP MB6000JEFND HPD9>			  at scbus3 target 54 lun 0 (pass19,da19)
<HP MB6000JEFND HPD9>			  at scbus3 target 55 lun 0 (pass20,da20)
<HP MB6000JEFND HPD9>			  at scbus3 target 56 lun 0 (pass21,da21)
<HP MB6000JEFND HPD9>			  at scbus3 target 57 lun 0 (pass22,da22)
<HP MB6000JEFND HPD9>			  at scbus3 target 58 lun 0 (pass23,da23)
<HP MB6000JEFND HPD9>			  at scbus3 target 59 lun 0 (pass24,da24)
<HP MB6000JEFND HPD9>			  at scbus3 target 60 lun 0 (pass25,da25)
<HP MB6000JEFND HPD9>			  at scbus3 target 61 lun 0 (pass26,da26)


Output of dmesg:

Code:
root@nas02:~ # dmesg
Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
		The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.1-STABLE #0 r321665+e0c4ca60dfc(freenas/11.1-stable): Wed May 30 14:18:20 EDT 2018
	root@nemesis.tn.ixsystems.com:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64
FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn)
VT(efifb): resolution 1024x768
CPU: Intel(R) Xeon(R) CPU E5-2678 v3 @ 2.50GHz (2497.17-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x306f2  Family=0x6  Model=0x3f  Stepping=2
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Structured Extended Features=0x37ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,NFPUSG>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 208305913856 (198656 MB)
avail memory = 200331149312 (191050 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <HP	 ProLiant>
FreeBSD/SMP: Multiprocessor System Detected: 48 CPUs
FreeBSD/SMP: 2 package(s) x 12 core(s) x 2 hardware threads
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
SMP: AP CPU #1 Launched!
SMP: AP CPU #16 Launched!
SMP: AP CPU #40 Launched!
SMP: AP CPU #24 Launched!
SMP: AP CPU #11 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #9 Launched!
SMP: AP CPU #17 Launched!
SMP: AP CPU #28 Launched!
SMP: AP CPU #14 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #8 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #46 Launched!
SMP: AP CPU #41 Launched!
SMP: AP CPU #10 Launched!
SMP: AP CPU #27 Launched!
SMP: AP CPU #18 Launched!
SMP: AP CPU #19 Launched!
SMP: AP CPU #36 Launched!
SMP: AP CPU #15 Launched!
SMP: AP CPU #25 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #26 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #34 Launched!
SMP: AP CPU #35 Launched!
SMP: AP CPU #45 Launched!
SMP: AP CPU #43 Launched!
SMP: AP CPU #23 Launched!
SMP: AP CPU #42 Launched!
SMP: AP CPU #13 Launched!
SMP: AP CPU #37 Launched!
SMP: AP CPU #20 Launched!
SMP: AP CPU #29 Launched!
SMP: AP CPU #44 Launched!
SMP: AP CPU #21 Launched!
SMP: AP CPU #38 Launched!
SMP: AP CPU #47 Launched!
SMP: AP CPU #22 Launched!
SMP: AP CPU #33 Launched!
SMP: AP CPU #12 Launched!
SMP: AP CPU #30 Launched!
SMP: AP CPU #31 Launched!
SMP: AP CPU #39 Launched!
SMP: AP CPU #32 Launched!
Timecounter "TSC-low" frequency 1248582512 Hz quality 1000
random: entropy device external interface
kbd1 at kbdmux0
module_register_init: MOD_LOAD (vesa, 0xffffffff80fc74c0, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
nexus0
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS,AES-GCM,AES-ICM> on motherboard
padlock0: No ACE support.
acpi0: <HP ProLiant> on motherboard
acpi0: Power Button (fixed)
atrtc0: <AT realtime clock> port 0x70-0x71,0x74-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
Event timer "HPET3" frequency 14318180 Hz quality 340
Event timer "HPET4" frequency 14318180 Hz quality 340
Event timer "HPET5" frequency 14318180 Hz quality 340
Event timer "HPET6" frequency 14318180 Hz quality 340
Event timer "HPET7" frequency 14318180 Hz quality 340
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <dasp, performance counters> at device 11.1 (no driver attached)
pci0: <dasp, performance counters> at device 11.2 (no driver attached)
pci0: <dasp, performance counters> at device 16.1 (no driver attached)
pci0: <dasp, performance counters> at device 16.6 (no driver attached)
pci0: <dasp, performance counters> at device 18.1 (no driver attached)
pci0: <dasp, performance counters> at device 18.5 (no driver attached)
pcib1: <ACPI Host-PCI bridge> on acpi0
pci1: <ACPI PCI bus> on pcib1
pci1: <dasp, performance counters> at device 11.1 (no driver attached)
pci1: <dasp, performance counters> at device 11.2 (no driver attached)
pci1: <dasp, performance counters> at device 16.1 (no driver attached)
pci1: <dasp, performance counters> at device 16.6 (no driver attached)
pci1: <dasp, performance counters> at device 18.1 (no driver attached)
pci1: <dasp, performance counters> at device 18.5 (no driver attached)
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> on acpi0
pcib2: <ACPI Host-PCI bridge> port 0xcf8-0xcff numa-domain 0 on acpi0
pcib2: _OSC returned error 0x10
pci2: <ACPI PCI bus> numa-domain 0 on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 1.0 numa-domain 0 on pci2
pci3: <ACPI PCI bus> numa-domain 0 on pcib3
ciss0: <HP Smart Array P440ar> port 0x5000-0x50ff mem 0x93300000-0x933fffff,0x93400000-0x934003ff at device 0.0 numa-domain 0 on pci3
ciss0: PERFORMANT Transport
pcib4: <ACPI PCI-PCI bridge> at device 1.1 numa-domain 0 on pci2
pci4: <ACPI PCI bus> numa-domain 0 on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 2.0 numa-domain 0 on pci2
pci5: <ACPI PCI bus> numa-domain 0 on pcib5
ciss1: <HP Smart Array H241> port 0x4000-0x40ff mem 0x93100000-0x931fffff,0x93200000-0x932003ff at device 0.0 numa-domain 0 on pci5
ciss1: PERFORMANT Transport
pcib6: <ACPI PCI-PCI bridge> at device 2.1 numa-domain 0 on pci2
pci6: <ACPI PCI bus> numa-domain 0 on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 2.2 numa-domain 0 on pci2
pci7: <ACPI PCI bus> numa-domain 0 on pcib7
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x3020-0x303f mem 0x92f00000-0x92ffffff,0x93004000-0x93007fff at device 0.0 numa-domain 0 on pci7
ix0: Using MSI-X interrupts with 9 vectors
ix0: Ethernet address: 38:ea:a7:8e:06:d4
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x3000-0x301f mem 0x92e00000-0x92efffff,0x93000000-0x93003fff at device 0.1 numa-domain 0 on pci7
ix1: Using MSI-X interrupts with 9 vectors
ix1: Ethernet address: 38:ea:a7:8e:06:d5
ix1: PCI Express Bus: Speed 5.0GT/s Width x8
pcib8: <ACPI PCI-PCI bridge> at device 2.3 numa-domain 0 on pci2
pci8: <ACPI PCI bus> numa-domain 0 on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 3.0 numa-domain 0 on pci2
pci9: <ACPI PCI bus> numa-domain 0 on pcib9
ix2: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x2020-0x203f mem 0x92c80000-0x92cfffff,0x92d04000-0x92d07fff at device 0.0 numa-domain 0 on pci9
ix2: Using MSI-X interrupts with 9 vectors
ix2: Ethernet address: 00:1b:21:bc:76:1c
ix2: PCI Express Bus: Speed 5.0GT/s Width x8
ix3: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x2000-0x201f mem 0x92c00000-0x92c7ffff,0x92d00000-0x92d03fff at device 0.1 numa-domain 0 on pci9
ix3: Using MSI-X interrupts with 9 vectors
ix3: Ethernet address: 00:1b:21:bc:76:1d
ix3: PCI Express Bus: Speed 5.0GT/s Width x8
pcib10: <ACPI PCI-PCI bridge> at device 3.1 numa-domain 0 on pci2
pci10: <ACPI PCI bus> numa-domain 0 on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 3.2 numa-domain 0 on pci2
pci11: <ACPI PCI bus> numa-domain 0 on pcib11
pcib12: <ACPI PCI-PCI bridge> at device 3.3 numa-domain 0 on pci2
pci12: <ACPI PCI bus> numa-domain 0 on pcib12
pci2: <unknown> at device 17.0 (no driver attached)
xhci0: <Intel Wellsburg USB 3.0 controller> mem 0x39ffff00000-0x39ffff0ffff at device 20.0 numa-domain 0 on pci2
xhci0: 32 bytes context size, 64-bit DMA
usbus0 numa-domain 0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
ehci0: <Intel Wellsburg USB 2.0 controller> mem 0x93501000-0x935013ff at device 26.0 numa-domain 0 on pci2
usbus1: EHCI version 1.0
usbus1 numa-domain 0 on ehci0
usbus1: 480Mbps High Speed USB v2.0
pcib13: <ACPI PCI-PCI bridge> at device 28.0 numa-domain 0 on pci2
pci13: <ACPI PCI bus> numa-domain 0 on pcib13
pcib14: <ACPI PCI-PCI bridge> at device 28.2 numa-domain 0 on pci2
pci14: <ACPI PCI bus> numa-domain 0 on pcib14
vgapci0: <VGA-compatible display> mem 0x91000000-0x91ffffff,0x92a88000-0x92a8bfff,0x92000000-0x927fffff at device 0.1 numa-domain 0 on pci14
vgapci0: Boot video device
uhci0: <HP iLO Standard Virtual USB controller> port 0x1300-0x131f at device 0.4 numa-domain 0 on pci14
usbus2 numa-domain 0 on uhci0
usbus2: 12Mbps Full Speed USB v1.0
pcib15: <ACPI PCI-PCI bridge> at device 28.4 numa-domain 0 on pci2
pci15: <ACPI PCI bus> numa-domain 0 on pcib15
bge0: <HP Ethernet 1Gb 4-port 331i Adapter, ASIC rev. 0x5719001> mem 0x92b90000-0x92b9ffff,0x92ba0000-0x92baffff,0x92bb0000-0x92bbffff at device 0.0 numa-domain 0 on pci15
bge0: APE FW version: NCSI v1.4.18.0
bge0: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus0: <MII bus> numa-domain 0 on bge0
brgphy0: <BCM5719C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Using defaults for TSO: 65518/35/2048
bge0: Ethernet address: 14:02:ec:34:7c:a8
bge1: <HP Ethernet 1Gb 4-port 331i Adapter, ASIC rev. 0x5719001> mem 0x92b60000-0x92b6ffff,0x92b70000-0x92b7ffff,0x92b80000-0x92b8ffff at device 0.1 numa-domain 0 on pci15
bge1: APE FW version: NCSI v1.4.18.0
bge1: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus1: <MII bus> numa-domain 0 on bge1
brgphy1: <BCM5719C 1000BASE-T media interface> PHY 2 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Using defaults for TSO: 65518/35/2048
bge1: Ethernet address: 14:02:ec:34:7c:a9
bge2: <HP Ethernet 1Gb 4-port 331i Adapter, ASIC rev. 0x5719001> mem 0x92b30000-0x92b3ffff,0x92b40000-0x92b4ffff,0x92b50000-0x92b5ffff at device 0.2 numa-domain 0 on pci15
bge2: APE FW version: NCSI v1.4.18.0
bge2: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus2: <MII bus> numa-domain 0 on bge2
brgphy2: <BCM5719C 1000BASE-T media interface> PHY 3 on miibus2
brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge2: Using defaults for TSO: 65518/35/2048
bge2: Ethernet address: 14:02:ec:34:7c:aa
bge3: <HP Ethernet 1Gb 4-port 331i Adapter, ASIC rev. 0x5719001> mem 0x92b00000-0x92b0ffff,0x92b10000-0x92b1ffff,0x92b20000-0x92b2ffff at device 0.3 numa-domain 0 on pci15
bge3: APE FW version: NCSI v1.4.18.0
bge3: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus3: <MII bus> numa-domain 0 on bge3
brgphy3: <BCM5719C 1000BASE-T media interface> PHY 4 on miibus3
brgphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge3: Using defaults for TSO: 65518/35/2048
bge3: Ethernet address: 14:02:ec:34:7c:ab
pcib16: <ACPI PCI-PCI bridge> at device 28.6 numa-domain 0 on pci2
pci16: <ACPI PCI bus> numa-domain 0 on pcib16
pcib17: <ACPI PCI-PCI bridge> at device 28.7 numa-domain 0 on pci2
pci17: <ACPI PCI bus> numa-domain 0 on pcib17
ehci1: <Intel Wellsburg USB 2.0 controller> mem 0x93500000-0x935003ff at device 29.0 numa-domain 0 on pci2
usbus3: EHCI version 1.0
usbus3 numa-domain 0 on ehci1
usbus3: 480Mbps High Speed USB v2.0
isab0: <PCI-ISA bridge> at device 31.0 numa-domain 0 on pci2
isa0: <ISA bus> numa-domain 0 on isab0
pcib18: <ACPI Host-PCI bridge> numa-domain 1 on acpi0
pcib18: _OSC returned error 0x10
pci18: <ACPI PCI bus> numa-domain 1 on pcib18
pcib19: <ACPI PCI-PCI bridge> at device 0.0 numa-domain 1 on pci18
pci19: <ACPI PCI bus> numa-domain 1 on pcib19
pcib20: <ACPI PCI-PCI bridge> at device 1.0 numa-domain 1 on pci18
pci20: <ACPI PCI bus> numa-domain 1 on pcib20
pcib21: <ACPI PCI-PCI bridge> at device 1.1 numa-domain 1 on pci18
pci21: <ACPI PCI bus> numa-domain 1 on pcib21
pcib22: <ACPI PCI-PCI bridge> at device 2.0 numa-domain 1 on pci18
pci22: <ACPI PCI bus> numa-domain 1 on pcib22
pcib23: <ACPI PCI-PCI bridge> at device 2.1 numa-domain 1 on pci18
pci23: <ACPI PCI bus> numa-domain 1 on pcib23
pcib24: <ACPI PCI-PCI bridge> at device 2.2 numa-domain 1 on pci18
pci24: <ACPI PCI bus> numa-domain 1 on pcib24
pcib25: <ACPI PCI-PCI bridge> at device 2.3 numa-domain 1 on pci18
pci25: <ACPI PCI bus> numa-domain 1 on pcib25
pcib26: <ACPI PCI-PCI bridge> at device 3.0 numa-domain 1 on pci18
pci26: <ACPI PCI bus> numa-domain 1 on pcib26
pcib27: <ACPI PCI-PCI bridge> at device 3.1 numa-domain 1 on pci18
pci27: <ACPI PCI bus> numa-domain 1 on pcib27
pcib28: <ACPI PCI-PCI bridge> at device 3.2 numa-domain 1 on pci18
pci28: <ACPI PCI bus> numa-domain 1 on pcib28
pcib29: <ACPI PCI-PCI bridge> at device 3.3 numa-domain 1 on pci18
pci29: <ACPI PCI bus> numa-domain 1 on pcib29
uart0: <Non-standard ns8250 class UART with FIFOs> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <Non-standard ns8250 class UART with FIFOs> port 0x2f8-0x2ff irq 3 on acpi0
ichwd0: <Intel Wellsburg watchdog timer> on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
ichwd0: <Intel Wellsburg watchdog timer> at port 0x430-0x437,0x460-0x47f on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
freenas_sysctl: adding account.
freenas_sysctl: adding directoryservice.
freenas_sysctl: adding middlewared.
freenas_sysctl: adding network.
freenas_sysctl: adding services.
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
ugen2.1: <HP UHCI root HUB> at usbus2
ugen0.1: <0x8086 XHCI root HUB> at usbus0
ugen3.1: <Intel EHCI root HUB> at usbus3
ugen1.1: <Intel EHCI root HUB> at usbus1
uhub0: <HP UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
uhub2: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub2: 21 ports with 21 removable, self powered
ugen2.2: <BMC Virtual Keyboard> at usbus2
ukbd0 numa-domain 0 on uhub0
ukbd0: <Virtual Keyboard > on usbus2
kbd2 at ukbd0
ugen0.2: <vendor 0x0424 product 0x2660> at usbus0
uhub4 numa-domain 0 on uhub2
uhub4: <vendor 0x0424 product 0x2660, class 9/0, rev 2.00/8.01, addr 1> on usbus0
ugen3.2: <vendor 0x8087 product 0x8002> at usbus3
uhub5 numa-domain 0 on uhub1
uhub5: <vendor 0x8087 product 0x8002, class 9/0, rev 2.00/0.05, addr 2> on usbus3
ugen1.2: <vendor 0x8087 product 0x800a> at usbus1
uhub6 numa-domain 0 on uhub3
uhub6: <vendor 0x8087 product 0x800a, class 9/0, rev 2.00/0.05, addr 2> on usbus1
uhub4: 2 ports with 1 removable, self powered
uhub5: 8 ports with 8 removable, self powered
uhub6: 6 ports with 6 removable, self powered
da0 at ciss0 bus 0 scbus0 target 0 lun 0
da1 at ciss1 bus 32 scbus3 target 16 lun 0
da2 at ciss1 bus 32 scbus3 target 17 lun 0
da3 at ciss1 bus 32 scbus3 target 18 lun 0
da4 at ciss1 bus 32 scbus3 target 19 lun 0
da5 at ciss1 bus 32 scbus3 target 20 lun 0
da6 at ciss1 bus 32 scbus3 target 21 lun 0
da7 at ciss1 bus 32 scbus3 target 22 lun 0
da8 at ciss1 bus 32 scbus3 target 23 lun 0
da0: <HP RAID 1(1+0) OK> Fixed Direct Access SPC-3 SCSI device
da0: Serial Number PDNLH0BRH9OAXK
da0: 135.168MB/s transfersda8: da5: da7: da2: da3: da4: <HP MO0800JEFPB HPD3> Fixed Direct Access SPC-4 SCSI device
da1: <HP MO0800JEFPB HPD3> Fixed Direct Access SPC-4 SCSI device
da6: <HP MO0800JDVEV HPD2> Fixed Direct Access SPC-4 SCSI device

<HP MO0800JEFPB HPD3> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number 0RY26W5A
<HP MO0800JDVEV HPD2> Fixed Direct Access SPC-4 SCSI device
<HP MO0800JEFPB HPD3> Fixed Direct Access SPC-4 SCSI device
da8: Serial Number 0RY26BSA
da2: Serial Number 2MVGZL3A
da1: Serial Number 2MVJNY3A
da8: 135.168MB/s transfersda0: Command Queueing enabled
da1: 135.168MB/s transfers
<HP MO0800JEFPB HPD3> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number 0RY26V6A
<HP MO0800JEFPB HPD3> Fixed Direct Access SPC-4 SCSI device
da7: 135.168MB/s transfersda5: Serial Number 0RY26ZUA

da3: Serial Number 0RY22Z0A
da6: Serial Number 0RY26V5A
da5: 135.168MB/s transfersda3: 135.168MB/s transfersda2: 135.168MB/s transfersda4: 135.168MB/s transfers
da1: Command Queueing enabled
da8: Command Queueing enabled
da0: 476908MB (976707632 512 byte sectors)


da3: Command Queueing enabled

da6: 135.168MB/s transfersda2: Command Queueing enabled
da3: 763097MB (1562824368 512 byte sectors)
da4: Command Queueing enabled
da8: 763097MB (1562824368 512 byte sectors)
da2: 763097MB (1562824368 512 byte sectors)

da19 at ciss1 bus 32 scbus3 target 54 lun 0
da4: 763097MB (1562824368 512 byte sectors)
da18 at ciss1 bus 32 scbus3 target 53 lun 0
da24 at ciss1 bus 32 scbus3 target 59 lun 0
da19: da6: Command Queueing enabled
da24: da18: da1: 763097MB (1562824368 512 byte sectors)

da7: Command Queueing enabled
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da14 at ciss1 bus 32 scbus3 target 46 lun 0
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da7: 763097MB (1562824368 512 byte sectors)
da6: 763097MB (1562824368 512 byte sectors)
da5: Command Queueing enabled
da14: da18: Serial Number Z4D2K3SS
da24: Serial Number Z4D2K450
da19: Serial Number Z4D2KPSV
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da19: 135.168MB/s transfersda24: 135.168MB/s transfersda26 at ciss1 bus 32 scbus3 target 61 lun 0
da16 at ciss1 bus 32 scbus3 target 48 lun 0
da14: Serial Number Z4D2K4A4

da5: 763097MB (1562824368 512 byte sectors)

da17 at ciss1 bus 32 scbus3 target 49 lun 0
da16: da14: 135.168MB/s transfersda26: da17:
da21 at ciss1 bus 32 scbus3 target 56 lun 0
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da10 at ciss1 bus 32 scbus3 target 42 lun 0
da18: 135.168MB/s transfersda19: Command Queueing enabled
da24: Command Queueing enabled
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device

da26: Serial Number Z4D2L9PS
da21: da16: Serial Number Z4D2L9QT
da14: Command Queueing enabled
da10: da16: 135.168MB/s transfers<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da24: 5723166MB (11721045168 512 byte sectors)
da26: 135.168MB/s transfersda18: Command Queueing enabled

da9 at ciss1 bus 32 scbus3 target 41 lun 0
da21: Serial Number Z4D2K46X
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da17: Serial Number Z4D2KFC0
da16: Command Queueing enabled
da14: 5723166MB (11721045168 512 byte sectors)

da19: 5723166MB (11721045168 512 byte sectors)
da17: 135.168MB/s transfersda16: 5723166MB (11721045168 512 byte sectors)
da9: da13 at ciss1 bus 32 scbus3 target 45 lun 0
da18: 5723166MB (11721045168 512 byte sectors)
da22 at ciss1 bus 32 scbus3 target 57 lun 0
da13: da21: 135.168MB/s transfersda26: Command Queueing enabled
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da12 at ciss1 bus 32 scbus3 target 44 lun 0

da10: Serial Number Z4D2GJDV
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device

da26: 5723166MB (11721045168 512 byte sectors)
da9: Serial Number Z4D2KYAZ
da21: Command Queueing enabled
da11 at ciss1 bus 32 scbus3 target 43 lun 0
da10: 135.168MB/s transfersda12: da9: 135.168MB/s transfersda21: 5723166MB (11721045168 512 byte sectors)
da17: Command Queueing enabled
da11: da13: Serial Number Z4D2K4BD
da20 at ciss1 bus 32 scbus3 target 55 lun 0
da22:
da13: 135.168MB/s transfersda15 at ciss1 bus 32 scbus3 target 47 lun 0
da20: da10: Command Queueing enabled

<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da17: 5723166MB (11721045168 512 byte sectors)
<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da15:
da11: Serial Number Z4D2K3ZJ
da13: Command Queueing enabled
da22: Serial Number Z4D2KQ9A
da12: Serial Number Z4D2KPSX
da20: Serial Number Z4D2K43E
da9: Command Queueing enabled
da13: 5723166MB (11721045168 512 byte sectors)
da22: 135.168MB/s transfersda20: 135.168MB/s transfers<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device

da23 at ciss1 bus 32 scbus3 target 58 lun 0

da10: 5723166MB (11721045168 512 byte sectors)
da15: Serial Number Z4D2KQ4V
da9: 5723166MB (11721045168 512 byte sectors)
da23: da12: 135.168MB/s transfersda20: Command Queueing enabled
da22: Command Queueing enabled

<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da20: 5723166MB (11721045168 512 byte sectors)
da15: 135.168MB/s transfersda11: 135.168MB/s transfersda22: 5723166MB (11721045168 512 byte sectors)
da12: Command Queueing enabled
da23: Serial Number Z4D2K45B

da25 at ciss1 bus 32 scbus3 target 60 lun 0
da12: 5723166MB (11721045168 512 byte sectors)
da11: Command Queueing enabled
da25:
da23: 135.168MB/s transfersda11: 5723166MB (11721045168 512 byte sectors)
da15: Command Queueing enabled

<HP MB6000JEFND HPD9> Fixed Direct Access SPC-4 SCSI device
da25: Serial Number Z4D2K49W
random: unblocking device.
da15: 5723166MB (11721045168 512 byte sectors)
da23: Command Queueing enabled
da25: 135.168MB/s transfersda23: 5723166MB (11721045168 512 byte sectors)

da25: Command Queueing enabled
da25: 5723166MB (11721045168 512 byte sectors)
Trying to mount root from zfs:freenas-boot/ROOT/11.1-U5 []...
ipmi0: <IPMI System Interface> port 0xca2-0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
ipmi0: IPMI device rev. 2, firmware rev. 2.55, version 2.0
ipmi0: Number of channels 1
ipmi0: Attached watchdog
ichwd0: <Intel Wellsburg watchdog timer> at port 0x430-0x437,0x460-0x47f on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7)
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI>
bge0: link state changed to DOWN
lagg0: link state changed to DOWN
lagg1: link state changed to DOWN
ix1: link state changed to UP
lagg1: link state changed to UP
ix0: link state changed to UP
lagg0: link state changed to UP
ix2: link state changed to UP
ix3: link state changed to UP
bge0: link state changed to UP
ums0 numa-domain 0 on uhub0
ums0: <Virtual Mouse> on usbus2
ix3: link state changed to DOWN
ix3: link state changed to UP
ix3: link state changed to DOWN
ix2: link state changed to DOWN
ix2: link state changed to UP
ix2: link state changed to DOWN
ix2: link state changed to UP
ix3: link state changed to UP
ugen2.2: <BMC Virtual Keyboard> at usbus2 (disconnected)
ukbd0: at uhub0, port 1, addr 2 (disconnected)
ukbd0: detached
ums0: at uhub0, port 1, addr 2 (disconnected)
ums0: detached


Interestingly, only ONE of the disks that iLO lists as pending failure seems to be showing up in FreeNAS, however FreeNAS shows that 2 sequential devices are apparently somehow failing?

Are we missing something here? I'm aware of the fact that the H241 may not be ideal, and if the answer is buy a 9300-8e and be done with it, then fine. What I'm hoping for is some diagnostic direction to take to ensure that before we spend more money on extra hardware, we determine whether this is a daisy chain limitation imposed by the D3600/D3700 arrays, or whether this is down to the fact that FreeNAS/FreeBSD treats the H241 card like a RAID controller, even though it seems to know enough to present the disks as direct-attached SCSI. We've run 3-DAS setups like this for quite a while now, without issue.

Firmware on the DAS's have been updated to latest available, as well as on the H241 card.

Thanks for any help - much appreciated.
 
Joined
Feb 2, 2016
Messages
574
When we go to create our first storage pool, we notice that only drives from the first three daisy chained DAS arrays are visible.

We have the D2600/D2700 units so take this with a grain of salt; I've never touched the D3000 series... You may need to check your cabling and make sure you're not hitting the maximum number of chained enclosures. The HP docs show four as the maximum number of enclosures for the D3600 and two for the D3700. I'm not sure what the maximum is when you are mixing the two enclosures. The HP241 is good for 200 physical drives, just make sure it is in HBA mode.

Cheers,
Matt
 

Sybari

Cadet
Joined
May 28, 2017
Messages
2
Matt,

Thanks for the response. It looks like the H241 utilizes the CISS driver, regardless of whether it's in "HBA" mode or not. Unfortunate.

We've purchased an LSI 9300-8e card for this project now, however we've run into new issues with multipath. All drives are now detected on both paths by the mpr driver, however about 1/2 of them do not correctly show up in multipath. Some do not show up as multipath disks at all, while others simply show up with only one active path.

Where do you file a bug for FreeNAS?
 
Joined
Feb 2, 2016
Messages
574
Some do not show up as multipath disks at all, while others simply show up with only one active path.

Make sure you're properly cabled. Make sure your backplanes support multipath. Make sure your drives support multipath. Make sure your HBAs support multipath.

MB6000JEFND = 6TB conventional = dual port
MO0800JEFPB = 800G SSD = single port?

Any commonality between drive type or enclosure that might indicate incompatibility with multipath?

Where do you file a bug for FreeNAS?

File your FreeNAS Bug Reports here.

Cheers,
Matt
 
Status
Not open for further replies.
Top