Degraded raidz2-0 disk offline cannot bring online

jkinning · Sep 10, 2015

I have searched the forums and Google as this appears to be an easy task but unfortunately my results are not the same. This was the guide I was reading - http://doc.freenas.org/9.3/freenas_storage.html#replacing-a-failed-drive I apparently had a hard drive failure which one of our Server Management folks handled by pulling out the drive and replacing it with a similiar 1tb drive. The hardware is a Dell with a Intel(R) Xeon(R) CPU E5520 @ 2.27GHz, 49116MB RAM, and 6 1TB drives. When I looked at the storage and Volume status the drive showed as unavailable. I was able to take it offline but now I cannot get it online. I am running FreeNAS-9.3-STABLE-201509022158 . When I click the disk in question and then replace I do not get the drop down to select anything from.

Running the zpool status from the command line shows this:
pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0h5m with 0 errors on Fri Aug 7 03:50:56 2015
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da0p2 ONLINE 0 0 0

errors: No known data errors

pool: freenas_pool
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: scrub in progress since Thu Sep 10 09:18:28 2015
81.5G scanned out of 1.60T at 263M/s, 1h40m to go
0 repaired, 4.99% done
config:

NAME STATE READ WRITE CKSUM
freenas_pool DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
gptid/d0b69775-9dab-11e4-a397-0026b9344159 ONLINE 0 0 0
gptid/d0fa5e3e-9dab-11e4-a397-0026b9344159 ONLINE 0 0 0
gptid/d1611dff-9dab-11e4-a397-0026b9344159 ONLINE 0 0 0
gptid/d1a4d611-9dab-11e4-a397-0026b9344159 ONLINE 0 0 0
gptid/d20fcaeb-9dab-11e4-a397-0026b9344159 ONLINE 0 0 0
3136789846265880697 OFFLINE 0 0 0 was /dev/gptid/d26a8a5b-9dab-11e4-a397-0026b9344159

errors: No known data errors

I then tried to manually replace the device and then get this message:
zpool replace freenas_pool 3136789846265880697 /dev/da0p1
cannot replace 3136789846265880697 with /dev/da0p1: device is too small

I am not sure if the drive that was replaced had existing data and if a quick format would correct that? I was told the existing data would be written over once the drive was added but that doesn't appear to be the case. I'm not real proficient with BSD and formatting a disk but decided to post here first as I have data on here that I don't want to lose. :)

Darren Myers · Sep 10, 2015

what raid card/HBA are you using?

jkinning · Sep 10, 2015

dmesg

Code:

Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.3-RELEASE-p25 #0 r281084+d3a5bf7: Wed Sep  2 15:00:10 PDT 2015
    root@build3.ixsystems.com:/tank/home/jkh/build/FN/objs/os-base/amd64/tank/home/jkh/build/FN/FreeBSD/src/sys/FREENAS.amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz (2261.05-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x106a5  Family = 0x6  Model = 0x1a  Stepping = 5
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x9ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  = 53687091200 (51200 MB)
avail memory = 49723641856 (47420 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <DELL   PE_SC3  >
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads
cpu0 (BSP): APIC ID: 16
cpu1 (AP): APIC ID: 17
cpu2 (AP): APIC ID: 18
cpu3 (AP): APIC ID: 19
cpu4 (AP): APIC ID: 20
cpu5 (AP): APIC ID: 21
cpu6 (AP): APIC ID: 22
cpu7 (AP): APIC ID: 23
cpu8 (AP): APIC ID:  0
cpu9 (AP): APIC ID:  1
cpu10 (AP): APIC ID:  2
cpu11 (AP): APIC ID:  3
cpu12 (AP): APIC ID:  4
cpu13 (AP): APIC ID:  5
cpu14 (AP): APIC ID:  6
cpu15 (AP): APIC ID:  7
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ioapic1: Changing APIC ID to 1
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 32-55 on motherboard
ispfw: registered firmware <isp_1040>
ispfw: registered firmware <isp_1040_it>
ispfw: registered firmware <isp_1080>
ispfw: registered firmware <isp_1080_it>
ispfw: registered firmware <isp_12160>
ispfw: registered firmware <isp_12160_it>
ispfw: registered firmware <isp_2100>
ispfw: registered firmware <isp_2200>
ispfw: registered firmware <isp_2300>
ispfw: registered firmware <isp_2322>
ispfw: registered firmware <isp_2400>
ispfw: registered firmware <isp_2400_multi>
ispfw: registered firmware <isp_2500>
ispfw: registered firmware <isp_2500_multi>
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
aesni0: No AESNI support.
padlock0: No ACE support.
acpi0: <DELL PE_SC3> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
cpu8: <ACPI CPU> on acpi0
cpu9: <ACPI CPU> on acpi0
cpu10: <ACPI CPU> on acpi0
cpu11: <ACPI CPU> on acpi0
cpu12: <ACPI CPU> on acpi0
cpu13: <ACPI CPU> on acpi0
cpu14: <ACPI CPU> on acpi0
cpu15: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x7f irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x5f irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 450
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
bce0: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0xd6000000-0xd7ffffff irq 36 at device 0.0 on pci1
miibus0: <MII bus> on bce0
brgphy0: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bce0: Ethernet address: 00:26:b9:34:41:59
bce0: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.0.6); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.3)
Coal (RX:6,6,18,18; TX:20,20,80,80)
bce1: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0xd8000000-0xd9ffffff irq 48 at device 0.1 on pci1
miibus1: <MII bus> on bce1
brgphy1: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bce1: Ethernet address: 00:26:b9:34:41:5b
bce1: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.0.6); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.3)
Coal (RX:6,6,18,18; TX:20,20,80,80)
pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci2: <ACPI PCI bus> on pcib2
bce2: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0xda000000-0xdbffffff irq 32 at device 0.0 on pci2
miibus2: <MII bus> on bce2
brgphy2: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus2
brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bce2: Ethernet address: 00:26:b9:34:41:5d
bce2: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.0.6); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.3)
Coal (RX:6,6,18,18; TX:20,20,80,80)
bce3: <QLogic NetXtreme II BCM5709 1000Base-T (C0)> mem 0xdc000000-0xddffffff irq 42 at device 0.1 on pci2
miibus3: <MII bus> on bce3
brgphy3: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus3
brgphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bce3: Ethernet address: 00:26:b9:34:41:5f
bce3: ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.0.6); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.3)
Coal (RX:6,6,18,18; TX:20,20,80,80)
pcib3: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci3: <ACPI PCI bus> on pcib3
mfi0: <Dell PERC 6> port 0xfc00-0xfcff mem 0xdf180000-0xdf1bffff,0xdf1c0000-0xdf1fffff irq 33 at device 0.0 on pci3
mfi0: Using MSI
mfi0: Megaraid SAS driver Ver 4.23
mfi0: FW MaxCmds = 1008, limiting to 128
mfip0: <SCSI Passthrough Bus> on mfi0
pcib4: <ACPI PCI-PCI bridge> at device 5.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 7.0 on pci0
pci6: <ACPI PCI bus> on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 9.0 on pci0
pci7: <ACPI PCI bus> on pcib7
pci0: <base peripheral, interrupt controller> at device 20.0 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.1 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.2 (no driver attached)
uhci0: <Intel 82801I (ICH9) USB controller> port 0xec40-0xec5f irq 17 at device 26.0 on pci0
usbus0 on uhci0
uhci1: <Intel 82801I (ICH9) USB controller> port 0xec60-0xec7f irq 18 at device 26.1 on pci0
usbus1 on uhci1
ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xdf0ff800-0xdf0ffbff irq 19 at device 26.7 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci0
uhci2: <Intel 82801I (ICH9) USB controller> port 0xec80-0xec9f irq 21 at device 29.0 on pci0
usbus3 on uhci2
uhci3: <Intel 82801I (ICH9) USB controller> port 0xeca0-0xecbf irq 20 at device 29.1 on pci0
usbus4 on uhci3
ehci1: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xdf0ffc00-0xdf0fffff irq 21 at device 29.7 on pci0
usbus5: EHCI version 1.0
usbus5 on ehci1
pcib8: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci8: <ACPI PCI bus> on pcib8
vgapci0: <VGA-compatible display> mem 0xd5800000-0xd5ffffff,0xde7fc000-0xde7fffff,0xde800000-0xdeffffff irq 19 at device 3.0 on pci8
vgapci0: Boot video device
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
qpi0: <QPI system bus> on motherboard
pcib9: <QPI Host-PCI bridge> pcibus 255 on qpi0
pci255: <PCI bus> on pcib9
pcib10: <QPI Host-PCI bridge> pcibus 254 on qpi0
pci254: <PCI bus> on pcib10
ichwd0 on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xd2000-0xd2fff,0xec000-0xeffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est0 attach returned 6
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est1 attach returned 6
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est2 attach returned 6
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est3 attach returned 6
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est4 attach returned 6
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est5 attach returned 6
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est6 attach returned 6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est7 attach returned 6
coretemp8: <CPU On-Die Thermal Sensors> on cpu8
est8: <Enhanced SpeedStep Frequency Control> on cpu8
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est8 attach returned 6
coretemp9: <CPU On-Die Thermal Sensors> on cpu9
est9: <Enhanced SpeedStep Frequency Control> on cpu9
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est9 attach returned 6
coretemp10: <CPU On-Die Thermal Sensors> on cpu10
est10: <Enhanced SpeedStep Frequency Control> on cpu10
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est10 attach returned 6
coretemp11: <CPU On-Die Thermal Sensors> on cpu11
est11: <Enhanced SpeedStep Frequency Control> on cpu11
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est11 attach returned 6
coretemp12: <CPU On-Die Thermal Sensors> on cpu12
est12: <Enhanced SpeedStep Frequency Control> on cpu12
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est12 attach returned 6
coretemp13: <CPU On-Die Thermal Sensors> on cpu13
est13: <Enhanced SpeedStep Frequency Control> on cpu13
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est13 attach returned 6
coretemp14: <CPU On-Die Thermal Sensors> on cpu14
est14: <Enhanced SpeedStep Frequency Control> on cpu14
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est14 attach returned 6
coretemp15: <CPU On-Die Thermal Sensors> on cpu15
est15: <Enhanced SpeedStep Frequency Control> on cpu15
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 12
device_attach: est15 attach returned 6
mfi0: 36405 (495204532s/0x0020/info) - Shutdown command received from host
mfi0: 36406 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0060/1000/1f0c/1028)
mfi0: 36407 (boot + 3s/0x0020/info) - Firmware version 1.22.02-0612
mfi0: 36408 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0060/1000/1f0c/1028)
mfi0: 36409 (boot + 3s/0x0020/info) - Firmware version 1.22.02-0612
mfi0: 36410 (boot + 23s/0x0008/info) - Battery Present
mfi0: 36411 (boot + 23s/0x0020/info) - Controller hardware revision ID (0x0)
mfi0: 36412 (boot + 23s/0x0020/info) - Package version 6.2.0-0013
mfi0: 36413 (boot + 23s/0x0020/info) - Board Revision
mfi0: 36414 (boot + 38s/0x0004/info) - Enclosure PD 20(c None/p0) communication restored
mfi0: 36415 (boot + 38s/0x0002/info) - Inserted: Encl PD 20
mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
mfi0: 36416 (boot + 38s/0x0002/info) - Inserted: PD 20(c None/p0) Info: enclPd=20, scsiType=d, portMap=09, sasAddr=5002609038539900,0000000000000000
mfi0: 36417 (boot + 38s/0x0002/info) - Inserted: PD 00(e0x20/s0)
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 480Mbps High Speed USB v2.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 480Mbps High Speed USB v2.0
mfid0 on mfi0
mfid0: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
mfid1 on mfi0
mfid1: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
mfid2 on mfi0
mfid2: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
mfid3 on mfi0
mfid3: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
mfid4 on mfi0
mfid4: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen5.1: <Intel> at usbus5
uhub5: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus5
mfi0: 36418 (boot + 38s/0x0002/info) - Inserted: PD 00(e0x20/s0) Info: enclPd=20, scsiType=0, portMap=00, sasAddr=1221000000000000,0000000000000000
mfi0: 36419 (boot + 38s/0x0002/info) - Inserted: PD 01(e0x20/s1)
mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
mfi0: 36420 (boot + 38s/0x0002/info) - Inserted: PD 01(e0x20/s1) Info: enclPd=20, scsiType=0, portMap=01, sasAddr=1221000001000000,0000000000000000
mfi0: 36421 (boot + 38s/0x0002/info) - Inserted: PD 02(e0x20/s2)
mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
mfi0: 36422 (boot + 38s/0x0002/info) - Inserted: PD 02(e0x20/s2) Info: enclPd=20, scsiType=0, portMap=02, sasAddr=1221000002000000,0000000000000000
mfi0: 36423 (boot + 38s/0x0002/info) - Inserted: PD 03(e0x20/s3)
mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
mfi0: 36424 (boot + 38s/0x0002/info) - Inserted: PD 03(e0x20/s3) Info: enclPd=20, scsiType=0, portMap=03, sasAddr=1221000003000000,0000000000000000
mfi0: 36425 (boot + 38s/0x0002/WARN) - PD 03(e0x20/s3) is not a certified drive
mfi0: 36426 (boot + 38s/0x0002/info) - Inserted: PD 04(e0x20/s4)
mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
mfi0: 36427 (boot + 38s/0x0002/info) - Inserted: PD 04(e0x20/s4) Info: enclPd=20, scsiType=0, portMap=04, sasAddr=1221000004000000,0000000000000000
mfi0: 36428 (boot + 38s/0x0002/info) - Inserted: PD 05(e0x20/s5)
mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
mfi0: 36429 (boot + 38s/0x0002/info) - Inserted: PD 05(e0x20/s5) Info: enclPd=20, scsiType=0, portMap=05, sasAddr=1221000005000000,0000000000000000
mfi0: 36430 (495204603s/0x0020/info) - Time established as 09/10/15 12:50:03; (51 seconds since power on)
mfi0: 36431 (495204640s/0x0008/info) - Battery temperature is normal
mfi0: 36432 (495204640s/0x0008/FATAL) - Battery needs replacement - SOH Bad
mfi0: 36433 (495204640s/0x0008/WARN) - Current capacity of the battery is below threshold
mfi0: 36434 (495204640s/0x0008/WARN) - BBU disabled; changing WB virtual disks to WT
mfi0: 36435 (495204640s/0x0001/info) - Policy change on VD 00/0 to [ID=00,dcp=01,ccp=00,ap=0,dc=0,dbgi=0] from [ID=00,dcp=01,ccp=01,ap=0,dc=0,dbgi=0]
mfi0: 36436 (495204640s/0x0001/info) - Policy change on VD 01/1 to [ID=01,dcp=01,ccp=00,ap=0,dc=0,dbgi=0] from [ID=01,dcp=01,ccp=01,ap=0,dc=0,dbgi=0]
mfi0: 36437 (495204640s/0x0001/info) - Policy change on VD 02/2 to [ID=02,dcp=01,ccp=00,ap=0,dc=0,dbgi=0] from [ID=02,dcp=01,ccp=01,ap=0,dc=0,dbgi=0]
mfi0: 36438 (495204640s/0x0001/info) - Policy change on VD 03/3 to [ID=03,dcp=01,ccp=00,ap=0,dc=0,dbgi=0] from [ID=03,dcp=01,ccp=01,ap=0,dc=0,dbgi=0]
mfi0: 36439 (495204640s/0x0001/info) - Policy change on VD 05/4 to [ID=05,dcp=01,ccp=00,ap=0,dc=0,dbgi=0] from [ID=05,dcp=01,ccp=01,ap=0,dc=0,dbgi=0]
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub4: 2 ports with 2 removable, self powered
uhub2: 4 ports with 4 removable, self powered
uhub5: 4 ports with 4 removable, self powered
ugen2.2: <vendor 0x0424> at usbus2
uhub6: <vendor 0x0424 product 0x2514, class 9/0, rev 2.00/0.00, addr 2> on usbus2
uhub6: MTT enabled
uhub6: 3 ports with 3 removable, self powered
ugen3.2: <Avocent> at usbus3
ugen0.2: <Avocent> at usbus0
ukbd0: <Keyboard> on usbus3
kbd2 at ukbd0
ukbd1: <EP1 Interrupt> on usbus0
ums0: <Mouse> on usbus3
kbd3 at ukbd1
ums0: 3 buttons and [Z] coordinates ID=0
ums1: <EP2 Interrupt> on usbus0
ums1: 5 buttons and [XYZ] coordinates ID=1
ugen2.3: <Kingston> at usbus2
umass0: <Kingston DataTraveler G3, class 0/0, rev 2.00/1.00, addr 3> on usbus2
umass0:  SCSI over Bulk-Only; quirks = 0x8100
umass0:2:0:-1: Attached to scbus2
da0 at umass-sim0 bus 0 scbus2 target 0 lun 0
da0: <Kingston DataTraveler G3 PMAP> Removable Direct Access SCSI-0 device
da0: Serial Number 001CC0EC347CBBA0C71C002E
da0: 40.000MB/s transfers
da0: 7636MB (15638528 512 byte sectors: 255H 63S/T 973C)
da0: quirks=0x2<NO_6_BYTE>
ses0 at mfi0 bus 0 scbus0 target 32 lun 0
ses0: <DP BACKPLANE 1.07> Fixed Enclosure Services SCSI-5 device
ses0: 150.000MB/s transfers
ses0: SCSI-3 ENC Device
pass0 at mfi0 bus 0 scbus0 target 0 lun 0
pass0: <ATA WDC WD1003FBYX-1 1V02> Fixed Uninstalled SCSI-5 device
pass0: Serial Number      WD-WCAW32112170
pass0: 150.000MB/s transfers
pass1 at mfi0 bus 0 scbus0 target 1 lun 0
pass1: <ATA ST1000NM0033-9ZM GA06> Fixed Uninstalled SCSI-5 device
pass1: Serial Number             Z1W1RTLC
pass1: 150.000MB/s transfers
pass2 at mfi0 bus 0 scbus0 target 2 lun 0
pass2: <ATA ST31000524NS KA02> Fixed Uninstalled SCSI-5 device
pass2: Serial Number             9WK0620E
pass2: 150.000MB/s transfers
pass3 at mfi0 bus 0 scbus0 target 3 lun 0
pass3: <ATA ST1000DM003-1CH1 CC49> Fixed Uninstalled SCSI-5 device
pass3: Serial Number             S1DGXFA9
pass3: 150.000MB/s transfers
pass4 at mfi0 bus 0 scbus0 target 4 lun 0
pass4: <ATA ST31000524NS KA02> Fixed Uninstalled SCSI-5 device
pass4: Serial Number             9WK06L9Y
pass4: 150.000MB/s transfers
pass5 at mfi0 bus 0 scbus0 target 5 lun 0
pass5: <ATA ST31000524NS KA06> Fixed Uninstalled SCSI-5 device
pass5: Serial Number             9WK5FTBM
pass5: 150.000MB/s transfers
SMP: AP CPU #1 Launched!
SMP: AP CPU #11 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #10 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #8 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #14 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #12 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #13 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #15 Launched!
SMP: AP CPU #9 Launched!
Timecounter "TSC-low" frequency 1130524625 Hz quality 1000
Trying to mount root from zfs:freenas-boot/ROOT/FreeNAS-9.3-STABLE-201509022158 []...
GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7)
mfi0: 36440 (495204705s/0x0008/info) - Battery started charging
ipmi0: <IPMI System Interface> on isa0
ipmi0: KCS mode found at io 0xca8 alignment 0x4 on isa
ipmi0: IPMI device rev. 0, firmware rev. 1.10, version 2.0
ipmi0: Number of channels 5
ipmi0: Attached watchdog
bce0: Gigabit link up!
bce0: Gigabit link up!
bce0: Gigabit link up!
GEOM_ELI: Device mfid4p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: software
GEOM_ELI: Device mfid0p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: software
GEOM_ELI: Device mfid1p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: software
GEOM_ELI: Device mfid2p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: software
GEOM_ELI: Device mfid3p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: software
vboxdrv: fAsync=0 offMin=0x2e1 offMax=0x3279
mfi0: 36441 (495204965s/0x0008/info) - Battery charge complete

jkinning · Sep 10, 2015

gpart list

Code:

Geom name: mfid0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 1952448478
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: mfid0p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 65536
   Mode: r1w1e1
   rawuuid: d0ec4e8e-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: mfid0p2
   Mediasize: 997506068480 (929G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 2147549184
   Mode: r1w1e2
   rawuuid: d0fa5e3e-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 997506068480
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 1952448471
   start: 4194432
Consumers:
1. Name: mfid0
   Mediasize: 999653638144 (931G)
   Sectorsize: 512
   Mode: r2w2e5

Geom name: mfid1
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 1952448478
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: mfid1p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 65536
   Mode: r1w1e1
   rawuuid: d1579881-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: mfid1p2
   Mediasize: 997506068480 (929G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 2147549184
   Mode: r1w1e2
   rawuuid: d1611dff-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 997506068480
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 1952448471
   start: 4194432
Consumers:
1. Name: mfid1
   Mediasize: 999653638144 (931G)
   Sectorsize: 512
   Mode: r2w2e5

Geom name: mfid2
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 1952448478
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: mfid2p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 65536
   Mode: r1w1e1
   rawuuid: d19d1e13-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: mfid2p2
   Mediasize: 997506068480 (929G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 2147549184
   Mode: r1w1e2
   rawuuid: d1a4d611-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 997506068480
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 1952448471
   start: 4194432
Consumers:
1. Name: mfid2
   Mediasize: 999653638144 (931G)
   Sectorsize: 512
   Mode: r2w2e5

Geom name: mfid3
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 1952448478
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: mfid3p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 65536
   Mode: r1w1e1
   rawuuid: d1fed021-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: mfid3p2
   Mediasize: 997506068480 (929G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 2147549184
   Mode: r1w1e2
   rawuuid: d20fcaeb-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 997506068480
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 1952448471
   start: 4194432
Consumers:
1. Name: mfid3
   Mediasize: 999653638144 (931G)
   Sectorsize: 512
   Mode: r2w2e5

Geom name: mfid4
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 1952448478
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: mfid4p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 65536
   Mode: r1w1e1
   rawuuid: d0ad19a5-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: mfid4p2
   Mediasize: 997506068480 (929G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 2147549184
   Mode: r1w1e2
   rawuuid: d0b69775-9dab-11e4-a397-0026b9344159
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 997506068480
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 1952448471
   start: 4194432
Consumers:
1. Name: mfid4
   Mediasize: 999653638144 (931G)
   Sectorsize: 512
   Mode: r2w2e5

Geom name: da0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 15638494
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da0p1
   Mediasize: 524288 (512k)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 17408
   Mode: r0w0e0
   rawuuid: d176c741-85e6-11e4-ab4f-0026b9344159
   rawtype: 21686148-6449-6e6f-744e-656564454649
   label: 1
   length: 524288
   offset: 17408
   type: bios-boot
   index: 1
   end: 1057
   start: 34
2. Name: da0p2
   Mediasize: 8006361088 (7.5G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 544768
   Mode: r1w1e1
   rawuuid: d18dcca8-85e6-11e4-ab4f-0026b9344159
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 8006361088
   offset: 544768
   type: freebsd-zfs
   index: 2
   end: 15638487
   start: 1064
Consumers:
1. Name: da0
   Mediasize: 8006926336 (7.5G)
   Sectorsize: 512
   Mode: r1w1e2

Darren Myers · Sep 10, 2015

It might help us to read the outputs if edit it and use [ code ] stuff here [ /code ] ; remove the spaces between the words and brackets and it will work. What does the output of camscontrol devlist command give you?

jkinning · Sep 10, 2015

camcontrol devlist
<ATA WDC WD1003FBYX-1 1V02> at scbus0 target 0 lun 0 (pass0)
<ATA ST1000NM0033-9ZM GA06> at scbus0 target 1 lun 0 (pass1)
<ATA ST31000524NS KA02> at scbus0 target 2 lun 0 (pass2)
<ATA ST1000DM003-1CH1 CC49> at scbus0 target 3 lun 0 (pass3)
<ATA ST31000524NS KA02> at scbus0 target 4 lun 0 (pass4)
<ATA ST31000524NS KA06> at scbus0 target 5 lun 0 (pass5)
<DP BACKPLANE 1.07> at scbus0 target 32 lun 0 (ses0,pass6)
<Kingston DataTraveler G3 PMAP> at scbus2 target 0 lun 0 (pass7,da0)

Robert Trevellyan · Sep 10, 2015

jkinning said:
had a hard drive failure which one of our Server Management folks handled by pulling out the drive and replacing it

Did they do this before you offlined it? If so, that would put you in a bad spot from the start.

jkinning said:
I then tried to manually replace the device

You need to find a solution that doesn't require this.

Is the new drive one of the Seagates, or is it the Western Digital?

jkinning said:
I am not sure if the drive that was replaced had existing data and if a quick format would correct that?

You may be on the right track here, but "format" in the sense of what Windows does won't do it. To be effective, you need to wipe out the partition table, which you can do with dd or even with a single badblocks write pass. Of course, if you have physical access to the drive then you can connect it to another box and wipe it there.

Slight differences in raw drive capacity can theoretically be accommodated by varying the size of the swap partition on each drive, but I don't know if that's automatic and if so, which minimum version of FreeNAS you would need to be using. So, please post full system details as described in the forum rules (linked from the top of every forum page).

Darren Myers · Sep 10, 2015

Did you happen to find out what RAID card or HBA is installed on the server? i.e what does the drive backplane connect too.

Bidule0hm · Sep 10, 2015

Robert Trevellyan said:
Slight differences in raw drive capacity can theoretically be accommodated by varying the size of the swap partition on each drive, but I don't know if that's automatic and if so, which minimum version of FreeNAS you would need to be using.

Yep, but it's a totally manual process.

jkinning · Sep 10, 2015

Darren Myers said:
Did you happen to find out what RAID card or HBA is installed on the server? i.e what does the drive backplane connect too.

Is there a command I could run to get that information? No one appears to know and if I can find out without opening up the box that would be great.

Darren Myers · Sep 10, 2015

jkinning said:
Is there a command I could run to get that information? No one appears to know and if I can find out without opening up the box that would be great.

far as i know best and quickest is to open it up, if the server has a PERC card in it it may show in iDrac.....i say may because i know with newer HP's the RAID card shows in the iLo screen.

Robert Trevellyan · Sep 10, 2015

I don't see anything useful in the debug output you posted. I do see that the machine originally shipped as a Google Search appliance. I guess it's been repurposed since then.

I believe the shortest path to where you want to be, which is a pool that is no longer degraded, is to follow the drive replacement process again, but this time use a 2TB drive.

However, since you (or whoever set up the system) were smart enough to go with RAIDZ2, you still have redundancy in your pool, which means you have time to seek a more elegant solution if you want to.

jkinning · Sep 10, 2015

Yes, I elected to go down the RAIDZ2 path. I think I am going to try and migrate the data to the other FreeNAS server, which was also a repurposed Google Appliance :) and rebuild. I just noticed another drive is now marked as UNAVAIL

Code:

 zpool status
  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0h5m with 0 errors on Fri Aug  7 03:50:56 2015
config:

	NAME        STATE     READ WRITE CKSUM
	freenas-boot  ONLINE       0     0     0
	  da0p2     ONLINE       0     0     0

errors: No known data errors

  pool: freenas_pool
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
	the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://illumos.org/msg/ZFS-8000-2Q
  scan: scrub in progress since Thu Sep 10 15:28:32 2015
        1.38T scanned out of 1.60T at 382M/s, 0h9m to go
        0 repaired, 86.68% done
config:

	NAME                                            STATE     READ WRITE CKSUM
	freenas_pool                                    DEGRADED     0     0     0
	  raidz2-0                                      DEGRADED     0     0     0
	    9746994442031900892                         UNAVAIL      0     0     0  was /dev/gptid/d0b69775-9dab-11e4-a397-0026b9344159
	    gptid/d0fa5e3e-9dab-11e4-a397-0026b9344159  ONLINE       0     0     0
	    gptid/d1611dff-9dab-11e4-a397-0026b9344159  ONLINE       0     0     0
	    gptid/d1a4d611-9dab-11e4-a397-0026b9344159  ONLINE       0     0     0
	    gptid/d20fcaeb-9dab-11e4-a397-0026b9344159  ONLINE       0     0     0
	    3136789846265880697                         OFFLINE      0     0     0  was /dev/gptid/d26a8a5b-9dab-11e4-a397-0026b9344159

errors: No known data errors

Darren Myers · Sep 10, 2015

Crap.....found what i was looking for.

jkinning said:
fi0: <Dell PERC 6> port 0xfc00-0xfcff mem 0xdf180000-0xdf1bffff,0xdf1c0000-0xdf1fffff irq 33 at device 0.0 on pci3

mfi0: Using MSI

mfi0: Megaraid SAS driver Ver 4.23

mfi0: FW MaxCmds = 1008, limiting to 128

mfip0: <SCSI Passthrough Bus> on mfi0

pcib4: <ACPI PCI-PCI bridge> at device 5.0 on pci0

pci4: <ACPI PCI bus> on pcib4

pcib5: <ACPI PCI-PCI bridge> at device 6.0 on pci0

pci5: <ACPI PCI bus> on pcib5

pcib6: <ACPI PCI-PCI bridge> at device 7.0 on pci0

pci6: <ACPI PCI bus> on pcib6

pcib7: <ACPI PCI-PCI bridge> at device 9.0 on pci0

pci7: <ACPI PCI bus> on pcib7

As i suspected, HW RAID.....

Robert Trevellyan · Sep 10, 2015

jkinning said:
I just noticed another drive is now marked as UNAVAIL

Now you're in trouble, if you lose another drive your pool is gone.

I suspect either you offlined the wrong drive or your tech pulled out the wrong drive.

jkinning said:
rebuild

Don't use a RAID card in the rebuild unless it can be flashed to IT mode.

jkinning · Sep 10, 2015

Darren Myers said:
Crap.....found what i was looking for.

As i suspected, HW RAID.....

I thought when I set this up I just made it JBOD on the controller card. So from what you can tell this is still HW RAID? That is definitely not what I wanted or thought I had. I am currently rsyncing the data off this freeNAS instance onto a different one in which afterwards was going to blow away the volume and start fresh. But it sounds like I may need to do something different first like remove HW RAID. Is it possible to correct all this without installing freeNAS again? I have this running on a USB stick.

Darren Myers · Sep 10, 2015

A lot of HW raid cards don't do true JBOD just like making each disk a RAID0 doesn't work either. I think your server has a SAS backplane which means you'd need to pick up a LSI 9211-8i or a M1015 and both would need to be flashed to P16 if on FreeNAS 9.3, and flashed to P20 if on FreeNAS 9.3.1. The fact having a PERC 6 card doesn't give me a warm fuzzy feeling that things are working as they should, a lot of users come here with HP HW raid cards and Dell PERC cards some can in fact be flashed to IT mode which is what FreeNAS wants others can't

Sent from my iPhone using Tapatalk

jkinning · Sep 10, 2015

Darren Myers said:
A lot of HW raid cards don't do true JBOD just like making each disk a RAID0 doesn't work either. I think your server has a SAS backplane which means you'd need to pick up a LSI 9211-8i or a M1015 and both would need to be flashed to P16 if on FreeNAS 9.3, and flashed to P20 if on FreeNAS 9.3.1. The fact having a PERC 6 card doesn't give me a warm fuzzy feeling that things are working as they should, a lot of users come here with HP HW raid cards and Dell PERC cards some can in fact be flashed to IT mode which is what FreeNAS wants others can't

Sent from my iPhone using Tapatalk

I'll do a Google search and see if I can flash this card but if not is there a recommendation? Or just wait until I get a different server and use the HW RAID?

Darren Myers · Sep 10, 2015

Here are some relevant threads that will change your mind about using the PERC 6:

https://forums.freenas.org/index.php?threads/dell-1950-freenas-9-2-1-2-hanging.19445/#post-109043
https://forums.freenas.org/index.php?threads/scrub-cause-webgui-ssh-hang-normal.36946/#post-225590
https://forums.freenas.org/index.ph...ow-iscsi-lun-isnt-mounting.28377/#post-184940
https://forums.freenas.org/index.ph...-errors-have-been-detected.22685/#post-135790

Robert Trevellyan · Sep 10, 2015

jkinning said:
mfid0 on mfi0
mfid0: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
mfid1 on mfi0
mfid1: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
mfid2 on mfi0
mfid2: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
mfid3 on mfi0
mfid3: 953344MB (1952448512 sectors) RAID volume (no label) is optimal
mfid4 on mfi0
mfid4: 953344MB (1952448512 sectors) RAID volume (no label) is optimal

This looks suspiciously like a series of single-drive RAID volumes to me, which is not the same as JBOD.

Important Announcement for the TrueNAS Community.

Degraded raidz2-0 disk offline cannot bring online

Cadet

Guru

Cadet

Cadet

Guru

Cadet

Pony Wrangler

Guru

Server Electronics Sorcerer

Cadet

Guru

Pony Wrangler

Cadet

Guru

Pony Wrangler

Cadet

Guru

Cadet

Guru

Pony Wrangler

Similar threads