UNAVAIL missing device (but device is not missing..)

Status
Not open for further replies.
Joined
Sep 11, 2015
Messages
7
I have 10 drives in a ZFS pool and everything has been working great for about a year. 9 drives of 2TB and 1 drive of 3TB.

My motherboard died and I had to replace it. Luckly I had a identical motherboard at hand.
It's a motherboard from a HP XW6400. CPU is Intel(R) Xeon(R) CPU 5130 @ 2.00GHz. 8GB DDR2 RAM.
Chipset Type: Intel 5000X
Processor Socket: LGA771
RAM Technology: DDR2 SDRAM / ECC
Memory Speed: 667 MHz
Memory Specification Compliance: PC2-5300
Drives: WD Desktop Green
RAID Cards: 2x LSI 1064E-IR


I had 2 pools. Named pool01 and pool02 (great naming, I know).

When I booted the system up again with the new motherboard I ran the command "zpool import -f"
pool01 cam up with no problems, but the importent, large pool02 did not. I have used the last two days trying to reinstall OS, change position of the drives (changing up the sata-connections etc). But i'm finding no luck.
It looks like one drive are missing from pool02, but the drive is connected and I don't see any error.

I'm about to give up now, but I really hope there are someone out there who can help me.
Here are som data:

FreeNAS-9.3-STABLE-201506292332

Code:
[root@nas2] ~# zpool import -f
   pool: pool02
     id: 7346222624618935135
  state: UNAVAIL
 status: The pool was last accessed by another system.
 action: The pool cannot be imported due to damaged devices or data.
   see: http://illumos.org/msg/ZFS-8000-EY
 config:

        pool02                                          UNAVAIL  missing device
          raidz1-1                                      ONLINE
            gptid/f42669d6-bbae-11e4-a886-001cc41ff072  ONLINE
            gptid/f58467d3-bbae-11e4-a886-001cc41ff072  ONLINE
            gptid/f673a2ac-bbae-11e4-a886-001cc41ff072  ONLINE
            gptid/f72d88e7-bbae-11e4-a886-001cc41ff072  ONLINE
            gptid/f8364ceb-bbae-11e4-a886-001cc41ff072  ONLINE
            gptid/f9616ed7-bbae-11e4-a886-001cc41ff072  ONLINE
            gptid/fae1348e-bbae-11e4-a886-001cc41ff072  ONLINE
            gptid/fc1a58c0-bbae-11e4-a886-001cc41ff072  ONLINE
            gptid/fd068f4b-bbae-11e4-a886-001cc41ff072  ONLINE


Code:
[root@nas2] ~# camcontrol devlist
<ATA ST33000651AS CC43>            at scbus0 target 1 lun 0 (pass0,da0)
<ATA ST2000DM001-9YN1 CC4B>        at scbus0 target 2 lun 0 (pass1,da1)
<ATA WDC WD20EARX-55P AB51>        at scbus0 target 3 lun 0 (pass2,da2)
<ATA WDC WD20EARX-00P AB51>        at scbus2 target 1 lun 0 (pass3,da3)
<ATA WDC WD20EARX-00P AB51>        at scbus2 target 2 lun 0 (pass4,da4)
<ATA WDC WD20EARX-00P AB51>        at scbus2 target 3 lun 0 (pass5,da5)
<ST2000DL003-9VT166 CC98>          at scbus5 target 0 lun 0 (pass6,ada0)
<WDC WD20EARX-00PASB0 51.0AB51>    at scbus6 target 0 lun 0 (pass7,ada1)
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus7 target 0 lun 0 (pass8,ada2)
<WDC WD20EARX-00PASB0 51.0AB51>    at scbus8 target 0 lun 0 (pass9,ada3)
<Verbatim STORE N GO 1100>         at scbus12 target 0 lun 0 (pass10,da6)


Code:
[root@nas2] /# dmesg
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.3-RELEASE-p16 #1 r281084+74d0d7a: Mon Jun 29 14:32:16 PDT 2015
    root@build3.ixsystems.com:/tank/home/jkh/build/FN/objs/os-base/amd64/tank/home/jkh/build/FN/FreeBSD/src/sys/FREENAS.amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz (1995.04-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x6f6  Family = 0x6  Model = 0xf  Stepping = 6
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x4e33d<SSE3,DTES64,MON,DS_CPL,VMX,TM2,SSSE3,CX16,xTPR,PDCM,DCA>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  = 9395240960 (8960 MB)
avail memory = 8230187008 (7848 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <COMPAQ GREENCRK>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ACPI Error: 32/64X address mismatch in Gpe0Block: 0x0000F828/0x000000000001F030, using 32 (20111123/tbfadt-470)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ispfw: registered firmware <isp_1040>
ispfw: registered firmware <isp_1040_it>
ispfw: registered firmware <isp_1080>
ispfw: registered firmware <isp_1080_it>
ispfw: registered firmware <isp_12160>
ispfw: registered firmware <isp_12160_it>
ispfw: registered firmware <isp_2100>
ispfw: registered firmware <isp_2200>
ispfw: registered firmware <isp_2300>
ispfw: registered firmware <isp_2322>
ispfw: registered firmware <isp_2400>
ispfw: registered firmware <isp_2400_multi>
ispfw: registered firmware <isp_2500>
ispfw: registered firmware <isp_2500_multi>
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
aesni0: No AESNI support.
padlock0: No ACE support.
acpi0: <HPQOEM SLIC-WKS> on motherboard
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, cff00000 (3) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
hpet1: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Event timer "HPET" frequency 14318180 Hz quality 450
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0xf808-0xf80b on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
device_attach: hpet0 attach returned 12
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci16: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci16
pci30: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 0.0 on pci30
pci32: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 1.0 on pci30
pci31: <ACPI PCI bus> on pcib4
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x006001> mem 0xf2200000-0xf220ffff irq 17 at device 0.0 on pci31
bge0: CHIP ID 0x00006001; ASIC REV 0x06; CHIP REV 0x60; PCI-E
miibus0: <MII bus> on bge0
brgphy0: <BCM5752 1000BASE-T media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Ethernet address: 00:1c:c4:1f:f0:72
pcib5: <PCI-PCI bridge> mem 0xf2301000-0xf2301fff at device 0.3 on pci16
pci17: <PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci96: <ACPI PCI bus> on pcib6
mpt0: <LSILogic SAS/SATA Adapter> port 0x2000-0x20ff mem 0xf2110000-0xf2113fff,0xf2100000-0xf210ffff irq 16 at device 0.0 on pci96
mpt0: MPI Version=1.5.12.0
mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
mpt0: 0 Active Volumes (2 Max)
mpt0: 0 Hidden Drive Members (10 Max)
pcib7: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci64: <ACPI PCI bus> on pcib7
vgapci0: <VGA-compatible display> port 0x3000-0x307f mem 0xf0000000-0xf0ffffff,0xd0000000-0xdfffffff,0xf1000000-0xf1ffffff irq 16 at device 0.0 on pci64
vgapci0: Boot video device
pcib8: <PCI-PCI bridge> at device 5.0 on pci0
pci254: <PCI bus> on pcib8
pcib9: <PCI-PCI bridge> at device 6.0 on pci0
pci253: <PCI bus> on pcib9
pcib10: <PCI-PCI bridge> at device 7.0 on pci0
pci252: <PCI bus> on pcib10
pcib11: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci128: <ACPI PCI bus> on pcib11
mpt1: <LSILogic SAS/SATA Adapter> port 0x1000-0x10ff mem 0xf2010000-0xf2013fff,0xf2000000-0xf200ffff irq 16 at device 0.0 on pci128
mpt1: MPI Version=1.5.12.0
mpt1: Capabilities: ( RAID-0 RAID-1E RAID-1 )
mpt1: 0 Active Volumes (2 Max)
mpt1: 0 Hidden Drive Members (10 Max)
uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x4000-0x401f irq 16 at device 29.0 on pci0
usbus0 on uhci0
uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x4020-0x403f irq 19 at device 29.1 on pci0
usbus1 on uhci1
uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x4040-0x405f irq 18 at device 29.2 on pci0
usbus2 on uhci2
uhci3: <Intel 631XESB/632XESB/3100 USB controller USB-4> port 0x4060-0x407f irq 23 at device 29.3 on pci0
usbus3 on uhci3
ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xf2400000-0xf24003ff irq 16 at device 29.7 on pci0
usbus4: EHCI version 1.0
usbus4 on ehci0
pcib12: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib12
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel 63XXESB2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x40c0-0x40cf irq 17 at device 31.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ahci0: <Intel ESB2 AHCI SATA controller> port 0x40e0-0x40e7,0x40f8-0x40fb,0x40e8-0x40ef,0x40fc-0x40ff,0x4080-0x409f mem 0xf2400400-0xf24007ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
acpi_button0: <Power Button> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
ichwd0 on isa0
ichwd0: resuming after hardware watchdog timeout
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub4: 8 ports with 8 removable, self powered
ugen4.2: <Verbatim> at usbus4
umass0: <Verbatim STORE N GO, class 0/0, rev 2.10/11.00, addr 2> on usbus4
umass0:  SCSI over Bulk-Only; quirks = 0x8100
umass0:12:0:-1: Attached to scbus12
da0 at mpt0 bus 0 scbus0 target 1 lun 0
da0: <ATA ST33000651AS CC43> Fixed Direct Access SCSI-5 device
da0: Serial Number             9XK0C20Y
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 2097151MB (4294967295 512 byte sectors: 255H 63S/T 267349C)
da1 at mpt0 bus 0 scbus0 target 2 lun 0
da1: <ATA ST2000DM001-9YN1 CC4B> Fixed Direct Access SCSI-5 device
da1: Serial Number             Z1F17L7V
da1: 300.000MB/s transfers
da1: Command Queueing enabled
da1: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da1: quirks=0x8<4K>
da2 at mpt0 bus 0 scbus0 target 3 lun 0
da2: <ATA WDC WD20EARX-55P AB51> Fixed Direct Access SCSI-5 device
da2: Serial Number      WD-WCAZAJ286546
da2: 300.000MB/s transfers
da2: Command Queueing enabled
da2: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da2: quirks=0x8<4K>
da3 at mpt1 bus 0 scbus2 target 1 lun 0
da3: <ATA WDC WD20EARX-00P AB51> Fixed Direct Access SCSI-5 device
da3: Serial Number      WD-WCAZA9730138
da3: 300.000MB/s transfers
da3: Command Queueing enabled
da3: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da3: quirks=0x8<4K>
da4 at mpt1 bus 0 scbus2 target 2 lun 0
da4: <ATA WDC WD20EARX-00P AB51> Fixed Direct Access SCSI-5 device
da4: Serial Number      WD-WCAZA9556853
da4: 300.000MB/s transfers
da4: Command Queueing enabled
da4: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da4: quirks=0x8<4K>
ada0 at ahcich0 bus 0 scbus5 target 0 lun 0
da5 at mpt1 bus 0 scbus2 target 3 lun 0
da5: <ATA WDC WD20EARX-00P AB51> Fixed Direct Access SCSI-5 device
da5: Serial Number      WD-WCAZAC299396
da5: 300.000MB/s transfers
da5: Command Queueing enabled
da5: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)
da5: quirks=0x8<4K>
da6 at umass-sim0 bus 0 scbus12 target 0 lun 0
da6: <Verbatim STORE N GO 1100> Removable Direct Access SCSI-6 device
da6: Serial Number 1208000000000001
da6: 40.000MB/s transfers
da6: 15376MB (31490048 512 byte sectors: 255H 63S/T 1960C)
da6: quirks=0x2<NO_6_BYTE>
ada0: <ST2000DL003-9VT166 CC98> ATA-8 SATA 2.x device
ada0: Serial Number 5YD8X6HL
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus6 target 0 lun 0
ada1: <WDC WD20EARX-00PASB0 51.0AB51> ATA-8 SATA 3.x device
ada1: Serial Number WD-WCAZAL013711
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1: quirks=0x1<4K>
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus7 target 0 lun 0
ada2: <WDC WD20EARS-00MVWB0 51.0AB51> ATA-8 SATA 2.x device
ada2: Serial Number WD-WCAZA1234395
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2: quirks=0x1<4K>
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus8 target 0 lun 0
ada3: <WDC WD20EARX-00PASB0 51.0AB51> ATA-8 SATA 3.x device
ada3: Serial Number WD-WCAZAF232373
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3: quirks=0x1<4K>
ada3: Previously was known as ad10
SMP: AP CPU #1 Launched!
Timecounter "TSC" frequency 1995040806 Hz quality 1000
GEOM: da0: corrupt or invalid GPT detected.
GEOM: da0: GPT rejected -- may not be recoverable.
Trying to mount root from zfs:freenas-boot/ROOT/default []...
GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7)
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
device_attach: hpet0 attach returned 12
driver bug: Unable to set devclass (devname: (null))
vboxdrv: fAsync=0 offMin=0x3ba offMax=0xc96



I would really appreciate if anyone can give me some help, if possible.
 
Last edited:
Joined
Sep 11, 2015
Messages
7
Code:
[root@nas2] /# gpart list
Geom name: da1
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da1p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: fce5396e-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: da1p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: fd068f4b-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: da1
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

Geom name: da2
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da2p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f930fa2f-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: da2p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f9616ed7-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: da2
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

Geom name: da3
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da3p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f3f99396-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: da3p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f42669d6-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: da3
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

Geom name: da4
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da4p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: fbfde9fd-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: da4p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: fc1a58c0-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: da4
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

Geom name: da5
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da5p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f55647b1-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: da5p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f58467d3-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: da5
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

Geom name: da6
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 31490014
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: da6p1
   Mediasize: 524288 (512k)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 17408
   Mode: r0w0e0
   rawuuid: 8e0a9fdb-2ba2-11e5-a604-001cc41ff072
   rawtype: 21686148-6449-6e6f-744e-656564454649
   label: 1
   length: 524288
   offset: 17408
   type: bios-boot
   index: 1
   end: 1057
   start: 34
2. Name: da6p2
   Mediasize: 16121856000 (15G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 544768
   Mode: r1w1e2
   rawuuid: 8e17ae01-2ba2-11e5-a604-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 16121856000
   offset: 544768
   type: freebsd-zfs
   index: 2
   end: 31489063
   start: 1064
Consumers:
1. Name: da6
   Mediasize: 16122904576 (15G)
   Sectorsize: 512
   Mode: r1w1e3

Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada0p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f716d3fc-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada0p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f72d88e7-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: ada0
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

Geom name: ada1
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada1p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: fa944a36-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada1p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: fae1348e-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: ada1
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

Geom name: ada2
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada2p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f65444a2-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada2p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f673a2ac-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: ada2
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0

Geom name: ada3
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 3907029134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada3p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f8049a96-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada3p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   rawuuid: f8364ceb-bbae-11e4-a886-001cc41ff072
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: 1
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: ada3
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0


Code:
[root@nas2] ~# gpart status
  Name  Status  Components
da1p1      OK  da1
da1p2      OK  da1
da2p1      OK  da2
da2p2      OK  da2
da3p1      OK  da3
da3p2      OK  da3
da4p1      OK  da4
da4p2      OK  da4
da5p1      OK  da5
da5p2      OK  da5
da6p1      OK  da6
da6p2      OK  da6
ada0p1      OK  ada0
ada0p2      OK  ada0
ada1p1      OK  ada1
ada1p2      OK  ada1
ada2p1      OK  ada2
ada2p2      OK  ada2
ada3p1      OK  ada3
ada3p2      OK  ada3
 
Last edited:

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
@Kristoffer Nilsen
To receive help, please refer to the Forum Rules which mentions posting the required list of
hardware used in your system and the version of FreeNAS you are currently running.
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
Why did you use the cli? Did just booting the machine not import the pool? Are you running smart tests on all the drives? I'm assuming since you used the cli you have also tried lots of other cli command, can you please tell us what those where and provide the output of zfs history.
 
Joined
Sep 11, 2015
Messages
7
@BigDave thanks for your reply. I have updated the first post with more information about my setup.

@SweetAndLow I started using the web GUI. I selected Import Volum and pool01 was imported using the GUI, no problem there.
But pool02 was nowhere to be found. I selected View disks and all my disks where showing.

I then moved on over to the CLI. Used "zpool import -f" and got the missing device error.
After that I used the "gpart list" and all disks where showing "state: ok" so no errors I could work with there.
Using "gpart status" also just showing OK.

The important thing here is that i'm no expert and after reading alot of posts in this forum I have not tried many commands in CLI
I would not want to type in alot of commands and maybe making it worse.

I reinstalled the same version of FreeNAS and tried to put the drives in the same order as they where on the old setup.

Using "zpool history" only shows the pool01, and not information about the broken pool.
 
Last edited:

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
The device know as da4 seems to be absent from your logs at one point?
Is your first pool (the one that imported) a single disk AND your second pool (pool01) a 9 disks vdev? Could you explain this in detail?
How does a USB flash drive show up as: da11 ??? (Verbatim STORE N GO 1100 Removable Direct Access SCSI-6 device),
how is this Verbatim drive attached to the motherboard?
Are your drives hooked up directly to the motherboard's SATA ports or do you have add-on PCIe RAID/HBA cards?
 
Joined
Sep 11, 2015
Messages
7
Sorry for the late reply @BigDave
I have removed the imported pool so the only pool in my setup is pool02, the one that is not imported.
The pool has 9 disks (2TB) and 1 disk (3TB). For some reason it only shows 2.2TB but my motherboard probably don't support more then 2TB, not sure.
4 of the 2TB disks are connected to the motherboard (SATA) and the rest of the disks are connected to two LSI 1064E RAID-cards. But I dont use RAID, only JBOD.

All 10 disks shows up in "View disks", but only 9 are recognized in the pool.

I would have thought that I could lose one disk and the pool would live on, but its not that easy I guess.

The OS is loaded on the Verbatim USB-drive. Its connected to the backplane of the motherboard.
How it got da11 I have no idea. It happened after I had to replace the motherboard.
 
Last edited:

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
I have 10 drives in a ZFS pool and everything has been working great for about a year. 9 drives of 2TB and 1 drive of 3TB.
Has this pool always been configured with those 10 drives?
Was the pool set up as a RAIDZ1, RAIDZ2 or something else...
 

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
Yes, the pool was created with these drives. No disks was added on a later date. It was Created as a RAIDZ1.
Ok, thats a plus. With that 10 drive pool with only one drive of parity you are gonna need some really good luck here. First thing is you made have made a grave
mistake with the forced mount command at the CLI. We will assume for the moment no damage has been done...
Your assumption that the motherboards were indentical may be the issue here, in this case, if there are some bios settings that are different, this may be super hard
or just plain impossible. The second motherboard's configuration may differ from the first board, so if there is any way possible to check the bios
of the first board (or maybe it's completely dead)?
If you have only one drive broken, the pool should still import via the FreeNAS GUI, it will just show as degraded. If you have two drives broken, your data
is gone :(
 
Joined
Sep 11, 2015
Messages
7
I got to be honest. I gave up.. And after a few days of crying I started over. I hope I have learned my lesson.
I trew out the old hardware and started from scratch. The system looks like this now:

Intel® Server Board S1200BTSR
Intel® Xeon® Processor E3-1240v2
4x8GB Kingston 1333MHz DDR3 ECC CL9 DIMM
X-Case RM420 (20 hotswap caddys)
4x LS2 SAS9212-4i4e (B)

8x WD Desktop Green 2TB (change the default timer from 8 seconds to 300 seconds)
2x Seagate 2TB 7200 RPM ST2000DM001
1x Seagate Barracuda Green 2TB
1x Hitachi GST Deskstar 7K3000 2TB
1x TOSHIBA DT01ACA200 2TB
1x WD Red 2TB

1x Seagate 3TB Barracuda XT
1x WD Desktop Green 3TB (change the default timer from 8 seconds to 300 seconds)
4x WD Red 3TB

I run 2 pools with 7 disks (2TB). And one pool of 6 disks (3TB).
After reading as much as I can about freenas I found that with the Z2-configuration I have chosen I really should use 4,6,8,10 drives, and not
7 like I have choosen. Will this be a problem I should address?

Any thoughts about this system? Will it do a good job or are there things I should change before I begin filling my drives again.. I want it to be "perfect" this time.

Thank you very much for your effort @BigDave and @SweetAndLow
 
Last edited:

rogerh

Guru
Joined
Apr 18, 2014
Messages
1,111
The general view is that there is little or no performance penalty for using 7 drives in RAIDZ2, especially if compression is in operation.
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
7 disks is fine
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Status
Not open for further replies.
Top