GusGold
Cadet
- Joined
- Aug 17, 2017
- Messages
- 3
Hi,
Running 6 drives in RaidZ2. I was identifying the physical drives, one by one, by unplugging, getting the serial number, and re plugging them in (hot swapping). I got through 3 when I got an email saying the pool was UNAVAIL. I pulled out
I'm guessing the pool has lost the connection between the disconnected drives and their part in the array. How can I get them to be attached to the pool again?
Running 6 drives in RaidZ2. I was identifying the physical drives, one by one, by unplugging, getting the serial number, and re plugging them in (hot swapping). I got through 3 when I got an email saying the pool was UNAVAIL. I pulled out
ada0, ada1, and ada3. FreeNAS UI still shows the disks as attached and I can even import the 3 disks that are UNAVAIL to a new volume.
I'm guessing the pool has lost the connection between the disconnected drives and their part in the array. How can I get them to be attached to the pool again?
Code:
root@freenas:~ # zpool import pool: Ocean0 id: 7210756636996103959 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://illumos.org/msg/ZFS-8000-3C config: Ocean0 UNAVAIL insufficient replicas raidz2-0 UNAVAIL insufficient replicas 424928137642037504 UNAVAIL cannot open gptid/afaad383-8283-11e7-9735-d05099967fb8.eli ONLINE 9043206662708765391 UNAVAIL cannot open gptid/b0b63e45-8283-11e7-9735-d05099967fb8.eli ONLINE gptid/b1465820-8283-11e7-9735-d05099967fb8.eli ONLINE 18104134033649858603 UNAVAIL cannot open
Code:
root@freenas:~ # gpart status Name Status Components ada0p1 OK ada0 ada0p2 OK ada0 ada1p1 OK ada1 ada1p2 OK ada1 ada2p1 OK ada2 ada2p2 OK ada2 ada3p1 OK ada3 ada3p2 OK ada3 ada4p1 OK ada4 ada4p2 OK ada4 ada5p1 OK ada5 ada5p2 OK ada5 da0p1 OK da0 da0p2 OK da0 da1p1 OK da1 da1p2 OK da1
Code:
root@freenas:~ # dmesg Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-STABLE #0 r321665+c0805687fec(freenas/11.0-stable): Tue Sep 5 16:07:24 UTC 2017 root@gauntlet:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64 FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0) CPU: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz (2099.43-MHz K8-class CPU) Origin="GenuineIntel" Id=0x406f1 Family=0x6 Model=0x4f Stepping=1 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x121<LAHF,ABM,Prefetch> Structured Extended Features=0x21cbfbb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,NFPUSG,PQE,RDSEED,ADX,SMAP,PROCTRACE> XSAVE Features=0x1<XSAVEOPT> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 37044092928 (35328 MB) avail memory = 33158230016 (31622 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <ALASKA A M I > FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 1 package(s) x 8 core(s) x 2 hardware threads WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard random: entropy device external interface kbd1 at kbdmux0 module_register_init: MOD_LOAD (vesa, 0xffffffff80fa9d40, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" nexus0 cryptosoft0: <software crypto> on motherboard aesni0: <AES-CBC,AES-XTS,AES-GCM,AES-ICM> on motherboard padlock0: No ACE support. acpi0: <ALASKA A M I > on motherboard acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 cpu2: <ACPI CPU> on acpi0 cpu3: <ACPI CPU> on acpi0 cpu4: <ACPI CPU> on acpi0 cpu5: <ACPI CPU> on acpi0 cpu6: <ACPI CPU> on acpi0 cpu7: <ACPI CPU> on acpi0 cpu8: <ACPI CPU> on acpi0 cpu9: <ACPI CPU> on acpi0 cpu10: <ACPI CPU> on acpi0 cpu11: <ACPI CPU> on acpi0 cpu12: <ACPI CPU> on acpi0 cpu13: <ACPI CPU> on acpi0 cpu14: <ACPI CPU> on acpi0 cpu15: <ACPI CPU> on acpi0 atrtc0: <AT realtime clock> port 0x70-0x71,0x74-0x77 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: <ACPI Host-PCI bridge> on acpi0 pcib0: _OSC returned error 0x10 pci0: <ACPI PCI bus> on pcib0 pci0: <dasp, performance counters> at device 11.1 (no driver attached) pci0: <dasp, performance counters> at device 11.2 (no driver attached) pci0: <dasp, performance counters> at device 16.1 (no driver attached) pci0: <dasp, performance counters> at device 16.6 (no driver attached) pci0: <dasp, performance counters> at device 18.1 (no driver attached) pcib1: <ACPI Host-PCI bridge> port 0xcf8-0xcff numa-domain 0 on acpi0 pcib1: _OSC returned error 0x10 pci1: <ACPI PCI bus> numa-domain 0 on pcib1 pcib2: <ACPI PCI-PCI bridge> irq 32 at device 2.0 numa-domain 0 on pci1 pci2: <ACPI PCI bus> numa-domain 0 on pcib2 igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xe000-0xe01f mem 0xfbe00000-0xfbe1ffff,0xfbe20000-0xfbe23fff irq 32 at device 0.0 numa-domain 0 on pci2 igb0: Using MSIX interrupts with 3 vectors igb0: Ethernet address: d0:50:99:96:7f:b8 igb0: Bound queue 0 to cpu 0 igb0: Bound queue 1 to cpu 1 pci1: <unknown> at device 17.0 (no driver attached) ahci0: <Intel Wellsburg AHCI SATA controller> port 0xf130-0xf137,0xf120-0xf123,0xf110-0xf117,0xf100-0xf103,0xf040-0xf05f mem 0xfbf3d000-0xfbf3d7ff irq 16 at device 17.4 numa-domain 0 on pci1 ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported ahcich0: <AHCI channel> at channel 0 on ahci0 xhci0: <Intel Wellsburg USB 3.0 controller> mem 0xfbf20000-0xfbf2ffff irq 19 at device 20.0 numa-domain 0 on pci1 xhci0: 32 bytes context size, 64-bit DMA usbus0 numa-domain 0 on xhci0 pci1: <simple comms> at device 22.0 (no driver attached) em0: <Intel(R) PRO/1000 Network Connection 7.6.1-k> port 0xf020-0xf03f mem 0xfbf00000-0xfbf1ffff,0xfbf39000-0xfbf39fff irq 20 at device 25.0 numa-domain 0 on pci1 em0: Using an MSI interrupt em0: Ethernet address: d0:50:99:96:7f:b4 ehci0: <Intel Wellsburg USB 2.0 controller> mem 0xfbf38000-0xfbf383ff irq 18 at device 26.0 numa-domain 0 on pci1 usbus1: EHCI version 1.0 usbus1 numa-domain 0 on ehci0 pci1: <multimedia, HDA> at device 27.0 (no driver attached) pcib3: <ACPI PCI-PCI bridge> irq 16 at device 28.0 numa-domain 0 on pci1 pci3: <ACPI PCI bus> numa-domain 0 on pcib3 pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.4 numa-domain 0 on pci1 pci4: <ACPI PCI bus> numa-domain 0 on pcib4 xhci1: <XHCI (generic) USB 3.0 controller> mem 0xfbd00000-0xfbd07fff irq 16 at device 0.0 numa-domain 0 on pci4 xhci1: 32 bytes context size, 64-bit DMA xhci1: Unable to map MSI-X table usbus2 numa-domain 0 on xhci1 pcib5: <ACPI PCI-PCI bridge> irq 18 at device 28.6 numa-domain 0 on pci1 pci5: <ACPI PCI bus> numa-domain 0 on pcib5 pci5: <network> at device 0.0 (no driver attached) ehci1: <Intel Wellsburg USB 2.0 controller> mem 0xfbf37000-0xfbf373ff irq 18 at device 29.0 numa-domain 0 on pci1 usbus3: EHCI version 1.0 usbus3 numa-domain 0 on ehci1 isab0: <PCI-ISA bridge> at device 31.0 numa-domain 0 on pci1 isa0: <ISA bus> numa-domain 0 on isab0 ahci1: <Intel Wellsburg AHCI SATA controller> port 0xf090-0xf097,0xf080-0xf083,0xf070-0xf077,0xf060-0xf063,0xf000-0xf01f mem 0xfbf36000-0xfbf367ff irq 16 at device 31.2 numa-domain 0 on pci1 ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich4: <AHCI channel> at channel 0 on ahci1 ahcich5: <AHCI channel> at channel 1 on ahci1 ahcich6: <AHCI channel> at channel 2 on ahci1 ahcich7: <AHCI channel> at channel 3 on ahci1 ahcich8: <AHCI channel> at channel 4 on ahci1 ahcich9: <AHCI channel> at channel 5 on ahci1 acpi_button0: <Power Button> on acpi0 ichwd0: <Intel Wellsburg watchdog timer> on isa0 ichwd0: ICH WDT present but disabled in BIOS or hardware device_attach: ichwd0 attach returned 6 ichwd0: <Intel Wellsburg watchdog timer> at port 0x430-0x437,0x460-0x47f on isa0 ichwd0: ICH WDT present but disabled in BIOS or hardware device_attach: ichwd0 attach returned 6 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] coretemp0: <CPU On-Die Thermal Sensors> on cpu0 est0: <Enhanced SpeedStep Frequency Control> on cpu0 coretemp1: <CPU On-Die Thermal Sensors> on cpu1 est1: <Enhanced SpeedStep Frequency Control> on cpu1 coretemp2: <CPU On-Die Thermal Sensors> on cpu2 est2: <Enhanced SpeedStep Frequency Control> on cpu2 coretemp3: <CPU On-Die Thermal Sensors> on cpu3 est3: <Enhanced SpeedStep Frequency Control> on cpu3 coretemp4: <CPU On-Die Thermal Sensors> on cpu4 est4: <Enhanced SpeedStep Frequency Control> on cpu4 coretemp5: <CPU On-Die Thermal Sensors> on cpu5 est5: <Enhanced SpeedStep Frequency Control> on cpu5 coretemp6: <CPU On-Die Thermal Sensors> on cpu6 est6: <Enhanced SpeedStep Frequency Control> on cpu6 coretemp7: <CPU On-Die Thermal Sensors> on cpu7 est7: <Enhanced SpeedStep Frequency Control> on cpu7 coretemp8: <CPU On-Die Thermal Sensors> on cpu8 est8: <Enhanced SpeedStep Frequency Control> on cpu8 coretemp9: <CPU On-Die Thermal Sensors> on cpu9 est9: <Enhanced SpeedStep Frequency Control> on cpu9 coretemp10: <CPU On-Die Thermal Sensors> on cpu10 est10: <Enhanced SpeedStep Frequency Control> on cpu10 coretemp11: <CPU On-Die Thermal Sensors> on cpu11 est11: <Enhanced SpeedStep Frequency Control> on cpu11 coretemp12: <CPU On-Die Thermal Sensors> on cpu12 est12: <Enhanced SpeedStep Frequency Control> on cpu12 coretemp13: <CPU On-Die Thermal Sensors> on cpu13 est13: <Enhanced SpeedStep Frequency Control> on cpu13 coretemp14: <CPU On-Die Thermal Sensors> on cpu14 est14: <Enhanced SpeedStep Frequency Control> on cpu14 coretemp15: <CPU On-Die Thermal Sensors> on cpu15 est15: <Enhanced SpeedStep Frequency Control> on cpu15 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec freenas_sysctl: adding account. freenas_sysctl: adding directoryservice. freenas_sysctl: adding middlewared. freenas_sysctl: adding network. freenas_sysctl: adding services. ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled nvme cam probe device init usbus0: 5.0Gbps Super Speed USB v3.0 usbus1: 480Mbps High Speed USB v2.0 usbus2: 5.0Gbps Super Speed USB v3.0 usbus3: 480Mbps High Speed USB v2.0 ugen3.1: <Intel EHCI root HUB> at usbus3 uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3 ugen2.1: <0x1b21 XHCI root HUB> at usbus2 uhub1: <0x1b21 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus2 ugen1.1: <Intel EHCI root HUB> at usbus1 uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 ugen0.1: <0x8086 XHCI root HUB> at usbus0 uhub3: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 uhub1: 4 ports with 4 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 21 ports with 21 removable, self powered ugen0.2: <Broadcom Corp BCM20702A0> at usbus0 ugen1.2: <vendor 0x8087 product 0x800a> at usbus1 uhub4 numa-domain 0 on uhub2 uhub4: <vendor 0x8087 product 0x800a, class 9/0, rev 2.00/0.05, addr 2> on usbus1 ugen3.2: <vendor 0x8087 product 0x8002> at usbus3 uhub5 numa-domain 0 on uhub0 uhub5: <vendor 0x8087 product 0x8002, class 9/0, rev 2.00/0.05, addr 2> on usbus3 uhub4: 6 ports with 6 removable, self powered uhub5: 8 ports with 8 removable, self powered ugen0.3: <CPS CP1500EPFCLCD> at usbus0 ugen0.4: <Lexar USB Flash Drive> at usbus0 umass0 numa-domain 0 on uhub3 umass0: <Lexar USB Flash Drive, class 0/0, rev 3.00/11.00, addr 3> on usbus0 umass0: SCSI over Bulk-Only; quirks = 0x8100 umass0:8:0: Attached to scbus8 ugen0.5: <Lexar USB Flash Drive> at usbus0 umass1 numa-domain 0 on uhub3 umass1: <Lexar USB Flash Drive, class 0/0, rev 3.00/11.00, addr 4> on usbus0 umass1: SCSI over Bulk-Only; quirks = 0x8100 umass1:9:1: Attached to scbus9 ada0 at ahcich4 bus 0 scbus1 target 0 lun 0 ada0: <WDC WD80EFZX-68UW8N0 83.H0A83> ACS-2 ATA SATA 3.x device ada0: Serial Number VK1AZ4HY ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 7630885MB (15628053168 512 byte sectors) ada1 at ahcich5 bus 0 scbus2 target 0 lun 0 ada1: <WDC WD80EFZX-68UW8N0 83.H0A83> ACS-2 ATA SATA 3.x device ada1: Serial Number VK18PVZY ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 7630885MB (15628053168 512 byte sectors) ada2 at ahcich6 bus 0 scbus3 target 0 lun 0 ada2: <WDC WD80EFZX-68UW8N0 83.H0A83> ACS-2 ATA SATA 3.x device ada2: Serial Number VK18B4TY ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 7630885MB (15628053168 512 byte sectors) ada3 at ahcich7 bus 0 scbus4 target 0 lun 0 ada3: <WDC WD80EFZX-68UW8N0 83.H0A83> ACS-2 ATA SATA 3.x device ada3: Serial Number VK1AE90Y ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada3: Command Queueing enabled ada3: 7630885MB (15628053168 512 byte sectors) ada4 at ahcich8 bus 0 scbus5 target 0 lun 0 ada4: <WDC WD80EFZX-68UW8N0 83.H0A83> ACS-2 ATA SATA 3.x device ada4: Serial Number VK1D6N4Y ada4: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada4: Command Queueing enabled ada4: 7630885MB (15628053168 512 byte sectors) ada5 at ahcich9 bus 0 scbus6 target 0 lun 0 ada5: <WDC WD80EFZX-68UW8N0 83.H0A83> ACS-2 ATA SATA 3.x device ada5: Serial Number VK1BTL3Y ada5: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada5: Command Queueing enabled ada5: 7630885MB (15628053168 512 byte sectors) da0 at umass-sim0 bus 0 scbus8 target 0 lun 0 da0: <Lexar USB Flash Drive 1100> Removable Direct Access SPC-4 SCSI device da0: Serial Number AA48SBFDH4E0CXKZYJ da0: 400.000MB/s transfers da0: 30526MB (62517248 512 byte sectors) da0: quirks=0x2<NO_6_BYTE> da1 at umass-sim1 bus 1 scbus9 target 0 lun 0 da1: <Lexar USB Flash Drive 1100> Removable Direct Access SPC-4 SCSI device da1: Serial Number AA34IO8SNF6A0R23CU da1: 400.000MB/s transfers da1: 30526MB (62517248 512 byte sectors) da1: quirks=0x2<NO_6_BYTE> random: unblocking device. SMP: AP CPU #1 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #14 Launched! SMP: AP CPU #15 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #12 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #13 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #5 Launched! Timecounter "TSC" frequency 2099433180 Hz quality 1000 Trying to mount root from zfs:freenas-boot/ROOT/11.0-U3 []... GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7) GEOM_ELI: Device gptid/afaad383-8283-11e7-9735-d05099967fb8.eli created. GEOM_ELI: Encryption: AES-XTS 256 GEOM_ELI: Crypto: hardware GEOM_ELI: Device gptid/b0b63e45-8283-11e7-9735-d05099967fb8.eli created. GEOM_ELI: Encryption: AES-XTS 256 GEOM_ELI: Crypto: hardware GEOM_ELI: Device gptid/b1465820-8283-11e7-9735-d05099967fb8.eli created. GEOM_ELI: Encryption: AES-XTS 256 GEOM_ELI: Crypto: hardware ichwd0: <Intel Wellsburg watchdog timer> at port 0x430-0x437,0x460-0x47f on isa0 ichwd0: ICH WDT present but disabled in BIOS or hardware device_attach: ichwd0 attach returned 6 hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI> em0: link state changed to UP
Last edited:
