zpool lost after reboot

Status
Not open for further replies.
P

pattawongse

Cadet
Joined
Oct 18, 2012
Messages
8
my build
-INTEL XEON E1230 V2 3.30GHz (1155)
-INTEL M/B SERVER S1200BTLR (1155)
-60Gb SSD
-8 Gb x4 ECC RAM
-WD RED 3 Tb x 20
-Dell H310 OEM x2

running FreeBSD 8.3-RELEASE-p7 (FREENAS.amd64) #1 r249203M
set-up in 8 + 8 + 4
ZFS_1 : 8x3Tb (Raid-Z1) [H310 JBOD][mfisyspd0-7]
ZFS_2 : 8x3Tb (Raid-Z1) [H310 JBOD][mfisyspd8-15]
ZFS_3 : 4x3Tb (Raid-Z1) [On-Board Controller][ada0,ada1,ada2,ada4]

it run without any problem for 4 month
yesterday i found ZFS_1 pool lost

first i check with zpool status and found only 2 pool(ZFS_1 missing)
after that i run zpool import

Code:
  pool: ZFS_1
    id: 3333831992289118105
  state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
    devices and try again.
  see: http://www.sun.com/msg/ZFS-8000-3C
config:
 
    ZFS_1                                          UNAVAIL  insufficient replicas
      raidz1-0                                      UNAVAIL  insufficient replicas
        18446742974716428848                        UNAVAIL  cannot open
        14795978872498413746                        UNAVAIL  cannot open
        gptid/213f572c-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/21850ac3-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/21c6dcab-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/2207cba7-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/2246ff52-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/22877842-a71c-11e2-99f5-001e6777ba59  ONLINE


so i run addition test
mfiutil -u 0 show drives

Code:
mfi0 Physical Drives:
0 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0427074> SATA S0
1 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0517661> SATA S1
2 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0517687> SATA S2
3 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0517618> SATA S3
4 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0512595> SATA S4
5 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0512590> SATA S5
6 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0517637> SATA S6
7 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0326719> SATA S7


it seem everything seem fine
missing drive are
Code:
0 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0427074> SATA S0[mfisyspd0]
1 ( 2794G) JBOD <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0517661> SATA S1[mfisyspd1]


so i shutdown and remove ZFS_2 (H310 raid card and 8 Hdd[mfisyspd8-15])
to reduce confused

when booting up found some error msg
so i ran dmesg
getting these msg http://pastebin.com/xexX9YaJ

zpool status
Code:
  pool: ZFS_3
state: ONLINE
  scan: scrub repaired 0 in 2h5m with 0 errors on Sun Jun  2 02:05:31 2013
config:
 
    NAME                                            STATE    READ WRITE CKSUM
    ZFS_3                                          ONLINE      0    0    0
      raidz1-0                                      ONLINE      0    0    0
        gptid/6044c5e2-a71c-11e2-99f5-001e6777ba59  ONLINE      0    0    0
        gptid/60a008a4-a71c-11e2-99f5-001e6777ba59  ONLINE      0    0    0
        gptid/60fe0c4e-a71c-11e2-99f5-001e6777ba59  ONLINE      0    0    0
        gptid/61638bfc-a71c-11e2-99f5-001e6777ba59  ONLINE      0    0    0


zpool import

Code:
  pool: ZFS_1
    id: 3333831992289118105
  state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
    devices and try again.
  see: http://www.sun.com/msg/ZFS-8000-3C
config:
 
    ZFS_1                                          UNAVAIL  insufficient replicas
      raidz1-0                                      UNAVAIL  insufficient replicas
        18446742974716428848                        UNAVAIL  cannot open
        14795978872498413746                        UNAVAIL  cannot open
        gptid/213f572c-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/21850ac3-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/21c6dcab-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/2207cba7-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/2246ff52-a71c-11e2-99f5-001e6777ba59  ONLINE
        gptid/22877842-a71c-11e2-99f5-001e6777ba59  ONLINE


gpart show
Code:
=>        34  5860533101  mfisyspd2  GPT  (2.7T) [CORRUPT]
          34          94            - free -  (47k)
        128    4194304          1  freebsd-swap  (2.0G)
    4194432  5856338703          2  freebsd-zfs  (2.7T)
 
=>        34  5860533101  mfisyspd3  GPT  (2.7T) [CORRUPT]
          34          94            - free -  (47k)
        128    4194304          1  freebsd-swap  (2.0G)
    4194432  5856338703          2  freebsd-zfs  (2.7T)
 
=>        34  5860533101  mfisyspd4  GPT  (2.7T) [CORRUPT]
          34          94            - free -  (47k)
        128    4194304          1  freebsd-swap  (2.0G)
    4194432  5856338703          2  freebsd-zfs  (2.7T)
 
=>        34  5860533101  mfisyspd5  GPT  (2.7T) [CORRUPT]
          34          94            - free -  (47k)
        128    4194304          1  freebsd-swap  (2.0G)
    4194432  5856338703          2  freebsd-zfs  (2.7T)
 
=>        34  5860533101  mfisyspd6  GPT  (2.7T) [CORRUPT]
          34          94            - free -  (47k)
        128    4194304          1  freebsd-swap  (2.0G)
    4194432  5856338703          2  freebsd-zfs  (2.7T)
 
=>        34  5860533101  mfisyspd7  GPT  (2.7T) [CORRUPT]
          34          94            - free -  (47k)
        128    4194304          1  freebsd-swap  (2.0G)
    4194432  5856338703          2  freebsd-zfs  (2.7T)
 
=>        34  5860533101  ada0  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>        34  5860533101  ada1  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>        34  5860533101  ada2  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>      63  234441585  ada3  MBR  (111G)
        63    1930257    1  freebsd  [active]  (942M)
    1930320        63        - free -  (31k)
    1930383    1930257    2  freebsd  (942M)
    3860640      3024    3  freebsd  (1.5M)
    3863664      41328    4  freebsd  (20M)
    3904992  230536656        - free -  (110G)
 
=>        34  5860533101  ada4  GPT  (2.7T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  5856338696    2  freebsd-zfs  (2.7T)
  5860533128          7        - free -  (3.5k)
 
=>      0  1930257  ada3s1  BSD  (942M)
        0      16          - free -  (8.0k)
      16  1930241      1  !0  (942M)


gpart list : http://pastebin.com/YYjdHGVs

in gpart show , gpart list it not showing mfisyspd0,mfisyspd1

i tried zpool import ZFS_1 with zpool import ZFS_1 and zpool import -F ZFS_1
getting same error
Code:
cannot import 'ZFS_1': no such pool or dataset
    Destroy and re-create the pool from
    a backup source.


i searching for fix issue and found someone suggest
i plug-out mfisyspd0 <WDC WD30EFRX-68A 0A80 serial=WD-WCC1T0427074>[ada0] and new WD RED 3Tb[ada1] into another PC
booting with freenas usb to ran

recoverdisk -w worklist /dev/ada0 /dev/ada1
after ran for something like 2 minute got these error

Code:
GPT rejected -- may not be recoverable.
corrupt or invalid GPT detected.


anyone got idea for rescue my server please

PS.1.already tried to swap raid card,cable anything and still getting same error(issue cause from 2 hdd for sure)
2.sorry for my bad english
 
William Grzybowski

William Grzybowski

Wizard
iXsystems
Joined
May 27, 2011
Messages
1,754
Paste sysctl kern.disks

It looks like the GPT partitions simply vanished from the first two disks. Probably wwhy we recommend using HBA and not raid controllers.
You could try to backup the gpt from mfisyspd2 to mfisyspd0 and mfisyspd1
 
J

jpaetzel

Guest
My suggestion would be to run mfiutil show volumes. There should be 8 single drive units exported and there looks like there are only 6. Copying over the GPT won't help if the RAID card isn't exporting the volumes. You also may want to get LSI support involved.
 
S

Seattle

Cadet
Joined
Oct 16, 2012
Messages
8
OP, did you ever get this resolved. seems similar to an issue I'm having.
 
Status
Not open for further replies.

Similar threads

T
  • Locked
ZPOOL degraded, hdd failed and so did my attempt to replace
Replies
3
Views
2K
paleoN
paleoN
M
zpool disks showing offline
Replies
1
Views
1K
mroptman
mroptman
L
  • Locked
Issues with two drives, looking for some help.
Replies
8
Views
1K
lpittman
L
D
  • Locked
replace hard drive
Replies
5
Views
1K
paleoN
paleoN
D
  • Locked
ZFS Failure
2
Replies
23
Views
3K
titan_rw
T
Top