Reinstall On USB Lost 1 Zpool

Status
Not open for further replies.

bmh.01

Explorer
Joined
Oct 4, 2013
Messages
70
Reinstalled my system onto a usb thumb drive with a view to going mirrored instead of using 2x small HDDs. Took a backup of the config, shutdown, removed the two small HDDs, replaced with USB drives, reinstalled and restored the config.

2 of the 3 zpools has come back just fine, one however is missing and refuses to import siting missing devices which as far as I can see are only the l2arc drives (which are present and correct as well!).

I'm lost on this one at the moment, afaik a pool should import without l2arc if forced, which this doesn't. All mirror drives are there and online and all i've done is reinstall and restore the config which worked perfectly before.

Code:
[root@homevmsan1] ~# zpool import
   pool: VMSAN1-ZP2
     id: 16009759259503035717
  state: UNAVAIL
status: The pool was last accessed by another system.
action: The pool cannot be imported due to damaged devices or data.
   see: http://illumos.org/msg/ZFS-8000-EY
config:

        VMSAN1-ZP2                                      UNAVAIL  missing device
          mirror-0                                      ONLINE
            gptid/82c37d3c-a338-11e4-94a6-0015174b4449  ONLINE
            gptid/831e2347-a338-11e4-94a6-0015174b4449  ONLINE
          mirror-1                                      ONLINE
            gptid/a7e13348-a338-11e4-94a6-0015174b4449  ONLINE
            gptid/a845d339-a338-11e4-94a6-0015174b4449  ONLINE
          mirror-2                                      ONLINE
            gptid/ef089139-a338-11e4-94a6-0015174b4449  ONLINE
            gptid/ef695312-a338-11e4-94a6-0015174b4449  ONLINE
        cache
          gptid/4ba42e91-a339-11e4-94a6-0015174b4449
          gptid/4bd38ff7-a339-11e4-94a6-0015174b4449


Code:
[root@homevmsan1] ~# gpart show
=>        34  1953525101  da0  GPT  (931G)
          34          94       - free -  (47k)
         128     4194304    1  freebsd-swap  (2.0G)
     4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128           7       - free -  (3.5k)

=>        34  1953525101  da1  GPT  (931G)
          34          94       - free -  (47k)
         128     4194304    1  freebsd-swap  (2.0G)
     4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128           7       - free -  (3.5k)

=>        34  1953525101  da2  GPT  (931G)
          34          94       - free -  (47k)
         128     4194304    1  freebsd-swap  (2.0G)
     4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128           7       - free -  (3.5k)

=>        34  1953525101  da3  GPT  (931G)
          34          94       - free -  (47k)
         128     4194304    1  freebsd-swap  (2.0G)
     4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128           7       - free -  (3.5k)

=>        34  1953525101  da4  GPT  (931G)
          34          94       - free -  (47k)
         128     4194304    1  freebsd-swap  (2.0G)
     4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128           7       - free -  (3.5k)

=>        34  1953525101  da5  GPT  (931G)
          34          94       - free -  (47k)
         128     4194304    1  freebsd-swap  (2.0G)
     4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128           7       - free -  (3.5k)

=>       34  117231341  da6  GPT  (55G)
         34         94       - free -  (47k)
        128  117231240    1  freebsd-zfs  (55G)
  117231368          7       - free -  (3.5k)

=>       34  234441581  da7  GPT  (111G)
         34         94       - free -  (47k)
        128  234441480    1  freebsd-zfs  (111G)
  234441608          7       - free -  (3.5k)

=>        34  1953525101  da8  GPT  (931G)
          34          94       - free -  (47k)
         128     4194304    1  freebsd-swap  (2.0G)
     4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128           7       - free -  (3.5k)

=>        34  1953525101  da9  GPT  (931G)
          34          94       - free -  (47k)
         128     4194304    1  freebsd-swap  (2.0G)
     4194432  1949330696    2  freebsd-zfs  (929G)
  1953525128           7       - free -  (3.5k)

=>        34  1953522988  da10  GPT  (931G)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  1949328584     2  freebsd-zfs  (929G)
  1953523016           6        - free -  (3.0k)

=>        34  1953525101  da11  GPT  (931G)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  1949330696     2  freebsd-zfs  (929G)
  1953525128           7        - free -  (3.5k)

=>        34  1953525101  da12  GPT  (931G)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  1949330696     2  freebsd-zfs  (929G)
  1953525128           7        - free -  (3.5k)

=>        34  1953525101  da13  GPT  (931G)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  1949330696     2  freebsd-zfs  (929G)
  1953525128           7        - free -  (3.5k)

=>      34  15633341  da14  GPT  (7.5G)
        34      1024     1  bios-boot  (512k)
      1058         6        - free -  (3.0k)
      1064  15632304     2  freebsd-zfs  (7.5G)
  15633368         7        - free -  (3.5k)

=>        34  1250263661  ada0  GPT  (596G)
          34         222        - free -  (111k)
         256  1250247055     1  !6a898cc3-1dd2-11b2-99a6-080020736631  (596G)
  1250247311       16384     9  !6a945a3b-1dd2-11b2-99a6-080020736631  (8.0M)

=>        34  1250263661  ada1  GPT  (596G)
          34         222        - free -  (111k)
         256  1250247055     1  !6a898cc3-1dd2-11b2-99a6-080020736631  (596G)
  1250247311       16384     9  !6a945a3b-1dd2-11b2-99a6-080020736631  (8.0M)

=>        34  1250263661  ada2  GPT  (596G)
          34         222        - free -  (111k)
         256  1250247055     1  !6a898cc3-1dd2-11b2-99a6-080020736631  (596G)
  1250247311       16384     9  !6a945a3b-1dd2-11b2-99a6-080020736631  (8.0M)

=>        34  1250263661  ada3  GPT  (596G)
          34         222        - free -  (111k)
         256  1250247055     1  !6a898cc3-1dd2-11b2-99a6-080020736631  (596G)
  1250247311       16384     9  !6a945a3b-1dd2-11b2-99a6-080020736631  (8.0M)

=>       34  117231341  da15  GPT  (55G)
         34         94        - free -  (47k)
        128  117231240     1  freebsd-zfs  (55G)
  117231368          7        - free -  (3.5k)

=>       34  117231341  da16  GPT  (55G)
         34         94        - free -  (47k)
        128  117231240     1  freebsd-zfs  (55G)
  117231368          7        - free -  (3.5k)


Code:
gptid/aea1741c-a3fc-11e4-94a6-0015174b4449     N/A  da0p2
gptid/aefd2527-a3fc-11e4-94a6-0015174b4449     N/A  da1p2
gptid/f779086d-a3fc-11e4-94a6-0015174b4449     N/A  da2p2
gptid/f7d3e7ea-a3fc-11e4-94a6-0015174b4449     N/A  da3p2
gptid/310f3342-a3fd-11e4-94a6-0015174b4449     N/A  da4p2
gptid/316c4492-a3fd-11e4-94a6-0015174b4449     N/A  da5p2
gptid/af5e6eb4-a3fc-11e4-94a6-0015174b4449     N/A  da6p1
gptid/af35fbb6-a3fc-11e4-94a6-0015174b4449     N/A  da7p1
gptid/82b7cc1d-a338-11e4-94a6-0015174b4449     N/A  da8p1
gptid/82c37d3c-a338-11e4-94a6-0015174b4449     N/A  da8p2
gptid/831069c4-a338-11e4-94a6-0015174b4449     N/A  da9p1
gptid/831e2347-a338-11e4-94a6-0015174b4449     N/A  da9p2
gptid/a7cd94e7-a338-11e4-94a6-0015174b4449     N/A  da10p1
gptid/a7e13348-a338-11e4-94a6-0015174b4449     N/A  da10p2
gptid/a8385285-a338-11e4-94a6-0015174b4449     N/A  da11p1
gptid/a845d339-a338-11e4-94a6-0015174b4449     N/A  da11p2
gptid/eefe2bfd-a338-11e4-94a6-0015174b4449     N/A  da12p1
gptid/ef089139-a338-11e4-94a6-0015174b4449     N/A  da12p2
gptid/ef5c5812-a338-11e4-94a6-0015174b4449     N/A  da13p1
gptid/ef695312-a338-11e4-94a6-0015174b4449     N/A  da13p2
gptid/299c1da2-a598-11e4-81f5-0015174b4449     N/A  da14p1
gptid/29a531b3-a598-11e4-81f5-0015174b4449     N/A  da14p2
gptid/09d344eb-ff4c-97c2-bb4f-d2dbaa8b01f9     N/A  ada0p1
gptid/0ed5952f-f6b8-b668-af24-e99a9db01049     N/A  ada0p9
gptid/1ea02dbf-2b9c-42cf-e78f-8b4d71af3a27     N/A  ada1p1
gptid/69776e9e-ea9e-f3ef-e5f1-e4126335a984     N/A  ada1p9
gptid/538a93cc-8a20-07e3-e617-979c5e36645a     N/A  ada2p1
gptid/3ba934f7-3c8b-6dec-d28a-d5683413437b     N/A  ada2p9
gptid/669592af-d347-b4ce-bc9a-b96761158351     N/A  ada3p1
gptid/7755647f-dd1b-a9c6-d515-9712b034ab4e     N/A  ada3p9
gptid/4bd38ff7-a339-11e4-94a6-0015174b4449     N/A  da15p1
gptid/4ba42e91-a339-11e4-94a6-0015174b4449     N/A  da16p1
 

bmh.01

Explorer
Joined
Oct 4, 2013
Messages
70
Oh the SSD l2arc drives are da15 da16 in that code, the usb stick is da14 usually it would be the other way around but I tried booting with the l2arc drives connected to see if it would make any difference (which it didn't).

Also I can see the system accessing the cache drives when I try zpool import or even zpool import -f.
 

bmh.01

Explorer
Joined
Oct 4, 2013
Messages
70
Well ive given up blown the pool away and am currently restoring from backups.

The biggest question would be what did I do wrong?

Should I have exported the pools first?

In my head it wasn't necessary as it was going to be the same system after restoring the config.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
No clue. An l2arc being missing (or even corrupt) will not stop a pool from importing. There's insufficient information to really explain the original problem. That bug won't affect the importability of the pool. In fact, that bug was particular insidious because ZFS will be writing exactly what you tell it to, it's just that you'll be telling ZFS to store junk. He also validated this as he provided the output of zpool status. :)
 

bmh.01

Explorer
Joined
Oct 4, 2013
Messages
70
All the response were from me btw ;).

I think the devil is in the detail and the status shown in action where it says pool cannot be imported due to damaged devices or data is the action problem, that or i've stumbled across a very wierd bug as i've no idea how the data vdevs would have got damaged as they were working perfect before the clean shutdown to reinstall on usb.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Every time I've seen missing devices it's been because someone added a disk or device and it wasn't available. zdb outputs usually confirm that more devices are missing from the pool. It's a bit late for zdb outputs though.
 

bmh.01

Explorer
Joined
Oct 4, 2013
Messages
70
I didn't realise zdb would work on a non active pool.

I would've liked to have seen if I could have got the pool back as an academic exercise but unfortunately needing it working took a priority.

Going to have to reinstall again due to an unforseen need to change the motherboard of this system so plenty of chance to hose it again [emoji1].
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
You don't need to reinstall with a change of hardware.
 

bmh.01

Explorer
Joined
Oct 4, 2013
Messages
70
Was wondering that. Nics will chance am I best off just deleting the config first so they'll all drop back to dhcp?
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
That is an option. You can also fix it from the menu options after you go to the new hardware.
 
Status
Not open for further replies.
Top