Replace/Remove unavail vdev is not possible due to inability to import pool

[Steak]

All right, after several days of searching for a fix and learning hours and hours about zfs in general and finally messing up a bit with good help on stackexchange i decided to reach out to you guys.
FYI i'm quiet new to this whole zfs thing.

Basically i had a esxi on which i had a freenas vm with 2 hdd's mapped into the freenas vm with raw device mapping. After the boot usb drive failed i also couldn't get the ssd on which the freenas boot was stored get to import into the new esxi installation and now there i am, installing a fresh freenas in which i can't import my pool.

zpool import has following results:

Code:

   pool: datastore
     id: 9178951695516496991
  state: UNAVAIL
 status: The pool was last accessed by another system.
 action: The pool cannot be imported due to damaged devices or data.
   see: http://illumos.org/msg/ZFS-8000-EY
 config:

        datastore                                       UNAVAIL  insufficient replicas
          mirror-0                                      DEGRADED
            3073530158529594642                         UNAVAIL  cannot open
            gptid/381bc8c5-f2d4-11e5-a788-000c292268d7  ONLINE
          15540268895717230009                          UNAVAIL  cannot open

Now, what i already messed up is the '3073530158529594642' entry which originally was a 'gptid/4efeae5c-6fbe-11e6-a255-000c292268d7'.
My original configuration was 2x3tb on each one 2tb mirrored and the other tb striped together.

If i'm correct, the mirror is fine, however the stripe is broken. I don't need the stripe, i just want to restore the mirror and i would be fine with that.

Now the problem is, that i dont know how.
'zpool remove datastore 15540268895717230009' results in a 'cannot open 'datastore': no such pool'.

Same with replacing the hdd with the guid '3073530158529594642' which i messed up.
Importing isn't working because of either 'cannot import 'datastore': pool may be in use from other system, it was last accessed by...'
or with -f arg

Code:

cannot import 'datastore': no such pool or dataset
        Destroy and re-create the pool from
        a backup source.

I hope there is a solution to this, my girlfriend is going to kill me if all the pictures are gone...

 

[myoung]

My original configuration was 2x3tb on each one 2tb mirrored and the other tb striped together.

WHY?!?!

I don't need the stripe, i just want to restore the mirror and i would be fine with that.

You do need the stripe, the stripe is part of the pool. Without the stripe, your pool is dead

 

[Steak]

WHY?!?!

Because im poor

You do need the stripe, the stripe is part of the pool. Without the stripe, your pool is dead

And how could i reconfigure it?

 

[HoneyBadger]

Basically i had a esxi on which i had a freenas vm with 2 hdd's mapped into the freenas vm with raw device mapping.

Local RDM is disabled by default on ESXi because it often leads to Bad Things. You're seeing the results of this now, I believe.

zpool import has following results:

Code:

   pool: datastore
     id: 9178951695516496991
  state: UNAVAIL
status: The pool was last accessed by another system.
action: The pool cannot be imported due to damaged devices or data.
   see: http://illumos.org/msg/ZFS-8000-EY
config:

        datastore                                       UNAVAIL  insufficient replicas
          mirror-0                                      DEGRADED
            3073530158529594642                         UNAVAIL  cannot open
            gptid/381bc8c5-f2d4-11e5-a788-000c292268d7  ONLINE
          15540268895717230009                          UNAVAIL  cannot open

Your pool was not built correctly. You don't have "one mirror and a stripe" - you have one pool that consists of a mirror vdev and a single vdev. Top-level data vdevs are always striped across. Losing 15540268895717230009 results in "dead pool."

Now, what i already messed up is the '3073530158529594642' entry which originally was a 'gptid/4efeae5c-6fbe-11e6-a255-000c292268d7'.
My original configuration was 2x3tb on each one 2tb mirrored and the other tb striped together.

If i'm correct, the mirror is fine, however the stripe is broken. I don't need the stripe, i just want to restore the mirror and i would be fine with that.

Now the problem is, that i don't know how.
'zpool remove datastore 15540268895717230009' results in a 'cannot open 'datastore': no such pool'.

Same with replacing the hdd with the guid '3073530158529594642' which i messed up.

You need to replace the RDM that was supplying 15540268895717230009. If you can't, your pool is dead.

I hope there is a solution to this, my girlfriend is going to kill me if all the pictures are gone...

http://www.ftdflorists.com/

 

[pro lamer]

all the pictures are gone...

PhotoRecWin or something similar :/

Sent from my phone

 

[Steak]

PhotoRecWin or something similar :/

Gonna try that today.

It's strange though, that zfs is build that way.

I kind of don't understand, why on earth it does recognize the mirror but not the stripe.
There were only those both drives i mentioned...

Local RDM is disabled by default on ESXi because it often leads to Bad Things. You're seeing the results of this now, I believe.

It's not disabled, rather not easily accessible.
And why should it lead to bad things? ZFS as far as i understand does care about sn and guids for vdev mapping which are basically passed through with RDM hence the mirror is recognized correctly. I think this would be much worse with vmdk's, even if recoverable since guids are generated on mapping (not 100 % sure about that, but doesn't actually matter).

You need to replace the RDM that was supplying 15540268895717230009. If you can't, your pool is dead.

I'd love to, but there was no third vdev in the pool. There were only 2 hdd's which are correctly mapped into mirror-0 (well, not anymore, i messed one up). Also i can't replace the entry while the pool isn't imported and i can't import it, because i need to replace it...

http://www.ftdflorists.com/

:-D thanks for that one.

 

[HoneyBadger]

It's not disabled, rather not easily accessible.

Depending on the version of ESXi, you need to go poking about into a special menu and set a checkbox/flag that says "allow this behavior" - that to me says it's disabled. ;)

And why should it lead to bad things? ZFS as far as i understand does care about sn and guids for vdev mapping which are basically passed through with RDM hence the mirror is recognized correctly. I think this would be much worse with vmdk's, even if recoverable since guids are generated on mapping (not 100 % sure about that, but doesn't actually matter).

It has to do with the storage controller passing the SN/WWN/etc of the drive (referred to as "Vital Product Data") so that it can persist that GUID. If it doesn't, VMware might "helpfully" generate a pseudorandom GUID.

I'd love to, but there was no third vdev in the pool. There were only 2 hdd's which are correctly mapped into mirror-0 (well, not any more, i messed one up). Also i can't replace the entry while the pool isn't imported and i can't import it, because i need to replace it...

The way your pool layout is shown indicates that you do have a missing device at the top level; and also how the pool was described (two 3TB drives, partitioned into 2TB+1TB, 2TB partitions mirrored, 1TB partitions striped) there would be two pools with two vdevs in each.

I know one pool was named "datastore" but there should have been a second in this config.

Can you run the command zdb -eC datastore and post the output in [CODE] tags? If you remember your other pool name, run it against there. Or just try zdb -eC and see what comes up.