I can't mount my RaidZ Pool

[praetorian46]

Hello,

I was trying to add some additional disk to my ZFS Pool.
I wanted to rebalance the disks I have across multiple HBAs to have 1x SSD and 3x HDDs on each channel.
I rearanged the existing HDDs and added more HDDs and SSDs.

LIke a dummy I didn't write down the old location/ports I had the disks plugged into.
Now I get the following when trying to do an import:

Code:

# zpool import
   pool: Hill-San01_RaidZ
     id: 10579513820122726052
  state: UNAVAIL
 status: One or more devices are missing from the system.
 action: The pool cannot be imported. Attach the missing
    devices and try again.
   see: http://illumos.org/msg/ZFS-8000-6X
 config:

    Hill-San01_RaidZ                                UNAVAIL  missing device
      raidz2-0                                      ONLINE
        gptid/90edae67-9baf-11e5-8815-d8d385ae2580  ONLINE
        da5                                         ONLINE
        da4                                         ONLINE
        da3                                         ONLINE
        gptid/2dfe706c-626b-11e5-82f3-d8d385ae2580  ONLINE
        gptid/590e19cc-63ad-11e5-82f3-d8d385ae2580  ONLINE
        gptid/260672c4-6575-11e5-82f3-d8d385ae2580  ONLINE
        gptid/c4e9ea1c-689c-11e5-82f3-d8d385ae2580  ONLINE

    Additional devices are known to be part of this pool, though their
    exact configuration cannot be determined.

But all 9x disks are detected that were in the ZPool previously.
How do I make the zpool find the disk and use it?

Here are all the disks:

Code:

# camcontrol devlist 
<ATA TOSHIBA MD04ACA4 FP2A>        at scbus0 target 0 lun 0 (da14,pass14)
<ATA TOSHIBA MD04ACA4 FP2A>        at scbus0 target 1 lun 0 (da0,pass0)
<ATA TOSHIBA MD04ACA4 FP2A>        at scbus0 target 3 lun 0 (da1,pass1)
<ATA ST4000DM000-1F21 CC52>        at scbus0 target 4 lun 0 (da3,pass3)
<ATA ST4000DM000-1F21 CC52>        at scbus0 target 6 lun 0 (da4,pass4)
<ATA ST4000DM000-1F21 CC52>        at scbus0 target 7 lun 0 (da5,pass5)
<ATA HGST HDN724040AL A5E0>        at scbus1 target 0 lun 0 (da6,pass6)
<ATA HGST HDN724040AL A5E0>        at scbus1 target 1 lun 0 (da7,pass7)
<ATA ST4000DM000-1F21 CC52>        at scbus1 target 4 lun 0 (da2,pass2)

Any help would be great!

 

[praetorian46]

Come to think of it. I only had 8x disks in the Zpool I think. I messed up in creating it. And left one out originally. (Maybe...) Also. I had started creating the zpool with 2x partitions on the 4x TB disks. Then I replaced each partition with a 4TB disk so that I could shuffle my disks around and keep the same ZPool configuration. (7+2 I thought But I think it ended up being 6+2). So some of the disks in the zpool reference the full disk. And others reference a partition on the disk. I hope this helps.

This is for a home server with a bunch of Media on it. I'd be willing to pay a small amount of money for some help in this matter. But I'm not a company, so I can't afford full blown Professional Services.

Thank you!

 

[sretalla]

Well, the ports don't matter, as long as the OS can see the required disks, all will be OK.

Sorting out exactly which disks will be your focus, so you should consider first eliminating the new controller and then looking at the serial numbers of the drives in the FreeNAS GUI if your pool comes back in that situation, then reconnect the drives to the new controller in the mix that you wanted, making sure all the serial numbers are accounted for.

 

[praetorian46]

Thank you for your reply.
I did verify that all 9x disks with the correct serial numbers show up. And this is a RaidZ2 so even if 1x or 2x are missing. This should be fine. Correct?

Is there a way to dump which disks the ZPool is looking for to verify they are all accounted for?

 

[danb35]

And this is a RaidZ2 so even if 1x or 2x are missing.

...except that you've probably added a single disk striped into your pool, which means that if that disk is lost, your pool is lost with it.

 

[praetorian46]

I’m 99% sure I didn’t do this. And all disks that were there before are there now. How can I identify which disk(s) the raidz2 is asking for so I can ensure they are all available. I don’t have any failed disks to my knowledge.

 

[danb35]

I’m 99% sure I didn’t do this.

I'm about 99.9% sure you did--the language in the pool status saying "Additional devices are known to be part of this pool, though their exact configuration cannot be determined" pretty much guarantees it. And that's entirely consistent with the 8-disk RAIDZ2 your pool status output is showing, combined with your belief that the pool contained nine disks (and indeed, the only thing I can think of that is consistent with what you're saying).

How can I identify which disk(s) the raidz2 is asking for so I can ensure they are all available.

I'm not aware of any way to do this, though it's possible zdb could be used to give you this information. Are there any disks other than those nine?

 

[praetorian46]

I removed 3x older SSDs that used to be ZIL and L2ARC. But 2x of those SSDs failed so I removed the L2Arc and Zil some time ago (to not have the remaining ZIL fail unexpectedly). At least I though I did. I could add those SSDs back to the system. Would missing Zil or L2Arc cause this?

 

[Ericloewe]

LIke a dummy I didn't write down the old location/ports I had the disks plugged into.

That is completely irrelevant. No vaguely-usable RAID solution cares about ports, not even Intel FakeRAID.

Would missing Zil or L2Arc cause this?

No. You're missing data vdevs. The pool may be importable in FreeNAS 11.2 with some finessing and some data may even be readable, but any recent data is likely to be seriously mangled.

 

[praetorian46]

Hey folks. I was out of town last week. I wanted to give an update to this. When I got back home I added all 3x of my old SSDs (2x dead and 1x working) These were detected and it allowed me to mount my ZPool. I thought I added these SSDs as Zil and L2Arc. But I must have screwed up and added one of these as data disk. I am now backing up all of my data to a fresh RaidZ2 and once that is done I'll rebuild this RaidZ2 and I should be in good shape. Thanks for the help!