Destroy one mirror in multi-mirrored pool?

[yonkoc]

Hello,

simple question. Here's my zpool:

Code:

pool: HomeNAS
 state: ONLINE
 scrub: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	HomeNAS     ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada0    ONLINE       0     0     0
	    ada1    ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada2    ONLINE       0     0     0
	    ada3    ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada4    ONLINE       0     0     0
	    ada10    ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada6    ONLINE       0     0     0
	    ada7    ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada8    ONLINE       0     0     0
	    ada9    ONLINE       0     0     0

After mucking around with it, trying to remove the ada4 and ada10 mirror I ended up with this:

Code:

pool: HomeNAS
 state: ONLINE
 scrub: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	HomeNAS     ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada0    ONLINE       0     0     0
	    ada1    ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada2    ONLINE       0     0     0
	    ada3    ONLINE       0     0     0
	  ada10     ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada6    ONLINE       0     0     0
	    ada7    ONLINE       0     0     0
	  mirror    ONLINE       0     0     0
	    ada8    ONLINE       0     0     0
	    ada9    ONLINE       0     0     0
	  ada4      ONLINE       0     0     0

How can I remove ada4 and ada10. Here's what I've tried so far:

Code:

zpool detach HomeNAS ada10
cannot detach ada10: only applicable to mirror and replacing vdevs

Code:

zpool add HomeNAS mirror ada4 ada10
invalid vdev specification
use '-f' to override the following errors:
/dev/ada4 is part of active pool 'HomeNAS'

Code:

zpool add -f HomeNAS mirror ada4 ada10
invalid vdev specification
the following errors must be manually repaired:
/dev/ada4 is part of active pool 'HomeNAS'

Code:

zpool add -f HomeNAS mirror ada4 ada5
invalid vdev specification
the following errors must be manually repaired:
/dev/ada4 is part of active pool 'HomeNAS'

My end goal is to have the ada10 as a spare (I know how to add it, but need it removed first) and have the ada4 and ada5 to be a mirror inside the spool.

The pool has 3TB of data on it so can't destroy it, or can I. I don't want the data lost.


Thanks,

Better yet, how do I format ada4 and ada10 under shell?

 

[Milhouse]

I could be wrong, but haven't you already trashed your pool by deleting the vdev?

Your data in the pool is striped across all of your vdevs so removing one vdev removes the associated data from your pool, leaving the pool incomplete. When you originally had 5 vdevs, your 3TB of data should have been spread fairly evenly across all the vdevs, so removing one entire vdev has removed approximately 1/5th of the data from your pool...

At least, that's how I understand it works - once a vdev is added to a pool, it can never usually be removed.

 

[yonkoc]

I could be wrong, but haven't you already trashed your pool by deleting the vdev?

Your data in the pool is striped across all of your vdevs so removing one vdev removes the associated data from your pool, leaving the pool incomplete. When you originally had 5 vdevs, your 3TB of data should have been spread fairly evenly across all the vdevs, so removing one entire vdev has removed approximately 1/5th of the data from your pool...

At least, that's how I understand it works - once a vdev is added to a pool, it can never usually be removed.

Actually my data is still there, now I cannot AutoImport the pool under GUI, but I can under CLI via SSH. The problem is this:
Gpart show does not see the pool, it sees the Freenas partitions
Gpart /dev/ada? (? any of the 11 drives) says no such device
dmesg | grep ada4 show everything okay
dmesg | grep ada10 shows that primary GPT is corrupt and unreadable, using secondary. Recovery strongly suggested.
pulling the cable on either ada4 or ada10 offlines makes the entire pool unavailable.

At present I'm trying to load gparted live USB so I can see if I can recover that partition for ada10. Logging in via SSH and browsing to data folder data shows is there. So, if I can't restore GPT I'll get a 3TB drive and offload onto it, then rebuild array and restore data. See if that helps.

 

[Milhouse]

Actually my data is still there

All of it? I'd be surprised, unless the deleted vdev was added to the pool after you had loaded it with data (which could, I suppose, explain why it contained no data).

Unfortunately I'm not able to advise what is now wrong with your drives/partitions/pools, but I hope you have a backup! :)

 

[yonkoc]

Okay, playing around with many things right now trying to see if I can recover the data but here's my question. I'm putting a link of the zdb -C HomeNAS output. I need to know if anyone knows what needs to be done to get from point A to point B. I know all GUIDs would have to be changed but how and where to begin I have no clue. Has anyone done that?

I wanted to add this if it would shed more light.

As one of my commands was spool detach HomeNAS ada4 ada10 it managed to get rid of the ada4 I think, that's why it now probably has only 35kb of data.

 

[William Grzybowski]

I might be wrong but i dont think you can actually do that...

you _might_ try zpool offline pool devicename and then detach but i dont think it will work, ZFS just is not supposed to work like that