Change ashift on mirrored-vdev?

[justryan]

Hello,

I have two problems:
1. I need to remove two disks from my pool (a mirrored vdev)
2. I cannot do that because of an ashift mismatch

Here is what my pool looks like:

Code:

pool
  mirror-0
    disk1 (ashift 12, 5.5tb)
    disk2 (ashift 12, 5.5tb)
  mirror-1
    disk3 (ashift 12, 5.5tb)
    disk4 (ashift 12, 5.5tb)
  mirror-2
    disk5 (non-native ashift 9, 12.5tb) block size: 512B configured, 4096B native
    disk6 (non-native ashift 9, 12.5tb) block size: 512B configured, 4096B native

The default ashift value for my pool was 0, so I guess the mirrored-vdev with an asize of 9 got a misdetected size at some point, and now that is visible.

I cannot remove the mirrored-vdev from the pool due to lack of space:

Code:

# zpool remove datatank mirror-2
cannot remove mirror-2: out of space

which is interesting because zpool list shows 10.4T allocated (rebalance needed?)

I wanted to remove the two big disks from mirror-2, add them back with ashift=12, to allow myself to remove mirror-0.

(Unfortunately I do not have space elsewhere to copy the data.)

Can anyone think of a creative way of making ashift=12?

Thank you.

 

[Arwen]

It appears that you really don't have enough space to perform the Mirrored vDev removal. Even though you think 5.5TB + 5.5TB = 11TB, which is more than the 10.4TB allocated, it appears ZFS wants some more space for overhead or such.

I don't know of any way to change a vDev's ashift value after creation.

 

[justryan]

I freed up some space. mirror-0 now has 1.3T FREE, mirror-1 has 1.3T FREE.
mirror-2 has 1.15T ALLOC. The remove still fails. How can I see why?