Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

Unable to re-add a removed drive from mirrored pool

Western Digital Drives - The Preferred Drives of FreeNAS and TrueNAS CORE

amlamarra

Member
Joined
Feb 24, 2017
Messages
38
So I was replacing 1 of my 2 mirrored drives in a pool. Through the GUI I set the drive to offline then physically replaced the drive with another of the same size. However, back at the GUI, instead of selecting "Replace" on the drive's drop-down menu, I selected "Online." After doing that, instead of seeing "da0p2" for the name, it showed the GUID. I was unable to use the "offline" or "replace" options after that. This may not have been the best idea, but I ended up removing the drive from the pool. Now all I've got is the one drive.

I tried following some guides to re-add the drive to the pool.

However, I get this error:
Code:
# zpool attach storage /dev/gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc /dev/gptid/b16e8d1d-1389-11eb-b748-6805ca4284fc
cannot attach /dev/gptid/b16e8d1d-1389-11eb-b748-6805ca4284fc to /dev/gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc: can only attach to mirrors and top-level disks


What am I doing wrong?
 
Last edited:

amlamarra

Member
Joined
Feb 24, 2017
Messages
38
Just wanted to show the current output of a few commands:

Code:
# zpool status storage
  pool: storage
 state: ONLINE
status: One or more devices are configured to use a non-native block size.
        Expect reduced performance.
action: Replace affected devices with devices that support the
        configured block size, or migrate data to a properly configured
        pool.
  scan: scrub repaired 0B in 03:02:13 with 0 errors on Thu Oct 22 16:36:35 2020
config:

        NAME                                          STATE     READ WRITE CKSUM
        storage                                       ONLINE       0     0     0
          gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc  ONLINE       0     0     0

errors: No known data errors


Code:
# glabel status
                                      Name  Status  Components
gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc     N/A  da1p2
gptid/25e54ffc-2fef-11e9-9b9f-001a4b44b638     N/A  da2p2
gptid/b22f3939-39f4-11ea-96c3-6805ca4284fc     N/A  da3p1
gptid/b31d6d61-39f4-11ea-96c3-6805ca4284fc     N/A  da4p1
gptid/35f9a725-14d1-11eb-b748-6805ca4284fc     N/A  da0p1
gptid/39a8116a-14d1-11eb-b748-6805ca4284fc     N/A  da0p2


Code:
# gpart list
Geom name: da1
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 3907029127
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: da1p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,dcc67762-e994-11ea-92bd-6805ca4284fc,0x80,0x400000)
   rawuuid: dcc67762-e994-11ea-92bd-6805ca4284fc
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: da1p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e2
   efimedia: HD(2,GPT,dcdb9427-e994-11ea-92bd-6805ca4284fc,0x400080,0xe8a08808)
   rawuuid: dcdb9427-e994-11ea-92bd-6805ca4284fc
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: da1
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5

...

Geom name: da0
modified: false
state: CORRUPT
fwheads: 255
fwsectors: 63
last: 3907029127
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: da0p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   efimedia: HD(1,GPT,35f9a725-14d1-11eb-b748-6805ca4284fc,0x80,0x400000)
   rawuuid: 35f9a725-14d1-11eb-b748-6805ca4284fc
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: da0p2
   Mediasize: 1998251364352 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   efimedia: HD(2,GPT,39a8116a-14d1-11eb-b748-6805ca4284fc,0x400080,0xe8a08808)
   rawuuid: 39a8116a-14d1-11eb-b748-6805ca4284fc
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 1998251364352
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 3907029127
   start: 4194432
Consumers:
1. Name: da0
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
 

amlamarra

Member
Joined
Feb 24, 2017
Messages
38
I've seen other forum posts or blog articles indicate that I need to use the device's name. But no matter what name I try to use, I get the message no such device in pool.

For instance:
Code:
# zpool attach storage /dev/da1p2 gpt/da0_part2
cannot attach gpt/da0_part2 to /dev/da1p2: no such device in pool


I've even tried adding a label to da1p2 and using that. No luck.
 

sretalla

Dedicated Sage
Joined
Jan 1, 2016
Messages
3,028
zpool attach storage /dev/da1p2 gpt/da0_part2
For a start, there's no such partition as gpt/da0_part2

If you were going to add it that way, it would be /dev/da0p2 (but don't do that).

What you want is to add it with gptid/39a8116a-14d1-11eb-b748-6805ca4284fc, adding it to gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc

Don't put /dev in front of gptids.

So finally like this:

zpool attach storage gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc gptid/39a8116a-14d1-11eb-b748-6805ca4284fc
 

amlamarra

Member
Joined
Feb 24, 2017
Messages
38
Thank you for the help.

Sorry, forgot to mention that the last time I tried this, I created the partition with a label:
Code:
root@freenas ~ # gpart destroy -F /dev/da0
da0 destroyed
root@freenas ~ # gpart create -s gpt /dev/da0
da0 created
root@freenas ~ # gpart add -i 1 -b 128 -t freebsd-swap -s 2g /dev/da0
da0p1 added
root@freenas ~ # gpart add -i 2 -t freebsd-zfs -l da0_part2 /dev/da0
da0p2 added
root@freenas ~ # glabel status
                                      Name  Status  Components
gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc     N/A  da1p2
gptid/25e54ffc-2fef-11e9-9b9f-001a4b44b638     N/A  da2p2
gptid/b22f3939-39f4-11ea-96c3-6805ca4284fc     N/A  da3p1
gptid/b31d6d61-39f4-11ea-96c3-6805ca4284fc     N/A  da4p1
gptid/5d8783e8-17b9-11eb-b5fb-6805ca4284fc     N/A  da0p1
                             gpt/da0_part2     N/A  da0p2
gptid/6ef7c5a2-17b9-11eb-b5fb-6805ca4284fc     N/A  da0p2
root@freenas ~ # zpool attach storage gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc gptid/6ef7c5a2-17b9-11eb-b5fb-6805ca4284fc
cannot attach gptid/6ef7c5a2-17b9-11eb-b5fb-6805ca4284fc to gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc: can only attach to mirrors and top-level disks


And what's weird is that last time, the glabel status command didn't show the gptid of that partition. And even stranger is that this time, I run it a second time, and now it won't show the label:
Code:
root@freenas ~ # glabel status
                                      Name  Status  Components
gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc     N/A  da1p2
gptid/25e54ffc-2fef-11e9-9b9f-001a4b44b638     N/A  da2p2
gptid/b22f3939-39f4-11ea-96c3-6805ca4284fc     N/A  da3p1
gptid/b31d6d61-39f4-11ea-96c3-6805ca4284fc     N/A  da4p1
gptid/5d8783e8-17b9-11eb-b5fb-6805ca4284fc     N/A  da0p1
gptid/6ef7c5a2-17b9-11eb-b5fb-6805ca4284fc     N/A  da0p2


Regardless, it doesn't seem to think that da1p2 is a top-level disk.
 

Patrick M. Hausen

Dedicated Sage
Joined
Nov 25, 2013
Messages
1,912
Don't use device names like da1p2 for your zpool commands unless it is the boot-pool you are manipulating. Always refer to disks as e.g gptid/dcdb9427-e994-11ea-92bd-6805ca4284fc. Don't use manual GPT labels, either.

TrueNAS uses the gptid/uuid scheme only internally, so please stick to these.
 
Top