Pool offline after replacing disk

P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Ok, doing that now.
Fingers crossed.
(In bed on my iPad cause it’s 11pm here)
 

Attachments

  • CFD6CA07-B25A-4382-A1B2-AFDF18F5A42A.jpeg
    CFD6CA07-B25A-4382-A1B2-AFDF18F5A42A.jpeg
    117.1 KB · Views: 125
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
sretalla said:
Yes, if that's an option, go for it.
Click to expand...
Didn’t work sadly.
Code:
Error: Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/middlewared/job.py", line 367, in run
    await self.future
  File "/usr/local/lib/python3.8/site-packages/middlewared/job.py", line 403, in __run_body
    rv = await self.method(*([self] + args))
  File "/usr/local/lib/python3.8/site-packages/middlewared/schema.py", line 973, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/pool_/replace_disk.py", line 93, in replace
    enc_disks = await self.middleware.call(
  File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1238, in call
    return await self._call(
  File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1195, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/pool_/format_disks.py", line 56, in format_disks
    await asyncio_map(format_disk, disks.items(), limit=16)
  File "/usr/local/lib/python3.8/site-packages/middlewared/utils/asyncio_.py", line 16, in asyncio_map
    return await asyncio.gather(*futures)
  File "/usr/local/lib/python3.8/site-packages/middlewared/utils/asyncio_.py", line 13, in func
    return await real_func(arg)
  File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/pool_/format_disks.py", line 29, in format_disk
    await self.middleware.call(
  File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1238, in call
    return await self._call(
  File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1206, in _call
    return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
  File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1110, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
  File "/usr/local/lib/python3.8/site-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/disk_/format.py", line 25, in format
    raise CallError(f'Failed to wipe disk {disk}: {job.error}')
middlewared.service_exception.CallError: [EFAULT] Failed to wipe disk ada0: [EFAULT] Command dd if=/dev/zero of=/dev/ada0p2 bs=1M count=32 failed (code 1):
dd: /dev/ada0p2: Input/output error
28+0 records in
27+0 records out
28311552 bytes transferred in 572.102738 secs (49487 bytes/sec)
 

Attachments

  • 489600FC-5888-4B42-BA96-75D230C1C913.png
    489600FC-5888-4B42-BA96-75D230C1C913.png
    637.6 KB · Views: 127
sretalla

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
OK, I think we should just do it in CLI then.

zpool replace VOL-1gptid/existing_disk gptid/new_disk

Where you will need to replace existing_disk with the gptid from zpool status of the first (good) disk in mirror-1 and replace new_disk with the gptid we saved earlier from ada0p2
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Ok, so here are the GPTIDs from the earlier thing for new_disk:
gptid/db6b19b3-ad7f-11eb-bf92-94de80d4625f N/A ada0p1
gptid/f9c45ce7-ad7f-11eb-bf92-94de80d4625f N/A ada0p2

Where exactly do I get the existing_disk GPTID?
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
so
Code:
zpool replace VOL-1gptid/c821b7a9-160e-11e8-b0ee-94de80d4625f  gptid/f9c45ce7-ad7f-11eb-bf92-94de80d4625f

?
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Code:
root@freenas:~ # glabel status | grep ada0
gptid/db6b19b3-ad7f-11eb-bf92-94de80d4625f     N/A  ada0p1
gptid/f9c45ce7-ad7f-11eb-bf92-94de80d4625f     N/A  ada0p2
root@freenas:~ # gptid/db6b19b3-ad7f-11eb-bf92-94de80d4625f     N/A  ada0p1
gptid/db6b19b3-ad7f-11eb-bf92-94de80d4625f: Command not found.
root@freenas:~ # gptid/f9c45ce7-ad7f-11eb-bf92-94de80d4625f     N/A  ada0p2
gptid/f9c45ce7-ad7f-11eb-bf92-94de80d4625f: Command not found.
root@freenas:~ # zpool import -f VOL-1
cannot import 'VOL-1': one or more devices is currently unavailable
root@freenas:~ #
root@freenas:~ # zpool import -F VOL-1

root@freenas:~ #
root@freenas:~ #
root@freenas:~ # zpool status
  pool: VOL-1
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Mon May  3 18:00:52 2021
        1.03T scanned at 205M/s, 678G issued at 132M/s, 6.71T total
        309M resilvered, 9.86% done, 13:23:35 to go
config:

        NAME                                            STATE     READ WRITE CKSUM
        VOL-1                                           DEGRADED     0     0     0
          mirror-0                                      DEGRADED     0     0     0
            gptid/c73d4498-160e-11e8-b0ee-94de80d4625f  ONLINE       0     0     0  (resilvering)
            5428397735157554866                         UNAVAIL      0     0     0  was /dev/gptid/c821b7a9-16                                    0e-11e8-b0ee-94de80d4625f
          mirror-1                                      ONLINE       0     0     0
            gptid/460fc065-275d-11e9-aed9-94de80d4625f  ONLINE       0     0     0
            gptid/4700b30d-275d-11e9-aed9-94de80d4625f  ONLINE       0     0     0

errors: No known data errors

  pool: freenas-boot
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.
  scan: scrub repaired 0B in 00:04:08 with 0 errors on Wed May  5 03:49:08 2021
config:

        NAME          STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2       ONLINE       0     0     0

errors: No known data errors
root@freenas:~ #
 
sretalla

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
OK, so you can get the gptid of the existing disk from the zpool status you have there...

What's concerning is the resilver already in progress... maybe the GUI operation was sort-of successful.

I think there's some merit to waiting to allow that to complete... it's saying 13 hours. Although on top of that it's saying the resilver began on Monday... (2 days ago).
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
sretalla said:
OK, so you can get the gptid of the existing disk from the zpool status you have there...

What's concerning is the resilver already in progress... maybe the GUI operation was sort-of successful.

I think there's some merit to waiting to allow that to complete... it's saying 13 hours. Although on top of that it's saying the resilver began on Monday... (2 days ago).
Click to expand...
Yeah not sure what is going on. I'm at work now remoted in. The resilver is complete.
Should I try replace again?
 

Attachments

  • Capture.PNG
    Capture.PNG
    27.8 KB · Views: 124
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
sretalla said:
OK, so you can get the gptid of the existing disk from the zpool status you have there...

What's concerning is the resilver already in progress... maybe the GUI operation was sort-of successful.

I think there's some merit to waiting to allow that to complete... it's saying 13 hours. Although on top of that it's saying the resilver began on Monday... (2 days ago).
Click to expand...
Ok so I'm getting the GPTID of the bad disk, for "existing_disk" ? and the replacement disk for GPTID for "new_disk"?
 
sretalla

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
Paengaroa said:
Ok so I'm getting the GPTID of the bad disk, for "existing_disk" ? and the replacement disk for GPTID for "new_disk"?
Click to expand...
Yes. That's right. I was thinking about attaching a new member and not the replace action when I mentioned the other disk before.
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
sretalla said:
Yes. That's right. I was thinking about attaching a new member and not the replace action when I mentioned the other disk before.
Click to expand...
Ok great, I will run that now.
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Thanks so much again for all your time.
Getting so close!
Code:
root@freenas:~ # zpool replace VOL-1 gptid/c821b7a9-160e-11e8-b0ee-94de80d4625f                                         gptid/f9c45ce7-ad7f-11eb-bf92-94de80d4625f
cannot open 'gptid/f9c45ce7-ad7f-11eb-bf92-94de80d4625f': no such device in /dev
must be a full path or shorthand device name
root@freenas:~ #
 
Patrick M. Hausen

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
So the second gptid (your new disk) is not correct. Use gpart list to check again. The rawuuid field.
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Patrick M. Hausen said:
So the second gptid (your new disk) is not correct. Use gpart list to check again. The rawuuid field.
Click to expand...
Ok thanks
Code:
root@freenas:~ # gpart list
Geom name: ada1
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 7814037127
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: ada1p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   efimedia: HD(1,GPT,45ff4356-275d-11e9-aed9-94de80d4625f,0x80,0x400000)
   rawuuid: 45ff4356-275d-11e9-aed9-94de80d4625f
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada1p2
   Mediasize: 3998639456256 (3.6T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e2
   efimedia: HD(2,GPT,460fc065-275d-11e9-aed9-94de80d4625f,0x400080,0x1d180be00)
   rawuuid: 460fc065-275d-11e9-aed9-94de80d4625f
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 3998639456256
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 7814037119
   start: 4194432
Consumers:
1. Name: ada1
   Mediasize: 4000787030016 (3.6T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e3

Geom name: ada2
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 7814037127
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: ada2p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,c72d7f2d-160e-11e8-b0ee-94de80d4625f,0x80,0x400000)
   rawuuid: c72d7f2d-160e-11e8-b0ee-94de80d4625f
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada2p2
   Mediasize: 3998639456256 (3.6T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e2
   efimedia: HD(2,GPT,c73d4498-160e-11e8-b0ee-94de80d4625f,0x400080,0x1d180be00)
   rawuuid: c73d4498-160e-11e8-b0ee-94de80d4625f
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 3998639456256
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 7814037119
   start: 4194432
Consumers:
1. Name: ada2
   Mediasize: 4000787030016 (3.6T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5

Geom name: ada3
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 7814037127
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: ada3p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
   efimedia: HD(1,GPT,46f25454-275d-11e9-aed9-94de80d4625f,0x80,0x400000)
   rawuuid: 46f25454-275d-11e9-aed9-94de80d4625f
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 2147483648
   offset: 65536
   type: freebsd-swap
   index: 1
   end: 4194431
   start: 128
2. Name: ada3p2
   Mediasize: 3998639456256 (3.6T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e2
   efimedia: HD(2,GPT,4700b30d-275d-11e9-aed9-94de80d4625f,0x400080,0x1d180be00)
   rawuuid: 4700b30d-275d-11e9-aed9-94de80d4625f
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 3998639456256
   offset: 2147549184
   type: freebsd-zfs
   index: 2
   end: 7814037119
   start: 4194432
Consumers:
1. Name: ada3
   Mediasize: 4000787030016 (3.6T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5

Geom name: da0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 30274711
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: da0p1
   Mediasize: 272629760 (260M)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 20480
   Mode: r0w0e0
   efimedia: HD(1,GPT,15dec071-1679-11e8-a514-08002744d9f5,0x28,0x82000)
   rawuuid: 15dec071-1679-11e8-a514-08002744d9f5
   rawtype: c12a7328-f81f-11d2-ba4b-00a0c93ec93b
   label: (null)
   length: 272629760
   offset: 20480
   type: efi
   index: 1
   end: 532519
   start: 40
2. Name: da0p2
   Mediasize: 15227998208 (14G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 272650240
   Mode: r1w1e1
   efimedia: HD(2,GPT,1737ef14-1679-11e8-a514-08002744d9f5,0x82028,0x1c5d468)
   rawuuid: 1737ef14-1679-11e8-a514-08002744d9f5
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 15227998208
   offset: 272650240
   type: freebsd-zfs
   index: 2
   end: 30274703
   start: 532520
Consumers:
1. Name: da0
   Mediasize: 15500673024 (14G)
   Sectorsize: 512
   Mode: r1w1e2


So [/CODE} 15dec071-1679-11e8-a514-08002744d9f5 [/CODE] ?
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Or
1737ef14-1679-11e8-a514-08002744d9f5
?
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Ok, replacing is no longer an option, it says disks in use or partitioned won't show as available. So the new disk is partitioned now maybe by the attempted replace last night?
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Ok, so I think I got it. But new issue.
The disk is too small? Should I -f?
Code:
root@freenas:~ # zpool replace VOL-1 gptid/c821b7a9-160e-11e8-b0ee-94de80d4625f gptid/15dec071-1679-11e8-a514-08002744d9f5
cannot replace gptid/c821b7a9-160e-11e8-b0ee-94de80d4625f with gptid/15dec071-1679-11e8-a514-08002744d9f5: device is too small
root@freenas:~ #
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Paengaroa said:
Ok, so I think I got it. But new issue.
The disk is too small? Should I -f?
Code:
root@freenas:~ # zpool replace VOL-1 gptid/c821b7a9-160e-11e8-b0ee-94de80d4625f gptid/15dec071-1679-11e8-a514-08002744d9f5
cannot replace gptid/c821b7a9-160e-11e8-b0ee-94de80d4625f with gptid/15dec071-1679-11e8-a514-08002744d9f5: device is too small
root@freenas:~ #
Click to expand...
this is the gptid for da0p2
 
Patrick M. Hausen

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
da0 is probably your boot device (USB key)? gpart list shows 3 SATA disks with partition tables. What about camcontrol devlist?
 
P

Paengaroa

Dabbler
Joined
May 3, 2021
Messages
47
Oh the disk isn't showing up anywhere now. aho.
 
You must log in or register to reply here.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "Pool offline after replacing disk"

Similar threads

kaub07
Disk Died, couldn't offline, replaced, now can't bring pool online.
Replies
15
Views
2K
Arwen
Arwen
Raven9810
Newly replaced disk, shows FAULTED during resilver
Replies
3
Views
1K
Raven9810
Raven9810
M
Replacing Failed GELI Encrypted Disk TrueNAS
Replies
3
Views
1K
Teeps
T
T
Pool Offline after removing Dedup Disk
Replies
15
Views
2K
Arwen
Arwen
F
WD RED Drive not recognizing in Freenas or MiniXL
Replies
3
Views
1K
firedis01
F
Top