Error during disk replace, but seems to have started

[Evan Richardson]

I just tried replacing a disk, using the Offline, swap, replace method (no slots free), and after maybe 2 or 3 minutes, the UI gave me the following error:

Code:

Error: concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 277, in replace
    target.replace(newvdev)
  File "libzfs.pyx", line 391, in libzfs.ZFS.__exit__
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 277, in replace
    target.replace(newvdev)
  File "libzfs.pyx", line 2060, in libzfs.ZFSVdev.replace
libzfs.ZFSException: no such pool or dataset

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 111, in main_worker
    res = MIDDLEWARE._run(*call_args)
  File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 45, in _run
    return self._call(name, serviceobj, methodobj, args, job=job)
  File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 39, in _call
    return methodobj(*params)
  File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 39, in _call
    return methodobj(*params)
  File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 979, in nf
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 279, in replace
    raise CallError(str(e), e.code)
middlewared.service_exception.CallError: [EZFS_NOENT] no such pool or dataset
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 367, in run
    await self.future
  File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 403, in __run_body
    rv = await self.method(*([self] + args))
  File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 975, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/pool_/replace_disk.py", line 122, in replace
    raise e
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/pool_/replace_disk.py", line 102, in replace
    await self.middleware.call(
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1256, in call
    return await self._call(
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1221, in _call
    return await self._call_worker(name, *prepared_call.args)
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1227, in _call_worker
    return await self.run_in_proc(main_worker, name, args, job)
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1154, in run_in_proc
    return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1128, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
middlewared.service_exception.CallError: [EZFS_NOENT] no such pool or dataset

However, If i open another window and view the pool it seems to be replacing:

zpool iostat shows the same:

Code:

 raidz2                                          26.1T  17.4T     19      0  4.95M      0
    replacing                                         -      -      0      0      0      0
      gptid/cd422c96-db25-11e7-a7b6-0025905decac      -      -      0      0      0      0
      da0p2                                           -      -      0      0      0      0

so it seems everything is fine, but why the UI error then?

I'm running on TrueNAS-12.0-U8

Thanks!

 

[Evan Richardson]

edit: i think it might be because autoreplace is set to enabled, so my guess is that upon inserting the drive, it automatically started replacing, and when I tried to do it through the UI it timed out or something because it had already restarted

edit 2: nope its definitely some sort of bug. I swapped a second drive (different vdev, one disk), and let it sit for 10 minutes.. No automatic resilver until i started in the UI. Once started in the UI, same error.

 

[sretalla]

i think it might be because autoreplace is set to enabled,

That's not a thing unless you have a drive designated as spare in the pool already.

 

[Evan Richardson]

That's not a thing unless you have a drive designated as spare in the pool already.

ahh that makes sense.


So first disk finished replacing, second is close to being done, but one thing I've noticed now is that (again, via the UI), the new disk does not have a GPTID/1233abcd id, it has da0p2:

Code:

raidz2                                          26.1T  17.4T      0      0      0      0
    da0p2                                             -      -      0      0      0      0
    gptid/ce00d444-db25-11e7-a7b6-0025905decac        -      -      0      0      0      0
    gptid/ceaabe76-db25-11e7-a7b6-0025905decac        -      -      0      0      0      0
    gptid/cf4ec403-db25-11e7-a7b6-0025905decac        -      -      0      0      0      0
    gptid/cffae5f2-db25-11e7-a7b6-0025905decac        -      -      0      0      0      0
    gptid/d0ac6387-db25-11e7-a7b6-0025905decac        -      -      0      0      0      0

according to this: https://www.truenas.com/community/t...gptid-instead-of-da1p2-in-zpool-status.18441/ and this: https://www.truenas.com/community/t...is-now-ada6-instead-of-gptid-something.77618/

it seems this only happens via the CLI, not GUI, except I used the GUI to do this. Is there any issue with this?