[EZFS_IO] Failed to import 'MasterPool' pool: I/O error

[zsw12abc]

Hey Mates,
I faced an issue on my TrueNAS-SCALE-22.02.4
I tried to import my MasterPool from web UI
and it shows the error message below
And i saw a video with

Code:

zpool import -fFX MasterPool

to fix it
but that command runs whole night and didnt finish.
Please help me.

错误: concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 411, in import_pool
zfs.import_pool(found, pool_name, options, missing_log=missing_log, any_host=any_host)
File "libzfs.pyx", line 1168, in libzfs.ZFS.import_pool
File "libzfs.pyx", line 1196, in libzfs.ZFS.__import_pool
libzfs.ZFSException: I/O error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 114, in main_worker
res = MIDDLEWARE._run(*call_args)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 45, in _run
return self._call(name, serviceobj, methodobj, args, job=job)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 39, in _call
return methodobj(*params)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 39, in _call
return methodobj(*params)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1276, in nf
return func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 417, in import_pool
self.logger.error(
File "libzfs.pyx", line 411, in libzfs.ZFS.__exit__
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 415, in import_pool
raise CallError(f'Failed to import {pool_name!r} pool: {e}', e.code)
middlewared.service_exception.CallError: [EZFS_IO] Failed to import 'MasterPool' pool: I/O error
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 411, in run
await self.future
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 446, in __run_body
rv = await self.method(*([self] + args))
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1272, in nf
return await func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1140, in nf
res = await f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 1401, in import_pool
await self.middleware.call('zfs.pool.import_pool', guid, opts, any_host, use_cachefile, new_name)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1345, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1302, in _call
return await self._call_worker(name, *prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1308, in _call_worker
return await self.run_in_proc(main_worker, name, args, job)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1223, in run_in_proc
return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1206, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
middlewared.service_exception.CallError: [EZFS_IO] Failed to import 'MasterPool' pool: I/O error

 

[zsw12abc]

please help me

 

[Arwen]

Please follow the forum rules with your hardware setup, pool configuration and any preceding events.

Supply the output of zpool import in code tags.

 

[zsw12abc]

Please follow the forum rules with your hardware setup, pool configuration and any preceding events.

Supply the output of zpool import in code tags.

Hey @Arwen , thanks for helping me.
Here is the screenshot

 

[zsw12abc]

Code:

错误: concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 411, in import_pool
    zfs.import_pool(found, pool_name, options, missing_log=missing_log, any_host=any_host)
  File "libzfs.pyx", line 1168, in libzfs.ZFS.import_pool
  File "libzfs.pyx", line 1196, in libzfs.ZFS.__import_pool
libzfs.ZFSException: I/O error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 114, in main_worker
    res = MIDDLEWARE._run(*call_args)
  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 45, in _run
    return self._call(name, serviceobj, methodobj, args, job=job)
  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 39, in _call
    return methodobj(*params)
  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 39, in _call
    return methodobj(*params)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1276, in nf
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 417, in import_pool
    self.logger.error(
  File "libzfs.pyx", line 411, in libzfs.ZFS.__exit__
  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 415, in import_pool
    raise CallError(f'Failed to import {pool_name!r} pool: {e}', e.code)
middlewared.service_exception.CallError: [EZFS_IO] Failed to import 'MasterPool' pool: I/O error
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 411, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 446, in __run_body
    rv = await self.method(*([self] + args))
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1272, in nf
    return await func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1140, in nf
    res = await f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 1401, in import_pool
    await self.middleware.call('zfs.pool.import_pool', guid, opts, any_host, use_cachefile, new_name)
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1345, in call
    return await self._call(
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1302, in _call
    return await self._call_worker(name, *prepared_call.args)
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1308, in _call_worker
    return await self.run_in_proc(main_worker, name, args, job)
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1223, in run_in_proc
    return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1206, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
middlewared.service_exception.CallError: [EZFS_IO] Failed to import 'MasterPool' pool: I/O error

 

[Arwen]

Well, ZFS sees all the disks for the MasterPool, so the question comes up what is causing the I/O error.

Please follow the forum rules with your hardware setup, pool configuration and any preceding events.

 

[zsw12abc]

Well, ZFS sees all the disks for the MasterPool, so the question comes up what is causing the I/O error.

Please follow the forum rules with your hardware setup, pool configuration and any preceding events.

Sorry @Arwen, I am new here and don't know how to create the hide button as others did.

Motherboard: Z370
CPU: i7 8700K
GPU: none
M2 SSD: Samsung 960
Sata SSD: Samsung 660
HHD *4: WD Blue * 2 + WD Red * 2 (All of them are 2TB Each)

The issue happened after the power outrage.
It was working before and there is 2 pools in my TrueNas.
The MasterPool (in 4 HDD) is disconnected after Power Outrage but the other pool (in SSD) is still working.
So I exported the MasterPool and tried to import it again then face this I/O issue.

 

[Arwen]

Their is a known problem with some Western Digital Red disk drives. Please supply the model and firmware version for them. You can get this information from using smartctl -a /dev/XXX against the device.

 

[zsw12abc]

Their is a known problem with some Western Digital Red disk drives. Please supply the model and firmware version for them. You can get this information from using smartctl -a /dev/XXX against the device.

Hey @Arwen is this the one u want?

 

[zsw12abc]

Here is the screenshots for all 4 HDD

 

[Arwen]

Looks like your 2 WD Reds are CMR, (aka Red Pluses). One of your WD Blues is SMR, but I don't know if that affects your pool and it's problem.

I have no further suggestions.

 

[zsw12abc]

Looks like your 2 WD Reds are CMR, (aka Red Pluses). One of your WD Blues is SMR, but I don't know if that affects your pool and it's problem.

I have no further suggestions.

Thank @Arwen . I also dont think CRM and SMR will effect importing pool.
Now i am trying

Code:

zpool import -fFX Masterpool

but do u know how long will this command runs?
last time, it ran for 5 days and still not finished.