Error exporting/disconnecting pool through GUI

[unorthodox]

Hello everyone!

I received an error while trying out TrueNAS-SCALE-21.08-BETA.1 and attempting to Export/Disconnect a pool using the GUI. I'm not really concerned about getting rid of the pool, I could probably figure something out eventually. But I wasn't sure if this is an error that should be reported, if I did something stupid, or if I'm missing something obvious.

The chain of events that led to this is:
- I was curious about native OpenZFS encryption nearly a year ago and created a pool with an encrypted dataset on an Ubuntu Server system.
- I recently replaced Ubuntu with TrueNAS-SCALE-21.08-BETA.1 and imported the pool successfully.
- I attempted to "Export/Disconnect" the pool using the web interface and received the error "Error exporting/disconnecting pool."
- I rebooted the system, did not unlock the encrypted dataset this time, made sure no shares were active, made sure no scrub was happening, and no applications are installed. I tried to "Export/Disconnect" the pool again and once again got an error:

Error exporting/disconnecting pool.
[EFAULT] pool or dataset is busy

Show

Error: concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 142, in do_delete
zfs.destroy(name, force=options['force'])
File "libzfs.pyx", line 404, in libzfs.ZFS.__exit__
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 142, in do_delete
zfs.destroy(name, force=options['force'])
File "libzfs.pyx", line 1340, in libzfs.ZFS.destroy
libzfs.ZFSException: pool or dataset is busy

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 97, in main_worker
res = MIDDLEWARE._run(*call_args)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 45, in _run
return self._call(name, serviceobj, methodobj, args, job=job)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 39, in _call
return methodobj(*params)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 39, in _call
return methodobj(*params)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1098, in nf
res = f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1186, in nf
return func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 147, in do_delete
raise CallError(str(e), errno_)
middlewared.service_exception.CallError: [EFAULT] pool or dataset is busy
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 382, in run
await self.future
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 418, in __run_body
rv = await self.method(*([self] + args))
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1182, in nf
return await func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1092, in nf
res = await f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 1636, in export
await self.middleware.call('zfs.pool.delete', pool['name'])
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1305, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1262, in _call
return await methodobj(*prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/service.py", line 849, in delete
rv = await self.middleware._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1270, in _call
return await self._call_worker(name, *prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1276, in _call_worker
return await self.run_in_proc(main_worker, name, args, job)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1203, in run_in_proc
return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1177, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
middlewared.service_exception.CallError: [EFAULT] pool or dataset is busy

Motherboard: Supermicro X10SL7-F
CPU: Intel(R) Xeon(R) CPU E3-1246 v3 @ 3.50GHz
RAM: 32GB with ecc
Boot drive: Corsair 120 GB SSD
HDD: 4x WD Red WD40EFRX 4TB in RAIDZ2

 

[spambot]

Same problem with same error message as unorthodox. After disconnection of disk I get another error. No chance to remove the pool.

Code:

Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 409, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 445, in __run_body
    rv = await self.method(*([self] + args))
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1269, in nf
    return await func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1137, in nf
    res = await f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 1606, in export
    root_ds = await self.middleware.call('pool.dataset.get_instance', pool['name'])
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1324, in call
    return await self._call(
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1281, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1269, in nf
    return await func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/service.py", line 955, in get_instance
    return await self._get_instance(id, options)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1269, in nf
    return await func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/service.py", line 968, in _get_instance
    raise InstanceNotFound(f'{self._config.verbose_name} {id} does not exist')
middlewared.service_exception.InstanceNotFound: [ENOENT] None: PoolDataset TestPool does not exist

 

[timgjr]

I'm having same issues.. There are a few posts about an old method on CORE that allows one to remove manually. Thats not working for me under SCALE.

Anyone with understanding of the new CLI, maybe that could work? info on it seems slim though, not fully documented.

 

[sretalla]

Same problem with same error message as unorthodox. After disconnection of disk I get another error. No chance to remove the pool.

Not the same as the OP, but you now have a zombie pool... https://www.truenas.com/community/threads/deleting-a-zombie-pool.90592/

 

[sretalla]

Error exporting/disconnecting pool.
[EFAULT] pool or dataset is busy

Where's your system dataset? you may need to move it to another pool (you can always select the boot pool) before exporting the pool which holds it.

Advanced Settings |

Advanced Settings

www.truenas.com

 

[unorthodox]

@sretalla Thank you for the response! Sorry for the late reply, for some reason I didn't get an email notification of your response to my post and just found this open tab (I know, terribly tab management). As far as I can remember it was the Syslog that was being stored on the dataset, I had to go to System Settings -> Advanced -> Syslog 'Configure' and check the 'Use System Dataset' box.

Thank you!

 

[PyCoder]

Code:

lsof /mnt/poolname

can help as well

 

[unorthodox]

Code:

lsof /mnt/poolname

can help as well

Interesting, I wasn't aware of this, I'll definitely add this to my notes. Thank you!
I couldn't get it to do it recursively (even with +D) though , maybe it can't cross datasets? I know I've had issues with NFS not crossing nested datasets.