Pool from Core not importing after new install of Scale

[Kailee71]

Hi all,
so I have just installed a new Scale 23.10 server, moved the disks for my two pools over from my Core server (zpool has 2 striped mirrors, zbackup is single dev), and am now trying to import my zpool into Scale. The pools were properly exported through the GUI in Core before I moved them. The single dev zbackup imported without any hitches, but the zpool consisting of the two striped mirrors won't properly. I say properly because the zpool datasets are listed in Datasets, but the pool itself is not listed in the storage dashboard (curiously the zbackup pool imported just fine). I remember a small window popping up about NFS shares not being able to be recreated, but I thought little of it as I was going to do my nfs shares manually later anyway.

Rebooting makes Scale completely forget it even tried before, and will offer the 4-disk zpool for importation again.

How can I fix this?

Any hints greatly appreciated.

 

[morganL]

Hi all,
so I have just installed a new Scale 23.10 server, moved the disks for my two pools over from my Core server (zpool has 2 striped mirrors, zbackup is single dev), and am now trying to import my zpool into Scale. The pools were properly exported through the GUI in Core before I moved them. The single dev zbackup imported without any hitches, but the zpool consisting of the two striped mirrors won't properly. I say properly because the zpool datasets are listed in Datasets, but the pool itself is not listed in the storage dashboard (curiously the zbackup pool imported just fine). I remember a small window popping up about NFS shares not being able to be recreated, but I thought little of it as I was going to do my nfs shares manually later anyway.

Rebooting makes Scale completely forget it even tried before, and will offer the 4-disk zpool for importation again.

How can I fix this?

Any hints greatly appreciated.

Full hardware descriptions would be useful... including drives.

I'd suggest trying to reimport, but capture screenshots of any error messages or pool status. People need clues to resolve these issues.

 

[Kailee71]

Thank you for asking ... I was hoping this was a common thing to happen with a quick fix, but here goes;

-HP DL380P Gen 8, 2x E5-2690v2, 128GiB DDR3 ECC
-boot pool mirror on 2 ssd's on internal P420i
-remaining disks on LSI HBA
-zpool 4x WD Red 4TiB as 2 striped mirrors
-zbackup as single WD Red 6TiB vdev

This is now running Scale 23.10 on bare metal, the two pools came from Core 13 running as ESXi VM with HBA passed through.
Importing the single vdev zbackup went without a hitch.
I tried importing the zpool several times through the GUI, which every time finds the pool no problem after a reboot. When clicking on zpool, after a few seconds I get this error message:

The "More Info..." gives

Code:

Error: concurrent.futures.process._RemoteTraceback: """ Traceback (most recent call last):  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 250, in update_zfs_object_props    zfs_object.update_properties(properties)  File "libzfs.pyx", line 3626, in libzfs.ZFSResource.update_properties libzfs.ZFSException: cannot share 'zpool/media/tvshows: system error': NFS share creation failed During handling of the above exception, another exception occurred: Traceback (most recent call last):  File "/usr/lib/python3.11/concurrent/futures/process.py", line 256, in _process_worker    r = call_item.fn(*call_item.args, **call_item.kwargs)        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 112, in main_worker    res = MIDDLEWARE._run(*call_args)          ^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 46, in _run    return self._call(name, serviceobj, methodobj, args, job=job)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 34, in _call    with Client(f'ws+unix://{MIDDLEWARE_RUN_DIR}/middlewared-internal.sock', py_exceptions=True) as c:  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 40, in _call    return methodobj(*params)           ^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 50, in nf    res = f(*args, **kwargs)          ^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 181, in nf    return func(*args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 169, in do_update    with libzfs.ZFS() as zfs:  File "libzfs.pyx", line 529, in libzfs.ZFS.__exit__  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 178, in do_update    self.update_zfs_object_props(properties, dataset)  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 252, in update_zfs_object_props    raise CallError(f'Failed to update properties: {e!r}') middlewared.service_exception.CallError: [EFAULT] Failed to update properties: ZFSException("cannot share 'zpool/media/tvshows: system error': NFS share creation failed") """ The above exception was the direct cause of the following exception: Traceback (most recent call last):  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 426, in run    await self.future  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 464, in __run_body    rv = await self.method(*([self] + args))         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf    return await func(*args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 44, in nf    res = await f(*args, **kwargs)          ^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/import_pool.py", line 146, in import_pool    await self.middleware.call('zfs.dataset.update', pool_name, opts)  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1398, in call    return await self._call(           ^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1341, in _call    return await methodobj(*prepared_call.args)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 177, in update    return await self.middleware._call(           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1349, in _call    return await self._call_worker(name, *prepared_call.args)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1355, in _call_worker    return await self.run_in_proc(main_worker, name, args, job)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1267, in run_in_proc    return await self.run_in_executor(self.__procpool, method, *args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1251, in run_in_executor    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ middlewared.service_exception.CallError: [EFAULT] Failed to update properties: ZFSException("cannot share 'zpool/media/tvshows: system error': NFS share creation failed")

No errors are listed in /var/log/messages.

A "zpool list" gives the following:

Code:

root@truenas:/var/log# zpool list
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
boot-pool   216G  3.55G   212G        -         -     0%     1%  1.00x    ONLINE  -
zbackup    5.45T  2.44T  3.01T        -         -     0%    44%  1.00x    ONLINE  /mnt
zpool      7.25T  5.43T  1.82T        -         -    11%    74%  1.00x    ONLINE  /mnt

After this, the pool is not listed in the storage dashboard, but it is mounted and accessible through a shell. The datasets are also listed in the datasets dashboard. They are not however listed in any dialogs that might need them (share creation, user home directories, etc). The SMB and NFS services are running but without any shares created (as I can't select the datasets through the GUI).

I remember vaguely that ZFS is able to export datasets directly and I assume holds that information in the pool.

Code:

root@truenas:/var/log# zfs get all zpool/media/tvshows | grep nfs
zpool/media/tvshows  sharenfs                on                        local
zpool/media/tvshows  acltype                 nfsv4                     default

Is it safe to remove the NFS share information from the pool manually to enable the pool to be imported?

 

[morganL]

Thank you for asking ... I was hoping this was a common thing to happen with a quick fix, but here goes;

-HP DL380P Gen 8, 2x E5-2690v2, 128GiB DDR3 ECC
-boot pool mirror on 2 ssd's on internal P420i
-remaining disks on LSI HBA
-zpool 4x WD Red 4TiB as 2 striped mirrors
-zbackup as single WD Red 6TiB vdev

This is now running Scale 23.10 on bare metal, the two pools came from Core 13 running as ESXi VM with HBA passed through.
Importing the single vdev zbackup went without a hitch.
I tried importing the zpool several times through the GUI, which every time finds the pool no problem after a reboot. When clicking on zpool, after a few seconds I get this error message:

View attachment 74164

The "More Info..." gives

Code:

Error: concurrent.futures.process._RemoteTraceback: """ Traceback (most recent call last):  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 250, in update_zfs_object_props    zfs_object.update_properties(properties)  File "libzfs.pyx", line 3626, in libzfs.ZFSResource.update_properties libzfs.ZFSException: cannot share 'zpool/media/tvshows: system error': NFS share creation failed During handling of the above exception, another exception occurred: Traceback (most recent call last):  File "/usr/lib/python3.11/concurrent/futures/process.py", line 256, in _process_worker    r = call_item.fn(*call_item.args, **call_item.kwargs)        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 112, in main_worker    res = MIDDLEWARE._run(*call_args)          ^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 46, in _run    return self._call(name, serviceobj, methodobj, args, job=job)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 34, in _call    with Client(f'ws+unix://{MIDDLEWARE_RUN_DIR}/middlewared-internal.sock', py_exceptions=True) as c:  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 40, in _call    return methodobj(*params)           ^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 50, in nf    res = f(*args, **kwargs)          ^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 181, in nf    return func(*args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 169, in do_update    with libzfs.ZFS() as zfs:  File "libzfs.pyx", line 529, in libzfs.ZFS.__exit__  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 178, in do_update    self.update_zfs_object_props(properties, dataset)  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 252, in update_zfs_object_props    raise CallError(f'Failed to update properties: {e!r}') middlewared.service_exception.CallError: [EFAULT] Failed to update properties: ZFSException("cannot share 'zpool/media/tvshows: system error': NFS share creation failed") """ The above exception was the direct cause of the following exception: Traceback (most recent call last):  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 426, in run    await self.future  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 464, in __run_body    rv = await self.method(*([self] + args))         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 177, in nf    return await func(*args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 44, in nf    res = await f(*args, **kwargs)          ^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/import_pool.py", line 146, in import_pool    await self.middleware.call('zfs.dataset.update', pool_name, opts)  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1398, in call    return await self._call(           ^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1341, in _call    return await methodobj(*prepared_call.args)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 177, in update    return await self.middleware._call(           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1349, in _call    return await self._call_worker(name, *prepared_call.args)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1355, in _call_worker    return await self.run_in_proc(main_worker, name, args, job)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1267, in run_in_proc    return await self.run_in_executor(self.__procpool, method, *args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1251, in run_in_executor    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ middlewared.service_exception.CallError: [EFAULT] Failed to update properties: ZFSException("cannot share 'zpool/media/tvshows: system error': NFS share creation failed")

No errors are listed in /var/log/messages.

A "zpool list" gives the following:

Code:

root@truenas:/var/log# zpool list
NAME        SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
boot-pool   216G  3.55G   212G        -         -     0%     1%  1.00x    ONLINE  -
zbackup    5.45T  2.44T  3.01T        -         -     0%    44%  1.00x    ONLINE  /mnt
zpool      7.25T  5.43T  1.82T        -         -    11%    74%  1.00x    ONLINE  /mnt

After this, the pool is not listed in the storage dashboard, but it is mounted and accessible through a shell. The datasets are also listed in the datasets dashboard. They are not however listed in any dialogs that might need them (share creation, user home directories, etc). The SMB and NFS services are running but without any shares created (as I can't select the datasets through the GUI).

I remember vaguely that ZFS is able to export datasets directly and I assume holds that information in the pool.

Code:

root@truenas:/var/log# zfs get all zpool/media/tvshows | grep nfs
zpool/media/tvshows  sharenfs                on                        local
zpool/media/tvshows  acltype                 nfsv4                     default

Is it safe to remove the NFS share information from the pool manually to enable the pool to be imported?

Which specific versions of CORE and SCALE?

It would be safe to disable NFS and SMB services while importing.

How do you plan to "remove the NFS share information"???

 

[anodos]

Looks like you've manually set the sharenfs property on datasets in Core (this is an unsupported configuration on both Core and SCALE). You can work around the import error by disabling the property.

 

[Kailee71]

Which specific versions of CORE and SCALE?

It would be safe to disable NFS and SMB services while importing.

How do you plan to "remove the NFS share information"???

What I did was

Code:

zfs set -r sharenfs=off zpool

and this did indeed fix it. I have no memory of ever setting this manually on Core (using 11, 12 and 13 over the years). Is there another way to disable the sharenfs property?

 

[anodos]

What I did was

Code:

zfs set -r sharenfs=off zpool

and this did indeed fix it. I have no memory of ever setting this manually on Core (using 11, 12 and 13 over the years). Is there another way to disable the sharenfs property?

That is the way to do it. This property is never set via the TrueNAS product.