TrueNAS 13.0 RELEASE replace 1 disk in mirrored boot pool

[macmuchmore]

I have 2 SSDs set up as a mirrored boot pool, and I need to replace one of them. In v13, it gives you the option to DETACH or REPLACE, and the documentation has not been updated with the correct procedure for replacing one of the disks in the boot pool. (To add a disk to a boot pool that only has one disk, you use ATTACH to accomplish this.) So my question is, for replacing one of the boot pool SSDs, which option should I use - DETACH or REPLACE? the old SSD? Thanks in advance for any assistance!

 

[sretalla]

To "replace" the disk, use REPLACE... (seems obvious)

You can also get to the same result via DETACH and then ATTACH the replacement disk.

 

[macmuchmore]

To "replace" the disk, use REPLACE... (seems obvious)

You can also get to the same result via DETACH and then ATTACH the replacement disk.

Thanks for the advice. Agreed it seemed obvious, but in 12, what seemed obvious (at least to me) when replacing a disk in a pool was not the correct option. Since the documentation isn’t clear, I’d rather ask than assume. Again, thanks for the advice.

 

[mav@]

REPLACE allows to do it without temporary loosing redundancy, assuming hardware allows to insert new disk before removing the old. Though for boot pool it is not a big deal, considering you should still have a config backup somewhere.

 

[macmuchmore]

ok - So I chose DETACH since I could not add the new disk before removing the old (the SSDs are NVME drives and I only have 2 slots). Once the old drive was detached, I shut down, changed out the drive for a new one, and booted back up. The new drive is identified and is available to ATTACH to the boot pool. However, when I go to attach it, I get the errors:

In the GUI:

Attach Device​

Error: 'NoneType' object is not subscriptable


On the display attached to the NAS:

cannot get CTL max ports
Initializationof plugin 'ctl' failed with status -1. Plugin will be unloaded.
Error: one or more plugin init callbacks failed.
GEOM_ELI: Devicemirror/swap0.eli destroyed.
GEOM_MIRROR: Device swap0: provider destroyed.
GEOM_MIRROR: Device swap0 destroyed.

Should I just reinstall TrueNAS 13 and use my backup configuration file?

 

[macmuchmore]

I rebooted and tried again, and I received the following errors:

In the GUI:

Attach Device​

Error: [EFAULT] concurrent.futures.process._RemoteTraceback: """ Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 212, in extend i['target'].attach(newvdev) File "libzfs.pyx", line 392, in libzfs.ZFS.__exit__ File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 212, in extend i['target'].attach(newvdev) File "libzfs.pyx", line 2092, in libzfs.ZFSVdev.attach libzfs.ZFSException: can only attach to mirrors and top-level disks During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker r = call_item.fn(*call_item.args, **call_item.kwargs) File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 111, in main_worker res = MIDDLEWARE._run(*call_args) File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 45, in _run return self._call(name, serviceobj, methodobj, args, job=job) File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 39, in _call return methodobj(*params) File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 39, in _call return methodobj(*params) File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 979, in nf return f(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 215, in extend raise CallError(str(e), e.code) middlewared.service_exception.CallError: [EZFS_BADTARGET] can only attach to mirrors and top-level disks """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 355, in run await self.future File "/usr/local/lib/python3.9/site-packages/middlewared/job.py", line 386, in __run_body rv = await self.middleware._call_worker(self.method_name, *self.args, job={'id': self.id}) File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1276, in _call_worker return await self.run_in_proc(main_worker, name, args, job) File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1192, in run_in_proc return await self.run_in_executor(self.__procpool, method, *args, **kwargs) File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1166, in run_in_executor return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs)) middlewared.service_exception.CallError: [EZFS_BADTARGET] can only attach to mirrors and top-level disks

On the display attached to the NAS:

GEOM_ELI: Devicemirror/swap0.eli destroyed.
GEOM_MIRROR: Device swap0: provider destroyed.
GEOM_MIRROR: Device swap0 destroyed.
Error: one or more plugin init callbacks failed.

 

[mav@]

Are you clicking ATTACH on the remaining disk? "can only attach to mirrors and top-level disks" error looks like either you are doing something else, or it is not implemented correctly. I'd propose you to create a ticket to make sure the problem have chance to be fixed in next release. After that you may indeed reinstall and restore config as you have said. But honestly, I don't see much good in mirroring NVMe boot device -- those are usually reliable enough to not bother more than periodically back up the config.

 

[macmuchmore]

Are you clicking ATTACH on the remaining disk? "can only attach to mirrors and top-level disks" error looks like either you are doing something else, or it is not implemented correctly. I'd propose you to create a ticket to make sure the problem have chance to be fixed in next release. After that you may indeed reinstall and restore config as you have said. But honestly, I don't see much good in mirroring NVMe boot device -- those are usually reliable enough to not bother more than periodically back up the config.

Yes, I am selecting ATTACH on the remaining disk (nvd1), and then selecting the new member disk (nvd0) and checking the "Use all disk space" option, then clicking Submit. I will open a ticket - I just wanted to make sure it is a problem that needs a ticket as I would rather not waste the developer's time on something that might be my fault. ;-)

 

[macmuchmore]

Are you clicking ATTACH on the remaining disk? "can only attach to mirrors and top-level disks" error looks like either you are doing something else, or it is not implemented correctly. I'd propose you to create a ticket to make sure the problem have chance to be fixed in next release. After that you may indeed reinstall and restore config as you have said. But honestly, I don't see much good in mirroring NVMe boot device -- those are usually reliable enough to not bother more than periodically back up the config.

I have opened ticket: NAS-116168