Cannot access jail interface when any pool is in an error state?

[brandonpoc]

I cannot access my jails web UI because I'm having an issue with a bad disk and ZFS. I have a single-disk ZFS pool for my /dev/da38 disk. It has errors and I cannot use 'zpool clear' because the following is reported:

root@wyoming[~]# zpool clear -F farm-16T-ZL20GVJ6 cannot clear errors for farm-16T-ZL20GVJ6: I/O error

When trying to access 'jails' in the web UI:

When I click 'CLOSE' I get a blank page without any jails listed and cannot click on the Options gear. 'ADD', or do anything. This is apparently some kind of bug. The jail does NOT run in this pool nor does it have this pool mounted or exported to it in any way. This is apparently some kind of bug.

I cannot get this ZFS pool to clear out even though the disk is damaged. I don't want to reboot and shouldn't have to considering this is an enterprise filesystem running on a rock solid UNIX operating system. When I remove the drive and re-seat it I can run the 'clear' command but it errors out before I can run export / disconnect.

Does anybody have any advice? I need to shut down my jail and add new bind mounts to my jail.

Thanks

 

[sretalla]

What is zpool status -v showing?

 

[brandonpoc]

What is zpool status -v showing?

There's an I/O failure as shown below. The 'zpool status -v' process locks up upon reaching the pool/disk with the error. I cannot kill it even with SIGKILL -9. Running 'df', 'ls' on the mount point, and some other commands result in the process locking up as well presumably waiting for the I/O that will never come. I have an easier time in Linux dealing with this kind of situation; it's leaving a bad taste in my mouth happening with BSD+ZFS.

Any ways to deal with this short of rebooting? This machine is a critical SAN server.

pool: farm-16T-ZL20GVJ6 state: UNAVAIL status: One or more devices are faulted in response to IO failures. action: Make sure the affected devices are connected, then run 'zpool clear'. see: http://illumos.org/msg/ZFS-8000-JQ scan: resilvered 0 in 0 days 00:00:00 with 0 errors on Wed Apr 14 03:58:28 2021 config: NAME STATE READ WRITE CKSUM farm-16T-ZL20GVJ6 UNAVAIL 0 0 0 16078973979506811357 UNAVAIL 0 0 0 was /dev/gptid/923430f5-9cda-11eb-8358-bc305be7f931 errors: Permanent errors have been detected in the following files: <metadata>:<0x35>