VM for Plex with GPU Passthought

[rschneider82]

I am trying to set up a VM for PLEX with GPU passthought and when I create the VM I get a display error. I have tried it with 2 different GPU Cards. Nivida Quadro P400 and Quadro P2000.

Running
Dell Poweredge R510
2x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
RAM 23.5GiB ECC
TrueNAS-SCALE-22.12.4

admin@truenas[~]$ lspci |grep NVIDIA
04:00.0 VGA compatible controller: NVIDIA Corporation GP106GL [Quadro P2000] (rev a1)
04:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)
05:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P400] (rev a1)
05:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1)

VM Setup

Virtual CPUs:
1
Cores:
4
Threads:
2
Memory Size:
8.00 GiB
Boot Loader Type:
UEFI_CSM
System Clock:
LOCAL
Display Port:
5900
Description:
N/A
Shutdown Timeout:
90 seconds

Here is the error Message

CallError
[EFAULT] internal error: process exited while connecting to monitor: 2023-10-04T13:37:01.950305Z qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=hostdev0,bus=pci.0,addr=0x8: vfio 0000:05:00.0: failed to setup container for group 15: Failed to set iommu for container: Operation not permitted

Error: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/supervisor/supervisor.py", line 172, in start
if self.domain.create() < 0:
File "/usr/lib/python3/dist-packages/libvirt.py", line 1353, in create
raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: internal error: process exited while connecting to monitor: 2023-10-04T13:37:01.950305Z qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=hostdev0,bus=pci.0,addr=0x8: vfio 0000:05:00.0: failed to setup container for group 15: Failed to set iommu for container: Operation not permitted

During handling of the above exception, another exception occurred:

Code:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 184, in call_method
    result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self)
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1317, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1379, in nf
    return await func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1247, in nf
    res = await f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/vm_lifecycle.py", line 46, in start
    await self.middleware.run_in_thread(self._start, vm['name'])
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1234, in run_in_thread
    return await self.run_in_executor(self.thread_pool_executor, method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1231, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/vm_supervisor.py", line 68, in _start
    self.vms[vm_name].start(vm_data=self._vm_from_name(vm_name))
  File "/usr/lib/python3/dist-packages/middlewared/plugins/vm/supervisor/supervisor.py", line 181, in start
    raise CallError('\n'.join(errors))
middlewared.service_exception.CallError: [EFAULT] internal error: process exited while connecting to monitor: 2023-10-04T13:37:01.950305Z qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=hostdev0,bus=pci.0,addr=0x8: vfio 0000:05:00.0: failed to setup container for group 15: Failed to set iommu for container: Operation not permitted

 

[rschneider82]

If I do not do GPU passthought, I can create VM with no issues.

 

[thirdgen89gta]

I had an issue after updating to Bluefin release, and had to roll back to angelfish. I think that was the error I was getting. But it also went away if I didn’t pass the gpu to the vm.

It was like the GPU was reserved for the system.

 

[DanReid21]

Make sure to set the primary display to the onboard video card, otherwise you are attempting to pass through a resource that is under use.

 

[thirdgen89gta]

Make sure to set the primary display to the onboard video card, otherwise you are attempting to pass through a resource that is under use.

My system has IPMI and that is the primary. The GPU was not in use.