GPU passthrough to VM and app

[eroji]

I am currently running BlueFin but I think this is a general TrueNAS Scale thing. I'm attempting to setup 2 Tesla P4 GPUs in my system where 1 GPU is assigned to a Windows VM for NVR (Blue Iris), and another to Kubernetes to run Plex. However, the issue I'm seeing is that if one if the GPU is assigned to the Windows VM, then the OS fails to see any GPUs (besides onboard). I also tried the "Isolated GPU Device(s)" configuration but that does not help either.

Code:

root@nas-02[~]# lspci | grep VGA
c4:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41)

Code:

root@nas-02[~]# nvidia-smi
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Code:

root@nas-02[~]# k3s kubectl get po -n kube-system
NAME                                   READY   STATUS             RESTARTS         AGE
coredns-75fc8f8fff-7nptv               0/1     Completed          0                122m
openebs-zfs-node-ndr4t                 2/2     Running            0                112m
svclb-traefik-tcp-47c94902-jbcpv       2/2     Running            0                112m
openebs-zfs-controller-0               5/5     Running            0                112m
coredns-75fc8f8fff-26689               1/1     Running            0                112m
svclb-traefik-5e54733d-qk7xl           1/1     Running            0                112m
nvidia-device-plugin-daemonset-txltj   0/1     CrashLoopBackOff   26 (3m56s ago)   112m

Inside VM

 

[morganL]

worth reporting as a bug.. you'll need full hardware descript.

Please report the bug-ID

 

[eroji]

Here it is: https://ixsystems.atlassian.net/browse/NAS-119250

 

[Saberwolf]

this is not possible due to the vm taking full control of gpu it get put into black list for the host machine you can have either vm or app with one gpu or you can add another card.