[Guide] Intel GVT-g for VMs

[Steinhose]

Hello,
this is a small tutorial on how to get a vgpu in a virtual machine. This is useful if, for example, you want to have a VM for Docker and run hardware transcoding there, or if you want graphics accelerated Windows.

This guide was tested on TrueNAS SCALE 22.12.4, in future versions this method may not work. Keep in mind, this guide uses GVT-g, so this only works with Intel Broadwell (5th gen) to Comet Lake (10th gen).


1. Access the Shell

2. run $ cli

3. run > system advanced update kernel_extra_options="i915.enable_gvt=1 i915.enable_guc=0"

4. run > exit

5. Reboot the System

6. Access the Shell again

7. run $ sudo su -

8. run # modprobe kvmgt mdev vfio-iommu-type1

9. find out id/path from the graphics card, in my case 0000:00:02.0, you can use lspci -D -nn

10. Look which types are supported by your gpu, with # ls /sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types
10.1. In my Case with an I5-6600 its i915-GVTg_V5_4 and i915-GVTg_V5_8
10.2. See what fits best for the use case, you can see what the individual modes can do with # cat /sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types/i915-GVTg_V5_4/description (A lower number is better, because they have a higher resolution, more memory and more performance)

11. Generate a uuid or just use this one 37d7d57d-de7b-47e0-ae05-d6e69b98d629

12. Create a vgpu # echo "37d7d57d-de7b-47e0-ae05-d6e69b98d629" > "/sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types/i915-GVTg_V5_4/create"

13. run # ln -s /run/truenas_libvirt/libvirt-sock /var/run/libvirt/libvirt-sock

14. Now search for the VM ID with, # virsh list

15. Edit the XML with # virsh edit 1_my_vm
15.1. Paste the following (abbove the last two lines)

Code:

    <hostdev mode='subsystem' type='mdev' managed='no' model='vfio-pci' display='off'>
      <source>
        <address uuid='af86b65f-13a3-4c17-a0f3-2b2b3aa573f9'/>
      </source>
    </hostdev>

16. Now run the VM with # virsh start 1_my_vm

Important Note
Only start via the command above, otherwise TrueNAS will overwrite the config, also disable autostart for the vm. Exit the vm via the UI does not change the config.

Not all changes are permanent, so here is a very simple script that sets all the required things and starts the VM when TrueNAS starts. Adapt it to your needs

Code:

#!/bin/zsh

modprobe kvmgt mdev vfio-iommu-type1

ln -s /run/truenas_libvirt/libvirt-sock /var/run/libvirt/libvirt-sock

echo "37d7d57d-de7b-47e0-ae05-d6e69b98d629" > "/sys/devices/pci0000:00/0000:00:02.0/mdev_supported_types/i915-GVTg_V5_4/create"

virsh start 1_my_vm

Store this in a dataset that is not encrypted or automatically decrypted at boot time.

Make this runnable with # chmod u+x myscript.sh

Under System Settings -> Advanced -> Init/Shutdown Scripts you can choose the script and run at Post Init

Relevant sources
https://wiki.archlinux.org/title/Intel_GVT-g
https://www.truenas.com/community/threads/21-02-alpha-1-virsh-commands-fail.91229/post-636109

 

[arkhoo]

Does anyone try it in 23.10-RC.1?

 

[Lipsum Ipsum]

Does anyone try it in 23.10-RC.1?

My CPU (i7-12700) doesn't support GVT-g so I can't test this. However I've accomplished very similar. I got SR-IOV working with Cobia, passing the iGPU as a VF (virtual function, think "virtual GPU" in this case) to a Ubuntu 23.04 VM. That VM in turn is running Jellyfin as a docker container with QuickSync hardware acceleration enabled.

To accomplish the above, I did need to break the metaphorical "warranty void if broken" sticker for my Scale installation using a custom compiled i915 driver enabling SR-IOV support.