Use intel gpu hardware encoding in plex kubernetes deployment

[ornias]

So should these nvidia cuda packages be installed and configured by default or is this something we should be installing as an end user? It looks like a bunch of the nvidia stuff is installed by default, except for this cuda related stuff

It's alpha, drivers should be installed by default, but they are not (not the right version that is)

 

[Saberwolf]

Please provide you list of installed Nvidia pacakes so we can compare

 

[dalnew]

I'm still not seeing the nvidia-uvm devices in the new 21.02 release even after running "/usr/bin/nvidia-modprobe -c0 -u" all I see are:
crw-rw-rw- 1 root root 195, 254 Feb 17 20:26 /dev/nvidia-modeset
crw-rw-rw- 1 root root 195, 0 Feb 17 20:26 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Feb 17 20:26 /dev/nvidiactl

I guess we still need to install the nvidia-cuda-dev and nvidia-cuda-toolkit packages? I'm hesitant to do that as installing those two packages has quite the list of new libraries that would get installed alongside it... over 5 GB worth!) The packages that came installed in the latest 21.02 release were:

Show : Nvidia libs

glx-alternative-nvidia/bullseye,now 1.2.0 amd64 [installed,automatic]
libglx-nvidia0/bullseye,now 460.32.03-1 amd64 [installed,automatic]
libnvidia-cbl/bullseye,now 460.32.03-1 amd64 [installed,automatic]
libnvidia-cfg1/bullseye,now 460.32.03-1 amd64 [installed,automatic]
libnvidia-container-tools/bullseye,now 1.3.2-1 amd64 [installed,automatic]
libnvidia-container1/bullseye,now 1.3.2-1 amd64 [installed,automatic]
libnvidia-encode1/bullseye,now 460.32.03-1 amd64 [installed]
libnvidia-glcore/bullseye,now 460.32.03-1 amd64 [installed,automatic]
libnvidia-glvkspirv/bullseye,now 460.32.03-1 amd64 [installed,automatic]
libnvidia-ml1/bullseye,now 460.32.03-1 amd64 [installed,automatic]
libnvidia-ptxjitcompiler1/bullseye,now 460.32.03-1 amd64 [installed,automatic]
libnvidia-rtcore/bullseye,now 460.32.03-1 amd64 [installed,automatic]
nvidia-alternative/bullseye,now 460.32.03-1 amd64 [installed,automatic]
nvidia-container-runtime/bullseye,now 3.4.1-1 amd64 [installed,automatic]
nvidia-container-toolkit/bullseye,now 1.4.1-1 amd64 [installed]
nvidia-installer-cleanup/bullseye,now 20151021+13 amd64 [installed,automatic]
nvidia-kernel-common/bullseye,now 20151021+13 amd64 [installed,automatic]
nvidia-kernel-dkms/bullseye,now 460.32.03-1 amd64 [installed]
nvidia-kernel-support/bullseye,now 460.32.03-1 amd64 [installed,automatic]
nvidia-legacy-check/bullseye,now 460.32.03-1 amd64 [installed,automatic]
nvidia-modprobe/bullseye,now 460.32.03-1 amd64 [installed,automatic]
nvidia-persistenced/bullseye,now 460.32.03-1 amd64 [installed,automatic]
nvidia-smi/bullseye,now 460.32.03-1 amd64 [installed]
nvidia-support/bullseye,now 20151021+13 amd64 [installed,automatic]
nvidia-vulkan-common/bullseye,now 460.32.03-1 amd64 [installed,automatic]
nvidia-vulkan-icd/bullseye,now 460.32.03-1 amd64 [installed,automatic]

Note that I am still trying to use my current method of docker to run plex as I still can't use the built in plex chart/app. I'm not sure if using that built-in app somehow gets around this restriction of needing cuda dev packages installed ?

@ornias Do you know of any way to change, or copy and modify, the IxSystems plex chart? I see they added the GPU option, but the default chart still doesn't have the ability to mount custom Host Path Volumes or Volumes which I need for my custom Plex setup. They only allow you to enter three mappings: config, transcode, and data which isn't how I've setup all my directories. And unfortunately if I try to use the "Launch Docker Image" and create my own, the GPU mapping option doesn't exist.

 

[ornias]

@ornias Do you know of any way to change, or copy and modify, the IxSystems plex chart? I see they added the GPU option, but the default chart still doesn't have the ability to mount custom Host Path Volumes or Volumes which I need for my custom Plex setup. They only allow you to enter three mappings: config, transcode, and data which isn't how I've setup all my directories. And unfortunately if I try to use the "Launch Docker Image" and create my own, the GPU mapping option doesn't exist.
[/QUOTE]
Do i know of ways? Yes. Should you? No.
It takes some time (releases) for these things to crystalise out.

To be fair, even for TrueCharts the option to add unlimited extra mountpaths is still a To-Do....

 

[Saberwolf]

Please refer to JIRA ticket for solution to adding Nvidia Hardware support to plex app NAS-109192

 

[ornias]

Please refer to JIRA ticket for solution to adding Nvidia Hardware support to plex app NAS-109192

Thanks for the headsup for those fixes! :)

 

[jsclayton]

I've got an ASRockRack E3C246D4U motherboard with an i7-9700K and the fix that went in to 21.04 doesn't seem to be accounting for the on-board IPMI.

From that linked PR, it looks like the middleware is determining what GPUs are available by looking for "VGA compatible controller" in the output of lspci -D, and then later on the KubernetesGPUService will configure the appropriate plugins in k8s. On this system, where the BMC is the primary adapter, but the Intel GPU is also available, the output of lspci -D includes the following:

Code:

0000:00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630]
0000:07:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41)

I can manually add the Intel GPU plugin as described in OP and now the "Resources" section and the ability to add the GPU to my Plex container works. When I reboot, though, the plugin daemonset is removed, which appears to be expected by the last section of the gpu.py file in that PR.

The fix might be as simple as looking for "Display controller" in addition to "VGA compatible controller". Would it be best to re-open the Jira ticket, comment on that GHI, open a new ticket or GHI, or something else entirely? Thanks!

 

[ornias]

Code:

0000:00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630]
0000:07:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41)

You would want both, either VGA compatible controller OR displaycontroller.
You can just as well directly send out the few lines required in a PR

The fix might be as simple as looking for "Display controller" in addition to "VGA compatible controller". Would it be best to re-open the Jira ticket, comment on that GHI, open a new ticket or GHI, or something else entirely? Thanks!

No, this would be a new jira ticket.
It's a bug introduced after NAS-108532 and only on specific platforms (like yours).

iX doesn't use GHI btw.

 

[Saberwolf]

i suggest you wright up your command that you did to get it to work and use the System advanced settings until the proper fix is completed set up the command in INIT/Shutdown scripts

this process is pretty straight forward it will work through reboots and apply the resource to the kubernetes you can either run commands or create a script for now it is suggested to run it pre init. please update the above ticket or if it does not quite flow into your problem create new one and advice them of what you had done to overcome the limitations.

 

[jsclayton]

Opened https://jira.ixsystems.com/browse/NAS-110656 for tracking in case anyone else ends up here with the same issue.

 

[Saberwolf]

midclt call -job chart.release.update 'plex' '{"values":{"gpuConfiguration": {"intel.com/gpu": 1}}}' should add support to the kubernetes deployment
just replace 'plex' with the name of your plex app name the number rep how many gpus that container can use so if you have more than one increase that number of if my memory is right that is the first gpu that is loaded so if it is 2 it will allow the second gpu to be used.

 

[jsclayton]

midclt call -job chart.release.update 'plex' '{"values":{"gpuConfiguration": {"intel.com/gpu": 1}}}'

Unfortunately the post-init scheduled tasks run before the middleware has started. When the middleware starts it is explicitly deleting the Intel GPU plugin if it doesn't detect an Intel GPU. I may need a script that sleeps for some amount of time, applys the plugin, and restart the Plex container.

 

[ornias]

Unfortunately the post-init scheduled tasks run before the middleware has started. When the middleware starts it is explicitly deleting the Intel GPU plugin if it doesn't detect an Intel GPU. I may need a script that sleeps for some amount of time, applys the plugin, and restart the Plex container.

uhmm... It should be relatively easy to just write to PR and fix the behavior...
Isn't that a more efficient use of your time? ;-)