Scale 22.02-RC.2 Ryzen 2200G APU for application container

hogen

Dabbler
Joined
Dec 30, 2021
Messages
10
Is it possible to get the APU or GPU part of Ryzen 2200G or any other amd ryzen to be available for the application containers? Specifically jellyfin for transcoding. Im uncertain if it is even possible to do this with Scale 22.02-RC.2 application or if there are some unknown problem I have.
How can I verify that the container or pod has access?
I have never used docker or kubernets before. My understanding is that docker is different from a classic VM and that it would be easier to passthrough a apu or igpu.

In Kubernetes settings GPU support is enable. Nothing is showing when I try to use it for Jellyfin only the label "GPU Configuration" but ther are no inputs.
1641074775488.png


Most other posts is about dedicated nvidia gpu, but it seams that jellyfins docker-image can use the APU. Most guides for jellyfin AMD gpu require amdgpu-pro on the host but is it safe to run apt update to install it? There are also some threds that say k8s-device-plugin https://github.com/RadeonOpenCompute/k8s-device-plugin is needed for kubernets.

Im haveing a hard time troubleshooting since my knowledge is limited. Im useing 22.02-RC.2 with Jellyfin 10.7.7_9.0.28 from truecharts. Scale 22.02-RC.2 is running fine and Jellyfin is working as expected without HW transcoding but no other problem.
 

truecharts

Guru
Joined
Aug 19, 2021
Messages
787
First off: SCALE does not run normal docker, it runs kubernetes.
GPU passthrough on kubernetes is not the same as normal docker.

SCALE only includes the required GPU support for Nvidia and Intel. No AMD GPU support is included.

It might be best making a Jira suggestion for it and calling it a day. Even if you install the required drivers and device plugin(s), it would still not shop up in the GUI anyway
 

FrostyCat

Explorer
Joined
Jan 4, 2022
Messages
79
@hogen It works with AMD as well but you need to run the device plugin. This is unsupported as far as I know so you need to know what you're doing.

I'm running a Ryzen 4750G Pro and the UI will notice (I was surprised it did, it was a nice surprise though) you have the GPU API available and offer to mount it to the pod. Please note you can only mount it in one single pod, no sharing is possible. Here's a snapshot of the GPU config for Truecharts Jellyfin (Jellyfin won't be able to use it because the Truecharts container image is based on an older Debian with too old libs, e.g. Mesa).

Let me know if you want to try it yourself but be warned, if it breaks you get to keep the pieces (it won't break though :) )

1641337559280.png


1641337592813.png
 

truecharts

Guru
Joined
Aug 19, 2021
Messages
787
@hogen It works with AMD as well but you need to run the device plugin. This is unsupported as far as I know so you need to know what you're doing.

I'm running a Ryzen 4750G Pro and the UI will notice (I was surprised it did, it was a nice surprise though) you have the GPU API available and offer to mount it to the pod. Please note you can only mount it in one single pod, no sharing is possible. Here's a snapshot of the GPU config for Truecharts Jellyfin (Jellyfin won't be able to use it because the Truecharts container image is based on an older Debian with too old libs, e.g. Mesa).

Let me know if you want to try it yourself but be warned, if it breaks you get to keep the pieces (it won't break though :) )

View attachment 52006

View attachment 52007
Interesting thanks for the headsup!
In that case we can throw the device plugin in as an App in a few weeks :)

One note:
We don't build containers, we just use the official container for Jellyfin. ;-)
 

FrostyCat

Explorer
Joined
Jan 4, 2022
Messages
79
Interesting thanks for the headsup!
In that case we can throw the device plugin in as an App in a few weeks :)

One note:
We don't build containers, we just use the official container for Jellyfin. ;-)
Yeah, should be possible as an app since k3s doesn't have much security enabled by default. I know you don't build the container images, that would be too much work, would it be easy to switch Jellyfin to a different/newer source?

Also, is the a quick primer on the Truecharts common chart somewhere or do I have to sit down and read it? I would like to contribute a few apps (some are on your apps to add list) I would like to see in Truecharts.

Cheers :)
 

truecharts

Guru
Joined
Aug 19, 2021
Messages
787
Yeah, should be possible as an app since k3s doesn't have much security enabled by default. I know you don't build the container images, that would be too much work, would it be easy to switch Jellyfin to a different/newer source?

Simply put: We knew it was already possible, but we where kinda holding-off on it unless it actually proved to be working with the GUI selector... No need to add non-functional Apps :')

Also, is the a quick primer on the Truecharts common chart somewhere or do I have to sit down and read it? I would like to contribute a few apps (some are on your apps to add list) I would like to see in Truecharts.

Not really a consolidated primer byond an explaination what it is.
But the defaults and their explaination should be available on the website and in code in the values.yaml for the common chart.
Combined with the existing Apps that should mostly be self-explainatory :)
 

hogen

Dabbler
Joined
Dec 30, 2021
Messages
10
@hogen It works with AMD as well but you need to run the device plugin. This is unsupported as far as I know so you need to know what you're doing.
Excellent, many thanks. Obviously I don't know what I'm doing :smile: . The only way to gain knowledge is by doing and studying. I will read up on installing drivers in scale, k3s and kubernets. Will worry about jellyfin support later.

I was linking to a k8s device driver is that the right thing, you where talkting about k3s? Guess I need to learn the differens between k3s and k8s.

What is the recommended or least problematic way to install amd driver on the host? Ihave read that just running apt update can brick things in Scale, but its hard to know if they were doing other stuff wrong or if it was just a alpha problem.

Let me know if you want to try it yourself but be warned, if it breaks you get to keep the pieces (it won't break though :) )
I will try this and keep the pieces for my self, failure is the best teacher after all :wink: .
 
Last edited by a moderator:

FrostyCat

Explorer
Joined
Jan 4, 2022
Messages
79
@hogen Excellent :)

Kubernetes (in no matter what flavour), may seem daunting at first and I agree, it has a steep learning curve but you'll soon notice how logical it can be. You'll soon notice how easier and safer it is to run your containers in Kubernetes than standalone with Docker or something else.

The main difference between vanilla K8s and K3s is that k3s is a very much stripped down compared to vanilla and lack the complexity and many of the plugins and feature gates in the standard K8s. There's is more, but it boils down to very particular features you would want in very specific scenarios so don't worry too much about it, in this case k3s is enough.

In this case, the AMD device plugin runs a pod as a DaemonSet (a DS is a type of Kubernetes workload that would run a replica of a pod on all nodes in a cluster - in our case, since SCALE is single node we do not need to worry about it). In Kubernetes lingo, a pod is a group of containers - this will be important later so make a note of it - the simplest pod will run only 1 container.

To run it, you'll have to do 2 things:

1. Create its own namespace (as opposed to me who ran it in the kube-system namespace)
2. Deploy the DaemonSet

So, let's start.

1. Run this command to create the namespace:
Code:
k3s kubectl create namespace amd-device-plugin

2. Copy the code below and save it as a file named amd.yaml somewhere on the Truenas machine. I would use the shell, run
Code:
nano amd.yaml
Then shift-insert to paste, ctrl-o, enter, ctrl-x

3. Apply the yaml:
Code:
k3s kubectl -n amd-device-plugin apply -f amd.yaml


Now you can check if the plugin runs:
Code:
k3s kubectl -n amd-device-plugin get pods


You should see something like in my screenshot below and the UI should detect the plugin. Let me know if you run into ay kind of issues.
Congrats on your first kube experience :).

1641417997721.png


YAML file follows:
Code:
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: amdgpu-device-plugin
spec:
  selector:
    matchLabels:
      name: amdgpu-dp-ds
  template:
    metadata:
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ""
      labels:
        name: amdgpu-dp-ds
    spec:
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
      containers:
      - image: rocm/k8s-device-plugin
        name: amdgpu-dp-cntr
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: ["ALL"]
        volumeMounts:
          - name: dp
            mountPath: /var/lib/kubelet/device-plugins
          - name: sys
            mountPath: /sys
      volumes:
        - name: dp
          hostPath:
            path: /var/lib/kubelet/device-plugins
        - name: sys
          hostPath:
            path: /sys
 

hogen

Dabbler
Joined
Dec 30, 2021
Messages
10
Thank you for excellent explanation.
I have it show up exactly as you have under GPU configuration now :smile:. Even inside jellyfin pod /dev/dri has card0 and renderD128. And I did understand what I was doing. Learned allot. Kubernetes documentation is actually good for beginners there is just so mutch to go through new concepts and lingo https://kubernetes.io/docs/tasks/tools/. Need to take a brake and continue this next weekend.

But guess I have to wait for jellyfin to update the driver in the container to actually make use of the GPU.
Im having no luck updating the driver inside jellyfin pod. Is there something similar to Docker_mods for Kubernetes? I have not seen what that corresponding thing is yet if there is any. https://github.com/linuxserver/docker-mods/tree/jellyfin-amd That looks like the only way if you dont want to build your own container. Maybe its easier too just build it.
 

FrostyCat

Explorer
Joined
Jan 4, 2022
Messages
79
Glad it worked for you. If you have Kubernetes questions don't hesitate to ask, I have a few years of experience :)

Regarding Jellyfin, I do indend to test a custom image with up to date libraries just to confirm it works. Stay tuned, I'll probably add a comment to this post as soon as I have data.
 

ChanningHe

Cadet
Joined
Jan 7, 2022
Messages
1
I also using 2200g but is “pro”,which is support ecc memory. I solved this problem same way in RC1. But it still had some problem with HW. When I use the Jellyfin, I can’t play H265 videos with HW. It also had a problem with Plex.(I used a special docker container to got HW to work on amd gpu.) But it still can’t transcode 4K or HEVC videos on truenas scale. It likes the driver on SCALE is too old. e.g. Mesa VA-API .
 

FrostyCat

Explorer
Joined
Jan 4, 2022
Messages
79
I just need to find some free time for this but I intend to build Jellyfin on Debian 11 the AMD Radeon proprietary dirver included, latest ffmpeg and latest Mesa, just to see how broken it still is. Fingers crossed.
 

hogen

Dabbler
Joined
Dec 30, 2021
Messages
10
Excellent!
I have completed the learning path at https://kube.academy/paths and i recommend them for beginners .

I have experimented with my own built images of Jellyfin nightly. Jellyfins next version 1.8 is built on debian:stable-slim (debian 11) and dose install mesa for AMD VAAPI it also has other improvements. It dose in my testing use my APU to some extent when transcoding 4k HDR HEVC to H264 1080P, that was not possible without it. Jellyfins 1.8 is in alpha 5 now so im guessing it will reach beta-status any day.
 

FrostyCat

Explorer
Joined
Jan 4, 2022
Messages
79
As said, it was added a few days ago ;-)
Am I looking in the wrong place? I can see it in apps but not in catalog. Is there a promotion process?

Not here:

But here:
 
Top