How much CPU and memory is being given to the Kubernetes node?

[docwhite]

Hello! I was wondering how much CPU and memory does TrueNAS SCALE give to the Kubernetes cluster node?

I am not very well versed with Kubernetes and container when it comes to checking resource and specs, so I tried running few commands that I would hope it would make sense:

First I figured out the namespace with k3s kubectl get namespaces.

Running k3s kubectl -n ix-mycloud describe nodes:

Code:

Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource            Requests   Limits
  --------            --------   ------
  cpu                 100m (2%)  0 (0%)
  memory              70Mi (0%)  170Mi (2%)
  ephemeral-storage   0 (0%)     0 (0%)
  hugepages-1Gi       0 (0%)     0 (0%)
  hugepages-2Mi       0 (0%)     0 (0%)
  gpu.intel.com/i915  0          0

I run a bash shell into the NextCloud chart I deployed from the catalogue with k3s kubectl -n ix-mycloud exec -ti mycloud-nextcloud-xxxxxxxxx-xxxxx bash
If I run free -ht it outputs:

Code:

              total        used        free      shared  buff/cache   available
Mem:          7.4Gi       5.1Gi       1.7Gi        51Mi       669Mi       2.1Gi
Swap:         2.0Gi          0B       2.0Gi
Total:        9.4Gi       5.1Gi       3.7Gi

If I jump into the pod and run top I get this:

Code:

top - 18:16:09 up  1:04,  0 users,  load average: 0.59, 0.74, 1.03
Tasks:  13 total,   1 running,  12 sleeping,   0 stopped,   0 zombie
%Cpu(s):  4.9 us,  3.5 sy,  0.0 ni, 90.1 id,  1.5 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   7611.5 total,   1699.3 free,   5243.1 used,    669.1 buff/cache
MiB Swap:   2047.9 total,   2047.9 free,      0.0 used.   2156.3 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
    138 root      20   0    8064   2244   1764 R   0.7   0.0   0:00.05 top
      1 root      20   0  278956  25440  16220 S   0.0   0.3   0:00.36 apache2
     30 www-data  20   0  316392  63400  44384 S   0.0   0.8   0:07.52 apache2
     32 www-data  20   0  314024  54092  37388 S   0.0   0.7   0:04.26 apache2
     33 www-data  20   0  320380  65316  42096 S   0.0   0.8   0:04.53 apache2
     34 www-data  20   0  314280  58924  41816 S   0.0   0.8   0:06.20 apache2
     35 www-data  20   0  388648  60592  43016 S   0.0   0.8   0:04.27 apache2
     39 www-data  20   0  389080  63336  45152 S   0.0   0.8   0:47.18 apache2
     75 www-data  20   0  360296  51308  36200 S   0.0   0.7   0:01.31 apache2
    110 www-data  20   0  389028  60536  42700 S   0.0   0.8   0:04.67 apache2
    111 www-data  20   0  313864  56948  40352 S   0.0   0.7   0:03.24 apache2
    116 www-data  20   0  286060  48984  34672 S   0.0   0.6   0:02.17 apache2
    129 root      20   0    3872   2416   1932 S   0.0   0.0   0:00.02 bash

Are the full specs from the host shared with the kubernetes cluster?

Thank you!

 

[ornias]

The "node" is supposed to just have all the ram and CPU it needs.
With K8S you can define limits on a per-pod basis.

I personally have not gone the route of doing so, but might in the future define such limits per pod. Because it prevents memory leaks inside pods taking down the rest of them...

 

[stavros-k]

The "node" is supposed to just have all the ram and CPU it needs.
With K8S you can define limits on a per-pod basis.

I personally have not gone the route of doing so, but might in the future define such limits per pod. Because it prevents memory leaks inside pods taking down the rest of them...

The limits was something i've been thinking the last days, It's also useful for power hungry apps.
Like Plex and Handbrake. I have limited the CPU on those 2 apps on Unraid (which im using until i turn to scale).
Handbrake runs 5days a week at least. So limiting is critical! Hopefully until scale RELEASE, we can have some sort of limiting :)

 

[ornias]

The limits was something i've been thinking the last days, It's also useful for power hungry apps.
Like Plex and Handbrake. I have limited the CPU on those 2 apps on Unraid (which im using until i turn to scale).
Handbrake runs 5days a week at least. So limiting is critical! Hopefully until scale RELEASE, we can have some sort of limiting :)

My current brainfart is to hardcode some very high values based on the host to at least prevent locking's, memory leaks and overloads. But thats really more of a 21.08 idea currently. Lets first get everything to run smoothly as non-root ;-)

 

[stavros-k]

My current brainfart is to hardcode some very high values based on the host to at least prevent locking's, memory leaks and overloads. But thats really more of a 21.08 idea currently. Lets first get everything to run smoothly as non-root ;-)

Sure, i think this will be easy to implement to each app (after the common library is ready of course) It will be a matter of some UI additions only.
but yea, running smooth is current priority!

 

[ornias]

Sure, i think this will be easy to implement to each app (after the common library is ready of course) It will be a matter of some UI additions only.
but yea, running smooth is current priority!

I'm not sure if it's ever going to become a UI feature actually. I don't plan to expose all k8s features in the installation wizard.
This one most likely gets added as a fail-safe with minimal limits placed on it which are dynamically determined.

 

[stavros-k]

Each Container has a request of 0.25 cpu and 64MiB (226 bytes) of memory. Each Container has a limit of 0.5 cpu and 128MiB of memory.

This is surely not true, i use the handbrake app and it pegs CPU to 95%, haven't used any memory intensive app yet to see.

Also what is this link? Is this relevant to something here?

[mod note: spam link trimmed -JG]

 

[ornias]

This is surely not true, i use the handbrake app and it pegs CPU to 95%, haven't used any memory intensive app yet to see.

Also what is this link? Is this relevant to something here?

Stavos, this was a spam account. Look at the link he added.
it's a new form of forum spamming: AI generates a somewhat plausible answer and adds a link to the post.

 

[stavros-k]

Stavos, this was a spam account. Look at the link he added.
it's a new form of forum spamming: AI generates a somewhat plausible answer and adds a link to the post.

Wow thats next level AI :D Haven't seen it on forums, only in Google Searches

 

[ornias]

Wow thats next level AI :D Haven't seen it on forums, only in Google Searches

It's the OpenAI framework they use, this video explains how to do it quite well:

I asked an AI for video ideas, and they were actually good

I didn't expect this to work so well. • Includes text generated by OpenAI's GPT-3 at my request: https://openai.com • Art by Chris Quay: https://www.chrisqua...

www.youtube.com

 

[jgreco]

I've resized Wuckert's container to oblivion

I'm also trimming a bit in thread comments to get rid of Wuck-krap.

Upon review, I guess it was actually just the one link.

 

[ornias]

I've resized Wuckert's container to oblivion

I'm also trimming a bit in thread comments to get rid of Wuck-krap.

Sorry for the intermezzo and thanks for the cleanup... :)

 

[jgreco]

Sorry for the intermezzo and thanks for the cleanup... :)

I was impressed that two people reported it within minutes.

There are some telltales such as that the poster is often from "new york" (no caps) or "USA"; as moderators, we can also see a lot of it comes from India. It may or may not be AI, it could also be IHI (inexpensive human idiots). We've actually been fighting this for a few years, sometimes more successfully, sometimes less, but I can promise you every report gets looked at, and my itchy trigger finger likes the occasional exercise. Thanks for being the forum's eyes and ears when there isn't always a moderator available to be keeping an eye out.