Inter Pod network communication

[truecharts]

Some input on this:
- None of the pods needs services exposed (though in a lot of cases it would need it to be reached by the end-user)
- If using TrueCharts for any of the pods, we only support using the full DNS name, not the short one.
- When using databases within kubernetes, it's highly advicable to use a trusted resource that specialises in it (us, Bitnami etc.) instead of loading a docker container yourself. There are a LOT of caveats

 

[AcrylicShrimp]

In order to reach one pod's ("A") service from another ("B"), there are (at least) two requirements:
1. Pod A must have its DNS resolution set to use the Kubernetes internal DNS.
2. Pod B must have its service exposed. You can check using "k3s kubectl get svc -A -o wide" on the command line.
When creating a pod by "Launching a Docker Image", the service gets exposed as "type: NodePort" only when you set a port in "port forwarding". There does not seem to be a way to expose a service as "ClusterIP" in the GUI for launching docker images.

Once you've done this, you can reach pod B from pod A using the 'internal' port and the DNS name. For example, after launching a docker named "simpleweb" that services port 80, forwarded as port 9080, you can reach that service on port 80 using the name "simpleweb-ix-chart.ix-simpleweb" (and its full DNS name would be "simpleweb-ix-chart.ix-simpleweb.svc.cluster.local").

I forwarded ports, and I checked that the NodePort service has been created. So you mean that there's no way to communicate between pods because we can't create ClusterIP type services via webUI? Then how about .yaml files?

 

[AcrylicShrimp]

Some input on this:
- None of the pods needs services exposed (though in a lot of cases it would need it to be reached by the end-user)
- If using TrueCharts for any of the pods, we only support using the full DNS name, not the short one.
- When using databases within kubernetes, it's highly advicable to use a trusted resource that specialises in it (us, Bitnami etc.) instead of loading a docker container yourself. There are a LOT of caveats

Ahaaa, we can't use short DNS name right? So the right one is `nextcloud-postgresql-ix-chart.ix-nextcloud-postgresql.svc.cluster.local` for my case.

 

[rmr]

I forwarded ports, and I checked that the NodePort service has been created. So you mean that there's no way to communicate between pods because we can't create ClusterIP type services via webUI? Then how about .yaml files?

No, what I was trying to say that for pods created by — Launch Docker Image — you cannot have communication between pods without ALSO exposing the port outside the cluster (well at least I haven’t figured out how to, YMMV). It’s a GUI limitation since you can (temporarily) do a ‘k3s kubectl edit‘ on the service and change NodePort to ClusterIP and then it will be reachable cluster-only.
There are four possible settings for name resolution, and for my pods (all created by launch docker image) short names do work.

 

[Antede]

No, what I was trying to say that for pods created by — Launch Docker Image — you cannot have communication between pods without ALSO exposing the port outside the cluster (well at least I haven’t figured out how to, YMMV). It’s a GUI limitation since you can (temporarily) do a ‘k3s kubectl edit‘ on the service and change NodePort to ClusterIP and then it will be reachable cluster-only.
There are four possible settings for name resolution, and for my pods (all created by launch docker image) short names do work.

Instead of doing a 'k3s kubectl edit' on the server, why not create your own ClusterIP with a unique name which is not temporary, my ClusterIP was retained for multiple restarts. I also did not need to perform any "port forwarding".

I executed the command:
k3s kubectl expose deployment <deploymentname> --type=ClusterIP --port=<containerport> --target-port=<port-to-access-from-other-pods> --namespace=<namespace_name> --name=<unique ClusterIP name>

Using the command "k3s kubectl get deployments -A" you can determine <deploymentname> and <namespace_name>

From the source/requesting pod you will need to target this 'ClusterIP' using the name :
<unique ClusterIP name>.<namespace_name>.svc.cluster.local

 

[truecharts]

nodePort, ClusterIP and even Loadbalancer type services all also allow for communication within the cluster by defualt.
You do not have to use clusterIP to have inter-server communications.

nodePort and LoadBalancer are "Just" clusterIP + Extra-Exposed-Port.
This is normal and stock kubernetes behavior, which works the same on SCALE.

 

[Antede]

Thanks @truecharts , I certainly don't disagree but the advantage of using clusterIP over nodePort or LoadBalancer is that with clusterIP you can have interPod communication without having the Port available outside of Kubernetes.

This was probably more something I was working on than a specific requirement identified in this conversation.

 

[truecharts]

In this case, it might be advisable to use our Custom-App, which is a expert variant (requires kubernetes experience) of the "Launch-Docker" button with a LOT more options exposed (like: ClusterIP, Loadbalancer, MetalLB support, Ingress, networkPolicy, securityContext etc)

 

[Antede]

@truecharts Thanks for all the assistance. I have now setup the same using the 'custom-app' after some security problems on the deployment I was able to use the 'custom-app' successfully. Supported by truecharts discord channel.

Certainly resolves the inter-pod communications discussion for me.

 

[mazay]

In this case, it might be advisable to use our Custom-App, which is a expert variant (requires kubernetes experience) of the "Launch-Docker" button with a LOT more options exposed (like: ClusterIP, Loadbalancer, MetalLB support, Ingress, networkPolicy, securityContext etc)

Is the "Custom-App" option still there for TrueNAS-SCALE-22.12.0? I can't seem to find it, would appreciate a screenshot or some sort of guideline on where to find it.

 

[truecharts]

Is the "Custom-App" option still there for TrueNAS-SCALE-22.12.0? I can't seem to find it, would appreciate a screenshot or some sort of guideline on where to find it.

We will continue to offer Custom-App for the foreseeable future.
However, we've no intention to make guides for it. It's designed for people with previous experience with kubernetes, that already understand some/most of the offered settings.

 

[mazay]

I have tons of k8s experience and almost none with TrueNAS. Would you be able to point me in the direction of how to start using this feature?

 

[mazay]

ok, it turns out `Custom-App` is basically a generic chart/app offered by the TrueCharts repo, that's the piece I've been missing...