Kubernetes and internal DNS.

[crkinard]

I am having issues resolving DNS names inside Kubernetes. From pod to pod.

The pod is set to use 'ClusterFirst' but I cannot resolve any other pod.

root@plex-meta-manager-ix-chart-66cbb7b6fb-nzwsk:/# nslookup plex.ix-plex
Server: 172.17.0.10
Address: 172.17.0.10:53

** server can't find plex.ix-plex: NXDOMAIN

** server can't find plex.ix-plex: NXDOMAIN

This is going by this page: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/

 

[SnoppyFloppy]

If your app is simply called "plex the internal dns name should be plex.ix-plex.svc.cluster.local

You can use the domain name "calculator" at the bottom of this page to "calculate" the internal dns name: Truecharts - linking apps internally

Alternatively you can use HeavyBullets heavysctipts which have a neat feature to list all your running apps internal dns names along their port: HeavyBullets - HeavyScript

 

[crkinard]

Still a no go.

root@plex-meta-manager-ix-chart-75459f5b84-b6cx7:/# ping plex.ix-plex.svc.cluster.local
ping: bad address 'plex.ix-plex.svc.cluster.local'
root@plex-meta-manager-ix-chart-75459f5b84-b6cx7:/# nslookup plex.ix-plex.svc.cluster.local
Server: 172.17.0.10
Address: 172.17.0.10:53

** server can't find plex.ix-plex.svc.cluster.local: NXDOMAIN

** server can't find plex.ix-plex.svc.cluster.local: NXDOMAIN

Will try the scripts in a bit.
Several bits later...

Code:

#    Name               DNS_Name                                                           Port
-    ----               --------                                                           ----
1)   freshrss           freshrss-ix-chart.ix-freshrss.svc.cluster.local
2)   heimdall           heimdall-ix-chart.ix-heimdall.svc.cluster.local
3)   lancache           lancache-ix-chart.ix-lancache.svc.cluster.local
4)   mariadb            mariadb-ix-chart.ix-mariadb.svc.cluster.local
5)   plex               plex-ix-chart.ix-plex.svc.cluster.local
6)   plex-meta-manager  plex-meta-manager-ix-chart.ix-plex-meta-manager.svc.cluster.local
7)   postgres           postgres-ix-chart.ix-postgres.svc.cluster.local
8)   tautulli           tautulli-ix-chart.ix-tautulli.svc.cluster.local
9)   transmission       transmission-ix-chart.ix-transmission.svc.cluster.local
10)  wikijs             wikijs-ix-chart.ix-wikijs.svc.cluster.local

Code:

root@plex-meta-manager-ix-chart-75459f5b84-b6cx7:/# nslookup plex-ix-chart.ix-plex.svc.cluster.local
Server:         172.17.0.10
Address:        172.17.0.10:53

** server can't find plex-ix-chart.ix-plex.svc.cluster.local: NXDOMAIN

** server can't find plex-ix-chart.ix-plex.svc.cluster.local: NXDOMAIN

root@plex-meta-manager-ix-chart-75459f5b84-b6cx7:/# ping plex-ix-chart.ix-plex.svc.cluster.local
ping: bad address 'plex-ix-chart.ix-plex.svc.cluster.local'

 

[crkinard]

Well, I sort have fixed the issue but overall, an undesirable result.

If I set a port forward, then DNS will resolve between pods. But then this seems to require me to use the <nodeip>:<port> to connect to the pod from outside which is one huge thing I do not want to do (having to remember what port what service is on, can't have two pods use port 80). I want each pod to have its own external IP, so I do not have to forward anything.

Basically, this that I finally came across.

Inter Pod network communication

Should two deployed pods (via two separate docker images) be able to communicate with each other? If so, how? I've deployed a 'db' image and an application container, each has its own pod, in its own namespace. I see two services listed in kubectl, one for each of the deployed containers...

www.truenas.com

Side note:
I have no idea why the GUI puts each pod in its own namespace. Seem rather dumb if I am understanding what namespaces are for.

 

[SnoppyFloppy]

What most people do and what I I would recommend you to, is to have a look at truecharts traefik app. It’s a reverse proxy which will lets you access your apps on your own domain, eg. plex.your-domain.io

Note that there are a few prerequisites:
1) Put the scale ui on owner ports
2) Get your own domain (from 10-15$/year) 3) get an acme cert

All of this is described on truecharts’ website.

 

[crkinard]

Godwin's Law of networking.

Everything ends up bringing up traefik.

Not about to buy a domain name for something that will never see outside access.

Just love how basically every single doc i read on this says it should 'just work. yet it does not or no doc goes into details on what SPECIFICALLY needs to be done to make it work. Even Kub's own docs say DNS should just... work.

I have to say the more and more I dig the more inclined I am to teach myself ZFS and rolling my own.

 

[SnoppyFloppy]

Everything ends up bringing up traefik.

Well kubernetes is really an industrial tool more than a homelab tool. It's used by some of the biggest cloud native providers out there. So I think it's pretty unusual to run user-facing apps in kubernetes without also deploying an ingress.

Not about to buy a domain name for something that will never see outside access.

One big advantage to have a public domain is that you can get an acme cert and avoid these irritating warnings regarcing self-signed certs and/or the hassle of adding your own certs to every browser you use. That alone is reason enough for me and is also count if you don't expose your apps to the internet.

I have to say the more and more I dig the more inclined I am to teach myself ZFS and rolling my own.

Well more power to you. Kubernetes is a great skill to know and can even help you to land a job in IT.

 

[Heavy]

Still a no go.


Will try the scripts in a bit.
Several bits later...

Code:

#    Name               DNS_Name                                                           Port
-    ----               --------                                                           ----
1)   freshrss           freshrss-ix-chart.ix-freshrss.svc.cluster.local
2)   heimdall           heimdall-ix-chart.ix-heimdall.svc.cluster.local
3)   lancache           lancache-ix-chart.ix-lancache.svc.cluster.local
4)   mariadb            mariadb-ix-chart.ix-mariadb.svc.cluster.local
5)   plex               plex-ix-chart.ix-plex.svc.cluster.local
6)   plex-meta-manager  plex-meta-manager-ix-chart.ix-plex-meta-manager.svc.cluster.local
7)   postgres           postgres-ix-chart.ix-postgres.svc.cluster.local
8)   tautulli           tautulli-ix-chart.ix-tautulli.svc.cluster.local
9)   transmission       transmission-ix-chart.ix-transmission.svc.cluster.local
10)  wikijs             wikijs-ix-chart.ix-wikijs.svc.cluster.local

Code:

root@plex-meta-manager-ix-chart-75459f5b84-b6cx7:/# nslookup plex-ix-chart.ix-plex.svc.cluster.local
Server:         172.17.0.10
Address:        172.17.0.10:53

** server can't find plex-ix-chart.ix-plex.svc.cluster.local: NXDOMAIN

** server can't find plex-ix-chart.ix-plex.svc.cluster.local: NXDOMAIN

root@plex-meta-manager-ix-chart-75459f5b84-b6cx7:/# ping plex-ix-chart.ix-plex.svc.cluster.local
ping: bad address 'plex-ix-chart.ix-plex.svc.cluster.local'

What's weird is heavyscript didn't pull your ports for your applications, unless you cut those out.

The ports definitely should populate with the DNS names.

What's the output of

Code:

cli -c 'app kubernetes config'

 

[darcos]

I'm having the same problem as OP

Here's my output

cli -c 'app kubernetes config' +--------------------+----------------------+ | id | 1 | | pool | tank | | cluster_cidr | 172.16.0.0/16 | | service_cidr | 172.17.0.0/16 | | cluster_dns_ip | 172.17.0.10 | | route_v4_interface | <null> | | route_v4_gateway | <null> | | route_v6_interface | <null> | | route_v6_gateway | <null> | | node_ip | 0.0.0.0 | | configure_gpus | true | | servicelb | true | | validate_host_path | true | | dataset | tank/ix-applications | +--------------------+----------------------+

and

# Name DNS_Name Port - ---- -------- ---- 1) heimdall heimdall.ix-heimdall.svc.cluster.local 10014 2) jackett jackett.ix-jackett.svc.cluster.local 9117 3) jellyfin jellyfin.ix-jellyfin.svc.cluster.local 8096 4) jellyseerr jellyseerr.ix-jellyseerr.svc.cluster.local 10241 5) netdata netdata.ix-netdata.svc.cluster.local 20489 6) prowlarr prowlarr.ix-prowlarr.svc.cluster.local 9696 7) qbittorrent qbittorrent.ix-qbittorrent.svc.cluster.local 10095 8) radarr radarr.ix-radarr.svc.cluster.local 7878 9) sonarr sonarr.ix-sonarr.svc.cluster.local 8989 10) vaultwarden vaultwarden.ix-vaultwarden.svc.cluster.local 10102 root@truenas[/mnt/tank/software/scripts/heavy_script]#

but when I try a DNS lookup it fails

nslookup radarr.ix-radarr.svc.cluster.local Server: 10.10.4.1 Address: 10.10.4.1#53 ** server can't find radarr.ix-radarr.svc.cluster.local: NXDOMAIN nslookup radarr.ix-radarr.svc Server: 10.10.4.1 Address: 10.10.4.1#53 ** server can't find radarr.ix-radarr.svc: NXDOMAIN


I don't know why this isn't working as it should. Any clues?

 

[darcos]

Anyone?

 

[majorgear]

Bump

 

[danb35]

but when I try a DNS lookup it fails

The app hostnames aren't expected to work, AFAIK, other than from other apps. So, your Sonarr app should be able to resolve radarr.ix-radarr.svc.cluster.local, but your NAS itself wouldn't. So this would be expected:

Code:

root@truenas[~]# nslookup radarr.ix-radarr.svc.cluster.local
Server:        192.168.1.1
Address:    192.168.1.1#53

** server can't find radarr.ix-radarr.svc.cluster.local: NXDOMAIN

But from another app, it should--e.g., this is from the shell of my Sonarr app:

Code:

$ nslookup radarr.ix-radarr.svc.cluster.local
Server:         172.17.0.10
Address:        172.17.0.10#53

Name:   radarr.ix-radarr.svc.cluster.local
Address: 172.17.166.73

Bump

It's highly unlikely that you're experiencing exactly the same problem, in the same environment, that another user posted over a year ago. If you're having problems and want help with them, you'd be better off to post details, including versions of all software in question.