Kubernetes DNS and Service Discovery Troubleshooting

Serdar Tekin
Serdar Tekin
Co-Founder & CTO
Updated Nov 17, 202614 min read
Written for: Developers and platform teams diagnosing Kubernetes Service discovery, CoreDNS, resolver, and in-cluster connectivity failures
KubernetesDNSCoreDNSService DiscoveryNetworkingTroubleshooting
Last reviewed:
Guide
Kubernetes DNS and Service Discovery Troubleshooting
rafftechnologies.com/learn
On this page

Key Takeaways

Troubleshoot Kubernetes DNS by separating name resolution from Service reachability. Start with a known cluster lookup, inspect the Pod resolver, verify the Service and EndpointSlices, then check CoreDNS Pods, Service, endpoints, logs, and Corefile only when evidence points to cluster DNS. For hostNetwork Pods on Linux that still need cluster DNS, Kubernetes documents ClusterFirstWithHostNet. Restrictive egress policy should allow both UDP and TCP port 53 where DNS access is required.

Free control plane

Run your services on managed Kubernetes, worker nodes from $9.99 a month.

CNCF-certified Kubernetes with a free control plane. Add nodes when you need them, with no egress fees.

Kubernetes DNS is the in-cluster name-resolution layer that lets Pods discover Services by stable names instead of tracking changing Pod IP addresses.

For small teams, most Kubernetes DNS incidents become easier once the failure is separated into the right layer. A short Service name may be wrong for the caller's namespace, the Service may exist without usable endpoints, a Pod may have unexpected resolver settings, CoreDNS may be unhealthy, or a NetworkPolicy may block the DNS path. Raff Technologies treats these as separate checks rather than one generic “networking problem.”

This guide focuses on that diagnostic boundary. It explains how Service names are formed, how namespaces affect lookup, what CoreDNS actually provides, how Service and EndpointSlice failures can look like DNS failures, and how to distinguish cluster DNS problems from application-connectivity problems.

Quick Kubernetes DNS troubleshooting checklist

If a Pod cannot resolve a Kubernetes Service, start with a known cluster name and move outward one layer at a time.

  1. Test cluster DNS from a Pod
Bash
nslookup kubernetes.default

If this fails, inspect the Pod resolver and cluster DNS before debugging the application Service.

  1. Check the Pod resolver
Bash
cat /etc/resolv.conf

Confirm that the nameserver and search domains match the cluster's expected DNS configuration.

  1. Verify the Service and namespace
Bash
kubectl get svc <service> -n <namespace>

A misspelled Service or wrong namespace can return NXDOMAIN even when CoreDNS is healthy.

  1. Verify ready endpoints
Bash
kubectl get endpointslice -n <namespace> \ -l kubernetes.io/service-name=<service>

If the name resolves but the Service has no usable endpoints, the failure is after DNS.

  1. Check CoreDNS only when the failure is broader
Bash
kubectl get pods -n kube-system -l k8s-app=kube-dns kubectl get svc -n kube-system kube-dns kubectl get endpointslice -n kube-system \ -l kubernetes.io/service-name=kube-dns kubectl logs -n kube-system -l k8s-app=kube-dns
  1. Separate internal DNS from upstream DNS

Test both a cluster Service name and an external domain. If Kubernetes names resolve but external names fail, investigate CoreDNS forwarding, the node/upstream resolver path, and egress controls instead of changing Service discovery.

This sequence follows the same diagnostic boundary used in the Kubernetes DNS debugging documentation: verify the client resolver, DNS Pods, DNS Service, and endpoints before changing cluster-wide configuration.

Kubernetes DNS turns Service identity into stable names

Pods are disposable. Their IP addresses can change as workloads roll out, restart, scale, or move between nodes. Kubernetes Services provide a stable identity in front of those Pods, while cluster DNS makes that identity available by name.

A simplified request path looks like this:

Text
Application Pod ↓ DNS query Cluster DNS ↓ resolves Service name ↓ stable Service identity Eligible endpoints ↓ Application Pods

For a normal Service, DNS records resolve to the Service's cluster IP. For a headless Service, DNS can return the addresses of the backing Pods instead. That difference is expected behavior, not evidence that DNS is inconsistent.

Kubernetes also creates SRV records for named Service ports, allowing clients that understand SRV lookups to discover both a port and the relevant Service target.

The important boundary is this: DNS answers the naming question; it does not prove the application behind that name is healthy.

A Service name can resolve successfully while the connection still fails because:

  • the Service selector matches no ready Pods;
  • EndpointSlices contain no usable endpoints;
  • the target port is wrong;
  • NetworkPolicy blocks the traffic;
  • the application is not listening;
  • readiness removes Pods from eligible endpoints;
  • the network path fails after name resolution.

This is why a successful nslookup is only one part of a connectivity diagnosis.

Service discovery depends on namespaces and DNS search paths

Kubernetes Service names are namespace-aware.

A Pod can normally resolve a Service in its own namespace by using the short Service name:

Text
api

A Pod in another namespace should normally qualify the name with the target namespace:

Text
api.production

The fully qualified form is commonly:

Text
api.production.svc.cluster.local

cluster.local is a common default cluster domain, but clusters can use another domain. Do not hard-code it into application logic without knowing the cluster's actual DNS configuration.

Kubernetes configures a Pod's resolver search path so short names can be expanded automatically. A typical /etc/resolv.conf inside a Pod includes search domains similar to:

Text
<namespace>.svc.cluster.local svc.cluster.local cluster.local

The exact values depend on cluster configuration.

This creates one of the most common debugging patterns: a short name fails while a namespace-qualified or fully qualified name works. That usually points to a naming or resolver-search issue rather than a broken Service.

A compact check is:

Bash
cat /etc/resolv.conf

Then compare what the application is requesting with the namespace where the Service actually exists.

At Raff, the DNS troubleshooting rule is to separate naming from reachability: verify the expected name and namespace first, then the Service and EndpointSlices, then cluster DNS health, and only after that the underlying network path.

That order avoids changing firewalls or cluster networking to solve what is actually a namespace typo.

DNS troubleshooting starts by separating resolution from connectivity

The first useful question is not “Is Kubernetes networking broken?” It is what exactly failed?

A diagnostic matrix makes the failure layer visible.

SymptomFirst checkLikely layer
Short Service name fails, qualified name worksNamespace and Pod search domainsNaming / resolver search
One Service name returns NXDOMAINService name and namespaceMissing or incorrect Service identity
Service name resolves, connection is refused or times outService ports and endpointsService/application/network path
One Pod cannot resolve any cluster namesPod /etc/resolv.conf and dnsPolicyPod resolver configuration
Many Pods cannot resolve cluster namesCoreDNS Pods, Service, logs, endpointsCluster DNS
Cluster Service names work, external domains failCoreDNS forwarding and upstream resolver pathUpstream DNS / egress
DNS queries time out rather than return NXDOMAINReachability to cluster DNSNetworkPolicy / network path
Headless Service returns several Pod IPsService type and endpointsExpected headless discovery

A simple lookup against the built-in Kubernetes Service is often useful because it tests cluster DNS independently from your application Service:

Bash
nslookup kubernetes.default

If that works but api.production does not, the investigation can move toward the specific Service and namespace instead of immediately treating CoreDNS as unhealthy.

If both fail from one Pod, inspect that Pod's resolver settings. If they fail from many unrelated Pods, cluster DNS becomes a stronger suspect.

Raff's troubleshooting framework separates four layers: name, Service endpoints, cluster DNS, and network reachability.

Service selectors and EndpointSlices can look like DNS failures

A Service can exist and resolve correctly while still sending traffic nowhere useful.

The Service selector determines which Pods belong behind the Service. Kubernetes represents the resulting backend addresses through EndpointSlices. If labels do not match, Pods are not ready, or ports are wrong, the DNS name may be perfectly healthy while the application remains unreachable.

Useful inspection commands include:

Bash
kubectl get svc <service> -n <namespace> kubectl get endpointslice -n <namespace> \ -l kubernetes.io/service-name=<service>

The key questions are:

  • Does the Service exist in the namespace the caller expects?
  • Does the selector match the intended Pods?
  • Are endpoints present?
  • Are those endpoints ready?
  • Does targetPort match the port the application actually listens on?
  • Is the client using the correct Service port?

This distinction matters because changing CoreDNS will not fix an empty EndpointSlice.

Likewise, if DNS returns the correct Service address but the application times out, focus on the traffic path after resolution. Check endpoints, readiness, network policies, and application listeners before modifying DNS configuration.

The related Kubernetes Services Explained: Ingress and Load Balancers guide owns the broader Service exposure model. This article stays on discovery and diagnosis.

CoreDNS health and Pod resolver settings narrow cluster-wide failures

CoreDNS is the standard DNS component used for Kubernetes Service discovery in current Kubernetes deployments. Even when CoreDNS is the implementation, the DNS Service and labels commonly retain the kube-dns name for compatibility.

When failures affect many Services or many Pods, inspect the cluster DNS components instead of debugging one application at a time.

Useful checks include:

Bash
kubectl get pods -n kube-system -l k8s-app=kube-dns kubectl logs -n kube-system -l k8s-app=kube-dns kubectl get svc -n kube-system kube-dns kubectl get endpointslice -n kube-system \ -l kubernetes.io/service-name=kube-dns

When the Pods, Service, and endpoints are healthy but resolution is still failing, inspect the CoreDNS configuration before changing it:

Bash
kubectl -n kube-system get configmap coredns -o yaml

Look for the kubernetes plugin that serves cluster records and the forward path used for non-cluster names. Treat Corefile edits as a cluster-wide change: collect logs and reproduce the failing query first rather than restarting or rewriting CoreDNS as a generic first step.

These checks answer four separate questions:

  1. Are the DNS Pods running?
  2. Do their logs show resolution or forwarding errors?
  3. Does the cluster DNS Service exist?
  4. Does that Service have usable endpoints?

The Pod's own resolver policy also matters.

ClusterFirst is the normal cluster-first DNS behavior for Pods. Default makes a Pod inherit the node's resolver configuration. None requires explicit DNS settings through dnsConfig. Host-networked Pods need special consideration because their resolver behavior differs from ordinary Pods. For Linux Pods that use hostNetwork: true but still need cluster DNS, Kubernetes documents dnsPolicy: ClusterFirstWithHostNet; using ordinary ClusterFirst with host networking falls back to node-style Default behavior.

When only one workload has DNS problems, compare its Pod specification and /etc/resolv.conf with a working workload before changing cluster-wide CoreDNS configuration.

Kubernetes documentation also calls out node-level resolver issues as a possible source of CoreDNS forwarding loops, particularly when the node points at a local stub resolver rather than an upstream resolver file suitable for the cluster DNS configuration. That is an infrastructure-level diagnosis, not the first assumption for every failed lookup.

NetworkPolicy and upstream DNS can break resolution after CoreDNS is healthy

A healthy CoreDNS deployment does not guarantee every Pod can reach it.

When egress NetworkPolicy isolates a workload, DNS traffic must still be permitted to the cluster DNS path if the application needs name resolution. DNS can use both UDP and TCP, so restrictive policy should account for both protocols where port 53 access is intentionally controlled.

DNS policy should permit both UDP and TCP on port 53 when restrictive egress policy requires name resolution.

This creates a useful diagnostic split.

If a Pod receives an immediate NXDOMAIN, the DNS server responded but did not find the requested name. Investigate spelling, namespace, Service existence, and search domains.

If the query times out, the resolver may not be reachable. Investigate policy and network reachability to cluster DNS.

External names add another layer. CoreDNS normally forwards non-cluster queries according to its configured upstream resolution path. That means this pattern is especially informative:

Text
kubernetes.default resolves api.production resolves example.com fails

When cluster names work but external names fail, the internal Service discovery path is functioning. Focus on CoreDNS forwarding, node/upstream resolver configuration, and any egress controls between cluster DNS and the upstream resolver.

The inverse pattern is also possible: external names may resolve through a custom Pod policy while cluster Service names fail. That points back toward the Pod's dnsPolicy, search domains, or cluster-DNS configuration.

For workload-level traffic isolation beyond DNS, use Kubernetes Network Policies for Small Teams.

The troubleshooting framework isolates the failure layer

Use this sequence during an incident before changing configuration.

Diagnostic layerQuestionEvidence to collectMove on when
NameIs the requested name correct for the namespace?Short, qualified, and full namesCorrect name still fails
Pod resolverIs this Pod using expected cluster DNS?/etc/resolv.conf, dnsPolicy, dnsConfigResolver config is expected
ServiceDoes the Service exist and expose the expected port?kubectl get svcService identity is correct
EndpointsDoes the Service have ready backends?EndpointSlices, Pod readinessBackends are present
Cluster DNSAre DNS Pods, Service, logs, and endpoints healthy?CoreDNS/kube-dns resourcesDNS infrastructure is healthy
Policy/networkCan the Pod reach DNS and then the target?NetworkPolicy and connectivity evidenceRequired paths are allowed
Upstream DNSDo only external names fail?Cluster-name vs external-name comparisonForwarding path is verified

The order is deliberate.

Do not restart CoreDNS because one Service name was misspelled. Do not relax NetworkPolicy because a Service selector is empty. Do not change Service exposure because one Pod uses the wrong DNS policy.

A good incident note should record the first layer that failed and the first layer that passed. That makes repeated DNS incidents easier to compare and prevents the team from rediscovering the same boundary every time.

NodeLocal DNSCache can be useful in some clusters to reduce DNS latency and connection-tracking pressure by running a local caching agent on nodes. It is an optimization architecture, not the first fix for an unresolved namespace, broken Service selector, or unhealthy CoreDNS deployment.

Raff keeps infrastructure networking separate from Kubernetes service discovery

Raff Managed Kubernetes provides the cluster infrastructure boundary, including private VPC networking and the managed control-plane model. Kubernetes Service discovery remains a workload-layer concern: teams still define Service names, namespaces, selectors, DNS policies, application dependencies, and NetworkPolicies.

That separation is useful during incidents.

Text
Raff VPC / cluster infrastructure ↓ Kubernetes cluster DNS ↓ Service identity ↓ EndpointSlices ↓ Application Pods

A private cluster network can be healthy while an application uses the wrong Service name. A Service can resolve while its EndpointSlice is empty. CoreDNS can be healthy while a restrictive egress policy blocks one namespace from reaching it.

A platform uptime commitment does not turn every failed Service lookup into an infrastructure incident; the diagnosis still needs to identify which layer owns the failure.

For Raff teams, the practical review is therefore:

  • verify the namespace and expected Service identity;
  • verify the Service and its ready endpoints;
  • verify the Pod resolver configuration;
  • verify CoreDNS health only when evidence points there;
  • verify NetworkPolicy and VPC-level reachability as separate layers;
  • verify external upstream DNS only when cluster discovery already works.

This keeps remediation narrow. A workload configuration problem should be fixed as a workload configuration problem instead of widening infrastructure access or changing cluster-wide DNS unnecessarily.

Kubernetes DNS incidents are easier when names and ownership are explicit

Kubernetes DNS and Service discovery work best when applications depend on stable Service names rather than Pod addresses and when teams understand the namespace boundary behind those names.

When a lookup fails, identify the failure layer before changing infrastructure. Test the correct name, inspect the Pod resolver, verify the Service and EndpointSlices, confirm CoreDNS health, then inspect NetworkPolicy and upstream DNS only where the evidence points. If the name resolves but the application does not connect, continue the investigation after DNS rather than treating resolution as the complete network path.

For the broader traffic model, continue with Kubernetes Networking and Storage Architecture. For workload isolation, use Kubernetes Network Policies for Small Teams.

Deploy Raff Kubernetes with private networking, then keep Service names, resolver behavior, and workload traffic paths explicit as the cluster grows.

Free control plane · 14-day money-back guarantee

Deploy a Cluster Now
Was this article helpful?
Serdar Tekin
Serdar Tekin
Co-Founder & CTO

Co-Founder & CTO at Raff, focused on cloud infrastructure, systems architecture, security, reliability, and production operations.

Free control plane

Run your services on managed Kubernetes, worker nodes from $9.99 a month.

CNCF-certified Kubernetes with a free control plane. Add nodes when you need them, with no egress fees.

14-day money-back guarantee

Frequently asked questions

How does Kubernetes DNS work?
Kubernetes DNS resolves stable Service names for Pods so applications do not need to track changing Pod IP addresses. Normal Services resolve to a cluster IP, while headless Services can resolve to backing Pod addresses.
What DNS name should I use for a Kubernetes Service?
Use the short Service name inside the same namespace, `service.namespace` across namespaces, or the cluster-qualified name when needed. The common default domain is `cluster.local`, but clusters can use another domain.
Why can a Pod resolve some Kubernetes Services but not others?
Check the target namespace, Service name, resolver search path, and whether the Service actually exists. If other names resolve, cluster DNS may be healthy and the failure may be specific to that Service identity.
Why does a Kubernetes Service name resolve but the connection still fail?
DNS only proves name resolution. Check the Service port, selector, EndpointSlices, Pod readiness, NetworkPolicy, application listener, and the network path after resolution.
How do I check whether CoreDNS is working in Kubernetes?
Test a known cluster name such as `kubernetes.default`, then inspect the kube-system DNS Pods, logs, DNS Service, and EndpointSlices. Compare results across more than one Pod before assuming a cluster-wide failure.
Can NetworkPolicy block Kubernetes DNS?
Yes. An egress-isolated Pod can lose access to cluster DNS unless policy permits the required DNS path. Restrictive policy should account for both UDP and TCP port 53 where DNS access is required.
Why do external DNS lookups fail while Kubernetes Service names still work?
Cluster DNS may be healthy while CoreDNS forwarding, the node or upstream resolver, or egress policy to external DNS is failing. Compare internal and external lookups before changing Service configuration.