Kubernetes Monitoring: Metrics, Logs, Events, and Traces

Key Takeaways

Use service metrics for user impact, Events for Kubernetes diagnosis, logs for component context, and traces for distributed request paths. Metrics Server is not a full monitoring stack.

Serdar Tekin
Serdar Tekin
Co-Founder & CTO
Updated 17 min read
Written for: Developers and platform teams designing Kubernetes monitoring, logging, alerts, and application observability.
KubernetesMonitoringObservabilityMetricsLogsEventsTracing
Last reviewed:
Guide
Kubernetes Monitoring: Metrics, Logs, Events, and Traces
rafftechnologies.com/learn
Free control plane

Watch cluster metrics and events from the Raff dashboard.

CNCF-certified Kubernetes with a free control plane. Add nodes when you need them, with no egress fees.

Kubernetes monitoring is the process of collecting and interpreting metrics, logs, Events, and traces so a team can detect failures, understand what changed, and decide what to do next.

For small teams, the goal is not maximum telemetry. It is to collect the smallest set of signals that answers production questions reliably.

The practical model is:

metrics → what changed

Events → what Kubernetes reported while reconciling

logs → what the application or component said

traces → where one request spent time

Raff Technologies provides a built-in infrastructure baseline for managed Kubernetes—cluster/node CPU and memory, workload views, live Pod logs, and an activity feed—while application-level service metrics, alert policy, durable log retention, and distributed tracing remain workload responsibilities.

This guide owns the monitoring and observability model. It does not prescribe one mandatory Prometheus, Grafana, Loki, or tracing stack.

Monitoring asks whether known conditions are healthy.

Examples:

  • Is error rate above the acceptable threshold?
  • Are replicas unavailable?
  • Is a node under memory pressure?
  • Are Pods restarting?
  • Has queue depth exceeded the operating range?

Observability is broader. Kubernetes describes observability through the collection and analysis of metrics, logs, and traces to understand the internal state, performance, and health of a cluster.

Kubernetes Events are useful alongside those three pillars, but they serve a different role: they are short-lived diagnostic records about Kubernetes activity rather than a durable observability store.

SignalBest questionExample
MetricsWhat changed, by how much, and for how long?CPU, memory, request rate, p95 latency, error rate
EventsWhat did Kubernetes report while reconciling state?FailedScheduling, image pull failure, eviction
LogsWhat did this process say happened?stack trace, timeout, auth failure
TracesWhere did this request spend time?API → worker → database latency

A production incident should start from the signal closest to user or workload impact, then move down the stack.

Metrics should be split into infrastructure, Kubernetes, and application layers

One metrics dashboard cannot answer every operational question.

A useful small-team model has three layers.

Infrastructure metrics

Examples:

  • node CPU;
  • node memory;
  • disk pressure;
  • node availability;
  • network saturation where relevant.

These answer:

Is the worker infrastructure healthy enough to run workloads?

Kubernetes workload metrics

Examples:

  • desired vs available replicas;
  • Pod restart counts;
  • Pending Pods;
  • failed Jobs;
  • rollout status;
  • HPA replica changes.

These answer:

Is Kubernetes maintaining the desired workload state?

Application metrics

Examples:

  • request rate;
  • error rate;
  • latency;
  • queue depth;
  • job completion rate;
  • checkout failures;
  • webhook failures.

These answer:

Is the application delivering the intended service?

The third layer is the one small teams most often miss.

A Deployment can show all replicas Ready while every request returns HTTP 500.

Infrastructure health is not the same thing as application health.

Metrics Server is not a complete monitoring system

Kubernetes resource metrics are intentionally narrow.

The Metrics API provides current CPU and memory usage for Nodes and Pods. It supports tools such as:

Text
kubectl top nodes kubectl top pods

and resource-metric-based autoscaling.

Kubernetes 1.37 promoted the Metrics API to stable as metrics.k8s.io/v1.

That graduation improves the API's stability guarantees. It does not turn the resource metrics pipeline into a historical monitoring platform.

Metrics Server still does not replace:

  • historical time-series storage;
  • service-level metrics;
  • dashboards over long periods;
  • alerting;
  • custom business metrics;
  • durable trend analysis.

A production metrics pipeline normally needs a scraper and time-series backend when historical visibility or alerting is required.

Prometheus is a common choice, but not a mandatory one.

Use the system that answers the operating questions your team actually has.

Kubernetes component metrics provide a deeper cluster view

Kubernetes components expose operational metrics that can be scraped by monitoring systems.

Common sources include:

  • kube-apiserver;
  • kube-scheduler;
  • kube-controller-manager;
  • kubelet;
  • kube-proxy.

The kubelet also exposes specialized metrics endpoints such as resource, cAdvisor, and probe metrics.

These signals are useful for questions such as:

  • Is API-server latency increasing?
  • Are scheduler operations failing?
  • Are admission operations becoming slow?
  • Are node-level workload metrics changing?

They are valuable when operating the cluster, but they still do not replace application-level indicators.

If users report an API failure, the correct recovery signal should ultimately be the API's user-facing health—not only a healthy scheduler metric.

Logs should survive the Pod when production diagnosis requires history

Kubernetes makes container stdout and stderr available through kubectl logs.

That is useful for immediate diagnosis.

It is not the same as durable cluster-level log storage.

Pods are replaceable. Containers restart. Nodes can disappear.

If production diagnosis requires historical logs, those logs need a lifecycle independent from the Pod or node that produced them.

A useful logging policy answers:

  • Which logs are required for incident response?
  • How long are they retained?
  • Can logs from terminated Pods still be found?
  • Are messages structured?
  • Is there a request or correlation ID?
  • Can the team filter by service, namespace, environment, or severity?
  • Are secrets, tokens, credentials, and sensitive payloads excluded?

Structured logging usually matters more than raw log volume.

Prefer predictable fields such as:

Text
timestamp service environment level request_id error_code message

over large volumes of unstructured debug text.

Kubernetes Events are diagnostic evidence, not historical monitoring

Events are especially useful when Kubernetes itself cannot reconcile an object as expected.

Examples:

  • FailedScheduling;
  • image pull errors;
  • volume mount failures;
  • node pressure;
  • eviction;
  • failed probe activity;
  • workload lifecycle changes.

Events are useful for questions such as:

Why is this Pod Pending?

Why could this image not start?

Why did this volume fail to attach?

Why was this Pod evicted?

They are less useful for:

What was API latency last Tuesday?

How many checkouts failed yesterday?

Which dependency caused one request to take 900 ms?

Kubernetes Events have limited retention and should be treated as best-effort diagnostic data.

Capture and correlate them when troubleshooting, but do not use Events as a long-term monitoring database.

Traces become valuable when requests cross several services

Distributed traces connect multiple operations into one request path.

Example:

Text
client ↓ 15 ms API ↓ 35 ms internal service ↓ 620 ms database ↓ response

Metrics may tell you p95 latency increased.

Logs may show timeouts.

A trace can show where one instrumented request actually spent its time.

Tracing becomes particularly valuable when:

  • requests cross multiple services;
  • latency can occur in several dependencies;
  • failures are intermittent;
  • the system is asynchronous or distributed;
  • metrics identify a problem but not the responsible hop.

Kubernetes 1.37 system components can export spans over OpenTelemetry Protocol (OTLP). Application tracing still requires the application and dependencies to emit and propagate trace context.

Do not make tracing the first observability project for a simple application that still lacks basic latency and error-rate metrics.

Start with the simplest signal that closes the current diagnostic gap.

Build a minimum production monitoring baseline

A small Kubernetes team does not need every possible signal on day one.

A useful baseline is:

AreaMinimum useful signal
User impactavailability, error rate, latency
Workload healthavailable replicas, restart loops, failed Jobs
Schedulingpersistent Pending Pods and reason
Worker healthCPU, memory, node Ready state, pressure
Deploymentsrollout success/failure
Autoscalingreplica changes, Pending Pods, worker scale events
Stateful workloadsstorage errors and recovery status
Application diagnosisstructured logs
Kubernetes diagnosisEvents
Distributed systemstraces where needed

The exact list should follow the application.

A background worker may care more about queue age than HTTP latency.

A SaaS API may care more about request success rate and p95/p99 latency.

Monitoring should represent the service the user actually receives.

Alerts should represent action

Do not convert every metric into an alert.

Prioritize alerts by operational consequence.

1. User-visible failures

Examples:

  • sustained availability loss;
  • error-rate threshold;
  • latency beyond the service objective;
  • failed critical business operation.

2. Workload failures that threaten service

Examples:

  • unavailable replicas;
  • repeated CrashLoopBackOff;
  • persistent Pending Pods;
  • failed critical Job.

3. Capacity conditions with a clear response

Examples:

  • sustained memory pressure;
  • worker exhaustion;
  • autoscaling maximum reached;
  • storage nearing a critical limit.

4. Platform conditions that will become incidents

Examples:

  • certificate issue;
  • repeated node failures;
  • storage attach failures;
  • control-plane or API degradation where visible.

Every alert should answer:

  • Who owns it?
  • Why does it matter?
  • What is the first diagnostic step?
  • What condition closes it?
  • How is repeated noise suppressed?

If no action follows from an alert, it probably belongs in a dashboard rather than a notification channel.

Use one troubleshooting path instead of opening every dashboard

Choose the first signal from the symptom.

SymptomStart withThen inspect
User errors increaseapplication metricslogs, traces
Latency increaseslatency/error metricstrace path, dependency metrics
Pod is PendingKubernetes Eventsscheduler reason, requests, capacity
Pod restartsrestart metricprevious logs, Events, memory/CPU
Node pressurenode metricsrequests, evictions, workload distribution
Deployment regressionapplication metric around deployrollout state, logs, traces
Cluster looks healthy but users failservice metricsapplication logs/traces

A simple incident flow is:

prove impact → identify scope → inspect Kubernetes state → inspect logs → use traces if distributed → confirm recovery with the original impact signal

The recovery signal should match the failure signal.

If the incident started because checkout errors increased, healthy node CPU does not close the incident.

Checkout errors returning to normal does.

Monitoring and autoscaling share some signals, but have different goals

Autoscaling may use metrics such as CPU or external workload demand.

Monitoring may observe the same metric.

But the intent differs.

Autoscaling asks:

Should capacity change?

Monitoring asks:

Is the system behaving correctly?

Do not assume an autoscaler is a monitoring system.

For HPA and worker scaling, use Kubernetes Autoscaling.

For request sizing that affects both scheduling and HPA utilization, use Kubernetes Requests vs Limits.

Logs and metrics also have cost boundaries

Observability can become a meaningful infrastructure cost.

Cost grows through:

  • long retention;
  • high-cardinality metrics;
  • verbose logs;
  • duplicate collection;
  • excessive scrape frequency;
  • traces with high sampling rates;
  • large dashboards querying long time ranges.

The correct answer is not "collect less" by default.

It is to match telemetry cost to operational value.

Examples:

  • keep detailed debug logs briefly;
  • retain production error logs longer;
  • avoid unbounded user IDs as metric labels;
  • sample traces where full capture is unnecessary;
  • use different retention for production and preview environments.

Use Kubernetes Cost Optimization for the broader infrastructure-cost model.

How Raff Kubernetes monitoring fits this model

Raff Kubernetes includes built-in dashboard visibility for:

  • live cluster CPU and memory;
  • live node CPU and memory;
  • Deployments and Pods;
  • live Pod logs with tailing;
  • cluster activity feed.

Those signals are available without installing an additional observability stack.

Raff also offers one-click Kubernetes applications such as Grafana and related observability tools through its application catalog.

The built-in baseline is useful for questions such as:

  • Is a worker under pressure?
  • Which workloads are running?
  • Is a Pod restarting?
  • What is the current container output?
  • What cluster activity happened around the failure?

It should not be confused with a complete application-observability system.

Application teams still own:

  • service-level metrics;
  • business/user-path indicators;
  • alert definitions;
  • durable log retention;
  • application tracing;
  • telemetry retention policy.

Use the Raff Kubernetes product page for the current platform capabilities.

Production monitoring checklist

Before calling a Kubernetes workload observable, verify:

Service

  • user-impact metrics exist;
  • latency/error/availability indicators match the workload;
  • alerts have owners and recovery conditions.

Kubernetes

  • replica health is visible;
  • restart loops are visible;
  • Pending Pods can be investigated;
  • deployment failures are visible;
  • node health and pressure are visible.

Logs

  • important application logs are structured;
  • sensitive values are excluded;
  • retention matches operational need;
  • terminated-Pod history is available when required.

Events

  • engineers know when to inspect Kubernetes Events;
  • Events are not treated as durable historical telemetry.

Traces

  • tracing is added where distributed request paths justify it;
  • context propagates across the services being investigated;
  • trace storage/sampling is bounded.

Operations

  • dashboards support diagnosis rather than vanity metrics;
  • alerts are actionable;
  • incident recovery is confirmed with the original impact signal.

Final recommendation

Start Kubernetes monitoring from operational questions, not tooling.

Track user-facing service health first. Add Kubernetes workload and worker signals to explain capacity and reconciliation problems. Keep structured logs long enough to investigate important failures. Use Events for short-lived Kubernetes diagnostic evidence. Add distributed tracing when request paths become complex enough that metrics and logs cannot explain where time is spent.

Metrics Server and the stable metrics.k8s.io/v1 API provide useful current CPU and memory data, but they are not a complete historical observability system.

On Raff Kubernetes, use the built-in cluster, node, workload, log, and activity visibility as the infrastructure baseline, then add application telemetry according to the incidents your service actually needs to detect and diagnose.

Continue with Kubernetes Cluster Management, Kubernetes Autoscaling, and Kubernetes Cost Optimization for the operational systems around monitoring.

Use Raff Kubernetes built-in monitoring as your infrastructure baseline, then add application telemetry around the service outcomes your team actually operates.

Free control plane · 14-day money-back guarantee

Deploy a Cluster Now
Was this article helpful?
Serdar Tekin
Serdar Tekin
Co-Founder & CTO

Co-Founder & CTO at Raff, focused on cloud infrastructure, systems architecture, security, reliability, and production operations.

Free control plane

Watch cluster metrics and events from the Raff dashboard.

CNCF-certified Kubernetes with a free control plane. Add nodes when you need them, with no egress fees.

14-day money-back guarantee

Frequently asked questions

What is Kubernetes monitoring?
Kubernetes monitoring is the collection and interpretation of infrastructure, workload, and application signals so teams can detect failures, understand behavior, and respond.
What is the difference between Kubernetes monitoring and observability?
Monitoring checks known health conditions. Observability uses metrics, logs, and traces to understand system behavior and investigate conditions that were not fully predicted in advance.
Is Metrics Server a complete Kubernetes monitoring system?
No. Metrics Server exposes current CPU and memory resource metrics for Nodes and Pods. Historical analysis, alerting, custom metrics, and dashboards require a broader telemetry system.
Are Kubernetes Events the same as logs?
No. Events are short-lived diagnostic records about Kubernetes activity. Logs are output from applications and system components and may need separate durable storage.
Does Kubernetes store container logs permanently?
No. Kubernetes does not provide a native durable cluster-wide log store. Important logs should be forwarded or retained outside the lifecycle of individual Pods and nodes.
When should I add distributed tracing?
Add tracing when requests cross multiple services or dependencies and metrics plus logs cannot explain where latency or failures occur.
What monitoring does Raff Kubernetes include?
Raff includes cluster and node CPU/memory views, workload views, live Pod logs, and a cluster activity feed. Application-level metrics, alerting, and durable telemetry remain workload responsibilities.