Kubernetes disaster recovery is the process of restoring a usable service after cluster, control-plane, storage, or application-state failure. A complete plan protects Kubernetes configuration and API state, persistent application data, and the external systems and credentials the workload needs to operate, while also defining the recovery infrastructure those layers will be restored into.
Kubernetes backup is one part of that plan. Backups create recovery points; disaster recovery defines which failure you are recovering from, how much data loss is acceptable (RPO), how quickly service must return (RTO), where the restore runs, and how recovery is validated. A completed backup job is not proof that the application can recover.
This guide owns backup and disaster recovery inside the Kubernetes Cluster Management for Small Teams operating model. It focuses on recovery architecture, backup and restore boundaries, RPO/RTO, and recovery validation rather than a step-by-step tool installation.
At Raff, the recovery rule is to name the failure before choosing the backup. A deleted namespace, corrupted database, lost cluster, and unavailable dependency require different recovery points and different restore paths.
Kubernetes disaster recovery architecture has three protected workload layers
A Kubernetes application usually has more state than the cluster API alone reveals.
A practical backup strategy separates three layers:
| Protection layer | Examples | Recovery question |
|---|---|---|
| Cluster configuration and API state | Deployments, Services, Ingress, RBAC, NetworkPolicy, CRDs, Kubernetes Secrets | Can the workload definition and required cluster objects be recreated? |
| Persistent application data | Databases, queues, PVC data, uploaded files | Can the business state be restored to an acceptable point? |
| External systems and credentials | DNS, object storage, external secret stores, certificates, third-party APIs | Can the restored workload reconnect and operate correctly? |
GitOps or infrastructure-as-code can make cluster configuration reproducible, but a Git repository does not contain the live contents of a database volume. An etcd snapshot can preserve Kubernetes objects, but it does not automatically protect data stored behind every PersistentVolume. A storage snapshot can preserve a disk state, but it may not capture an application-consistent database recovery point.
Raff's workload recovery framework separates 3 protected layers: cluster configuration and API state, persistent application data, and external systems and credentials.
The recovery cluster or underlying infrastructure is the destination for those protected layers rather than another copy of workload data. In a complete cluster-loss scenario, that destination still has to exist or be recreated first — including the cluster runtime, networking, access path, and other infrastructure dependencies.
That separation is the first decision because different failures damage different layers. A bad Deployment manifest may need configuration rollback. A deleted database row may need point-in-time recovery. A lost cluster may require both workload recreation and data restoration.
Backup, snapshot, replication, and disaster recovery are different controls
These terms are often used as if they solve the same problem, but they protect against different failure modes.
| Control | What it does | What it does not prove |
|---|---|---|
| Backup | Creates an independent recovery copy or recovery point | That the application can be restored and validated |
| Snapshot | Captures storage state at a point in time | Application consistency, portability, or independent retention |
| Replication | Maintains additional live copies for availability | Protection from deletion, corruption, or bad writes that replicate |
| Disaster recovery | Defines how a usable service is restored after a defined failure | Nothing by itself; it depends on tested backups, infrastructure, data, traffic, and runbooks |
This distinction matters because Kubernetes can make a workload highly available while still leaving it unrecoverable from logical corruption or complete environment loss. High availability reduces interruption from some failures; disaster recovery prepares for the failures that availability mechanisms do not absorb.
RPO and RTO determine the backup design
Recovery Point Objective (RPO) is the maximum amount of data loss the business can accept. Recovery Time Objective (RTO) is the maximum acceptable time to restore service.
Those two numbers should be defined before backup frequency or tooling.
Consider two illustrative workloads:
| Workload requirement | Possible recovery design |
|---|---|
| RPO 24 hours, RTO 8 hours | Nightly data backup, configuration in Git, documented manual restore |
| RPO 5 minutes, RTO 30 minutes | Database-native continuous recovery, frequent protected state, pre-tested recovery environment and automation |
These are examples, not universal targets. The correct RPO and RTO depend on customer impact, transaction value, contractual commitments, data-change rate, recovery complexity, and cost.
A common mistake is setting one RPO for the whole cluster. Different components can justify different recovery objectives. Static application manifests may tolerate recreation from source control. A payments database may require much tighter data-loss tolerance. Generated caches may need no backup at all.
Define RPO and RTO per critical data class, then map each class to a recovery mechanism.
RPO and RTO should be driven by the business impact of losing data or service, not by whether the application runs on Kubernetes or traditional VMs. Kubernetes can add more recovery dependencies — cluster provisioning, controllers, storage classes, secrets, ingress, and workload ordering — so meeting the same RTO may require more automation and rehearsal. The target itself should still come from the workload requirement.
A backup schedule without an RPO is a calendar setting, not a recovery requirement.
For broader workload-level RPO, retention, and restore planning, use Cloud Server Backup Strategies: RPO, RTO, and Recovery Planning. This guide keeps the Kubernetes-specific ownership and restore path as the main focus.
Control-plane backup and workload backup solve different failures
Kubernetes stores API objects and cluster state in etcd. The official Kubernetes etcd operations guide recommends maintaining a backup plan because etcd snapshots can be used to recover cluster state after disasters such as loss of all control-plane nodes.
That makes etcd protection important for self-managed clusters, but it should not be confused with complete application backup.
An etcd snapshot protects objects stored in the Kubernetes backing store. Persistent application data normally lives in storage systems outside etcd. If a database stores 500 GB on a PersistentVolume, the etcd snapshot contains the Kubernetes objects that describe the volume relationship, not the 500 GB database contents themselves.
For managed Kubernetes, the responsibility boundary changes again. The provider may operate control-plane state and recovery mechanisms while the workload team remains responsible for application data and restore readiness.
On Raff, the managed Kubernetes control plane, etcd, and platform networking are operated as part of the Kubernetes service. That reduces control-plane administration, but it does not turn cluster availability into application backup.
Raff's managed Kubernetes carries a 99.9% uptime SLA, but an uptime SLA does not define an application's RPO or RTO.
The workload team still needs a separate plan for database state, PVC contents, files, secrets, and external dependencies.
Persistent data needs application-aware recovery
PersistentVolumes keep data beyond an individual Pod lifecycle, but persistence is not the same as backup.
Replicated storage can keep data available through some infrastructure failures. It can also faithfully replicate an accidental deletion, bad write, ransomware event, or application corruption. That is why replication and backup answer different questions.
Kubernetes provides the VolumeSnapshot API as a standardized interface for point-in-time storage snapshots. The official Volume Snapshots documentation notes that snapshot support depends on the Container Storage Interface (CSI) driver and Kubernetes distribution. A snapshot can then be used as a source for a new PersistentVolumeClaim when the storage implementation supports it.
Kubernetes v1.36 moved Volume Group Snapshots to general availability, allowing supported CSI implementations to create a crash-consistent recovery point across a set of volumes. GA status in Kubernetes does not mean every managed cluster or CSI driver exposes VolumeGroupSnapshot support; verify the storage implementation before relying on it. Storage-level crash consistency is also not automatically the same as application-level transactional consistency.
Databases deserve their own recovery mechanism. Depending on the engine and RPO, that may mean:
- logical dumps;
- physical database backups;
- transaction-log or write-ahead-log archiving;
- point-in-time recovery;
- application-coordinated snapshots;
- replica-based backup workflows.
The Kubernetes Persistent Storage guide covers volumes and storage classes in more detail. The recovery rule here is simpler: back up data according to the system that owns its consistency model.
Do not assume that copying a live database filesystem produces a usable database backup. Do not assume that a PVC snapshot is portable to another storage implementation. Verify both consistency and restore destination before relying on the recovery point.
The decision framework maps failures to recovery mechanisms
Choose backup mechanisms by failure scenario, not by product category.
| Failure scenario | What must be recoverable | Strong starting mechanism |
|---|---|---|
| Bad manifest or configuration change | Kubernetes object definitions | Git or infrastructure-as-code rollback |
| Accidental namespace deletion | Namespaced resources plus affected state | Workload-aware cluster backup plus data recovery |
| Database row/table corruption | Database state at an earlier time | Database-native backup or PITR |
| PVC deletion or filesystem corruption | Persistent volume contents | Storage snapshot or independent data backup |
| Control-plane loss in self-managed Kubernetes | Kubernetes API state | Tested etcd snapshot recovery |
| Complete cluster loss | Configuration, data, secrets, dependencies | Recreate cluster plus restore all protected layers |
| Credential or secret loss | Secret material and external identity | Secure secret recovery process or external secret store |
| Backup tool failure | Recovery copies themselves | Independent copy, monitoring, and restore validation |
The strongest design often uses more than one mechanism.
For example, a production application might use:
Git repository → Deployments, Services, policies, Helm values Database-native backup / PITR → transactionally consistent business data Workload-aware Kubernetes backup → selected cluster objects and supported persistent data Object storage → independent backup artifacts outside the workload Restore test → proves the layers work together
Tools such as Velero can back up and restore Kubernetes resources and persistent volumes using supported integrations. That makes workload-aware tools useful, but tool selection should follow the recovery requirements rather than define them.
Evaluate a Kubernetes backup tool by whether it supports the resources, CSI/storage system, namespaces, CRDs, encryption requirements, retention, object-storage destination, selective restore, cross-cluster recovery, and auditability your workload actually needs.
Kubernetes backup and restore needs a full service recovery path
A disaster recovery plan should describe how the service becomes usable again, not only how data is copied back.
Recent CNCF disaster recovery guidance makes the same boundary explicit: backup tooling restores resources and data into a recovery environment; it does not by itself recreate the cluster, nodes, network, load balancers, or DNS.
For a complete cluster-loss scenario, the recovery sequence may include:
- Provision or recover a Kubernetes cluster.
- Re-establish networking, namespaces, access controls, and policy.
- Restore required CRDs and controllers in dependency order.
- Restore Kubernetes Secrets or reconnect the external secret source.
- Restore databases, PVC data, and durable files.
- Recreate Services, ingress or Gateway routes, and DNS dependencies.
- Start application workloads in a controlled order.
- Validate health, data integrity, queues, scheduled jobs, and external integrations.
- Re-enable user traffic only after acceptance checks pass.
That order is workload-specific. A database operator may need to exist before its custom resources can be restored. An ingress controller may need to be available before public routes become useful. A queue consumer may need to stay disabled until database recovery is complete.
A runbook should therefore include dependency order, credentials, owners, stop conditions, validation commands, expected recovery time, and escalation contacts.
If disaster recovery requires geographic or account-level isolation, design that explicitly. Storage replication inside one cluster should not be assumed to satisfy a requirement for an independent failure domain.
Restore testing is the real backup verification
A successful backup job proves that data was written somewhere. It does not prove that the application can return to service.
Restore tests should exercise the failure scenarios that matter to the business.
At minimum, test whether the team can recover:
- Kubernetes resource definitions;
- one namespace or workload selectively;
- a persistent volume or database;
- secrets and external credentials;
- DNS and ingress dependencies;
- application startup order;
- background jobs and queues;
- monitoring and alerting;
- the documented acceptance checks.
A useful restore test is isolated from production. Restore into a temporary namespace, test cluster, or other controlled environment where the team can verify the result without overwriting the live system.
Measure the restore. Record the time from incident declaration to usable service and compare it with the RTO. Record the newest recoverable transaction or object and compare it with the RPO.
A quarterly restore test may be sufficient for a lower-risk system, while higher-impact workloads can justify more frequent validation. The cadence should follow change rate and risk rather than a generic checklist.
Backups should also be monitored for age and failure. A backup job that has silently failed for three weeks can make the documented RPO meaningless.
At Raff, the recovery review ends with one question: can the team restore a usable service, within the stated RPO and RTO, without improvising the critical steps?
Raff separates Kubernetes availability from backup responsibility
Raff Kubernetes provides a managed control plane, worker pools, private networking, monitoring, and replicated Kubernetes storage nodes. Those capabilities reduce infrastructure operations and improve the availability foundation for workloads.
They should not be described as automatic application backup.
Raff's public Data Protection service currently covers server backups, VM recovery points, and block-volume snapshots. That is useful for VM-based workloads, but it is a different protection layer from Kubernetes-native application backup.
For Kubernetes workloads, teams should design protection around the workload itself: reproducible manifests, database-aware backups, supported volume snapshots where available, workload-aware backup tooling, and independent backup destinations.
Raff Object Storage is S3-compatible and can store backup artifacts from tools that support S3-compatible endpoints. Raff Object Storage does not currently provide native Object Lock, object versioning, or lifecycle rules, so backup software should manage repository history and retention where required. Workloads that require provider-enforced immutable retention should verify a storage target that exposes that control. Keep backup credentials separate from routine application credentials where possible.
For the broader object-storage decision, use S3-Compatible Object Storage: Use Cases for Developers.
For maintenance planning, pair this recovery strategy with Kubernetes Upgrade Strategy. An upgrade window should begin only after the team knows which state is protected and how recovery would work if maintenance exposes a failure.
