ClickHouse replication and backups protect different failure classes. Replication keeps additional current copies of data available across selected node failures. Backups provide an independent recovery path after destructive changes, cluster loss, or historical data problems.
For production analytics, the essential rule is replication improves availability; backups preserve recoverability. A replicated cluster can reproduce accidental deletes or bad mutations across every replica, while a backup can still be too slow to satisfy an availability objective.
This guide owns ClickHouse backup, replication, and restore strategy. For deployment ownership, use Managed ClickHouse vs Self-Hosted ClickHouse. For monitoring, use ClickHouse Monitoring: Queries, Merges, Replication & Alerts.
Replication and backups solve different failures
| Failure | Replication helps? | Backup helps? |
|---|---|---|
| One replica/node fails | Yes | Usually not the fastest path |
| One storage device fails | Yes, with healthy other replicas | Yes |
| Accidental delete/mutation replicates | No | Yes, if backup predates it |
| Bad schema/derived-data change | Often copies the problem | Historical restore/rebuild can help |
| Entire cluster is lost | No surviving replica | Yes |
| Region/site failure | Only if topology spans it | Yes if backup is independent |
| Data needs historical recovery | No | Yes |
Do not call a replica a backup merely because it contains the same rows.
ReplicatedMergeTree protects current data availability
Self-managed ClickHouse commonly uses ReplicatedMergeTree-family tables for replication. Replicas coordinate state using ClickHouse Keeper.
A simplified topology is:
ClickHouse replica A ↕ ClickHouse Keeper quorum ↕ ClickHouse replica B
Replication can allow another replica to continue serving data when one node becomes unavailable, depending on query routing and application behavior.
But the topology itself creates operational responsibilities:
- Keeper availability;
- replica synchronization;
- lag and queue monitoring;
- failed replica recovery;
- storage capacity on every replica;
- schema consistency;
- load-balancer or client routing.
Replication factor should follow failure tolerance, not a generic “three is always best” rule.
ClickHouse Keeper is part of the availability design
Keeper coordinates metadata and replicated-table operations in self-managed replicated clusters.
If the coordination layer becomes unavailable, replicated tables may be unable to perform some writes or metadata operations even while existing local data remains readable.
That means Keeper needs its own:
- quorum design;
- monitoring;
- disk and latency health;
- backup/recovery thinking where configuration state matters;
- upgrade and maintenance plan.
A two-replica data topology with a fragile coordination layer is not a complete HA design.
Backups need an independent failure domain
A backup stored only on the same host or storage system as the ClickHouse data does not protect against that system's loss.
Production backup design should identify:
- backup destination;
- credential isolation;
- schedule;
- retention;
- encryption/access controls;
- restore environment;
- expected restore duration;
- validation procedure.
ClickHouse supports backup and restore operations for tables, databases, and metadata using supported backup destinations and mechanisms.
The business question is how much history must remain recoverable and how quickly it must return to service.
RPO and RTO should drive the recovery design
Two objectives organize the decision:
- RPO (Recovery Point Objective): maximum acceptable data loss.
- RTO (Recovery Time Objective): maximum acceptable recovery duration.
Examples:
RPO 24 hours + RTO 8 hours → daily backup may be plausible if restore is fast enough RPO 15 minutes + RTO 30 minutes → backup-only design may be insufficient → stronger availability and ingestion replay may be required
Analytical systems often have another recovery source: the upstream event stream or object store from which ClickHouse can be rebuilt.
If source data is replayable, document that path and measure how long a rebuild actually takes. “We can replay Kafka” is not an RTO until the throughput and procedure are tested.
Backups and ingestion replay can complement each other
A ClickHouse recovery strategy can combine:
schema/configuration backup + ClickHouse data backup + upstream raw event archive + replay procedure
For append-only analytics, an upstream durable log or object-storage archive can provide a second way to recover data after the last backup.
But replay has limits:
- source retention may expire;
- transformations may have changed;
- duplicate handling must be safe;
- rebuilding materialized views can take time;
- external reference data may no longer match historical state.
Test replay with representative volume before treating it as the primary disaster-recovery plan.
Backup frequency should match change and rebuild cost
A small, easily replayable dataset may not need the same backup schedule as a large customer-facing analytics cluster.
Consider:
- daily ingest volume;
- upstream replay window;
- materialized-view complexity;
- schema-change frequency;
- customer-facing history;
- rebuild time;
- backup storage cost;
- operational impact of backup jobs.
The correct schedule is the one that meets RPO without creating unacceptable system pressure.
Restore testing is more important than backup success
A successful backup job proves that files were written, not that the system can be recovered.
A ClickHouse restore test should verify:
- backup metadata is readable;
- tables/databases restore into a clean target;
- expected row counts and date ranges exist;
- users/permissions/configuration are rebuilt as required;
- materialized views and derived tables are correct;
- replicas can be re-established;
- applications reconnect;
- critical analytical queries match expected results;
- restore duration fits the RTO.
Schedule restore tests after major version, topology, or backup-method changes.
Replication lag is a recovery warning
A replica that exists but is far behind is less useful during failover.
Monitor:
- replica queue size;
- delay/lag;
- read-only state;
- part fetch failures;
- Keeper connectivity;
- disk capacity;
- mutation backlog;
- consistency of table definitions.
An HA design should define when a replica is considered healthy enough to receive production traffic.
Do not route to a stale replica solely because the process is running.
Mutations and deletes can propagate to replicas
Replicated tables distribute current logical state. That is why replication is not historical protection.
Before high-risk operations such as large ALTER, mutation, or deletion:
- verify backup/recovery state;
- estimate mutation cost;
- monitor replica queues;
- consider testing against a copy;
- define rollback or restore procedure.
For retention-driven deletion, use ClickHouse Partitioning & TTL so lifecycle behavior is explicit rather than ad hoc.
Managed ClickHouse changes who operates replication and backups
Raff Managed ClickHouse currently provides managed backup workflows and optional high availability, reducing the platform work around database hosts, Keeper, monitoring, and recovery infrastructure.
The customer team still owns:
- RPO and RTO requirements;
- deciding what historical point is correct;
- validating data after restore;
- upstream replay logic;
- schema and analytical correctness;
- application reconnect behavior.
For self-hosted ClickHouse on Raff VMs, the team owns the full replication and backup platform.
At Raff, the recovery rule is: every availability mechanism needs an independent historical recovery path, and every backup policy needs a measured restore test.
ClickHouse recovery checklist
- RPO and RTO are documented.
- Replication factor matches node-failure tolerance.
- Keeper is treated as production infrastructure.
- Replica lag and queues are monitored.
- Backups leave the primary failure domain.
- Backup retention covers historical recovery needs.
- Upstream replay window is documented if used.
- Materialized views can be rebuilt or restored.
- Large mutations/deletes have recovery protection.
- Restore tests measure real elapsed time.
- Application reconnect/failover behavior is tested.
Conclusion
ClickHouse replication and backups are complementary, not interchangeable.
Use replication to keep current analytical data available through selected node failures, backups to preserve historical recovery, and upstream replay when it is genuinely durable and fast enough to meet recovery objectives. A production design is complete only when the team has tested both failover and restore.