The production baseline is a three-member replica set
MongoDB's standard production replica-set architecture is three members.
A typical topology is:
application
↓
MongoDB connection string
↓
┌──────────┬──────────┬──────────┐
│ primary │ secondary│ secondary│
│ data │ data │ data │
└──────────┴──────────┴──────────┘
The primary accepts writes. Secondaries replicate the primary's oplog and can participate in elections.
If the primary becomes unavailable, eligible members can elect a new primary.
A three-member replica set can tolerate one voting member becoming unavailable while retaining the majority required to elect a primary.
This is fundamentally different from backups: replication keeps live copies for availability, while backups provide historical recovery.
Prefer three data-bearing members over an arbiter when possible
MongoDB supports arbiters, but an arbiter:
- votes in elections;
- does not store application data;
- cannot become primary;
- does not add another recoverable copy of the dataset.
A primary-secondary-arbiter layout can reduce infrastructure cost, but it provides less data redundancy than three data-bearing members.
For production workloads where the data matters, prefer:
primary + secondary + secondary
over:
primary + secondary + arbiter
when the additional data-bearing node is operationally and financially reasonable.
MongoDB also warns against deploying multiple arbiters. An arbiter should be treated as a specific compromise, not a normal shortcut for high availability.
Keep an odd number of voting members
Replica-set elections require a majority of voting members.
An odd number of voting members generally avoids adding a vote that does not increase fault tolerance.
For example:
| Voting members | Majority required | Member failures tolerated for election |
|---|
| 3 | 2 | 1 |
| 4 | 3 | 1 |
| 5 | 3 | 2 |
Adding a fourth voting member to a three-member set does not increase the number of failures you can tolerate for election.
Do not add members without understanding whether they improve availability, read capacity, reporting isolation, backup operations, or another explicit requirement.
Replica sets protect availability, not every failure mode
A replica set can protect against:
- a MongoDB process failure;
- a VM failure;
- selected host failures;
- some maintenance events.
It does not automatically protect against:
- accidental deletion replicated to every member;
- application bugs that corrupt data;
- compromised credentials;
- a full failure domain that contains every replica;
- logically invalid writes;
- ransomware or destructive administrative action;
- loss of backups.
That is why replication and backup must remain separate controls.
For the broader principle, use PostgreSQL Replication vs Backups vs Snapshots as a database-agnostic comparison of availability and recovery layers, and use LC08 for backup/recovery ownership.
Spread members across independent failure domains
Three replicas on one physical host are not meaningful infrastructure redundancy.
Where the infrastructure model allows it, distribute members across independent hosts or failure domains.
A good objective is:
member 1 → failure domain A
member 2 → failure domain B
member 3 → failure domain C
If all members are inside one data center, the replica set can still protect against individual VM or host failures but not full-site loss.
MongoDB's production guidance for multi-data-center deployments recommends distributing production replica-set members across multiple data centers when site-level availability is required.
Your architecture should explicitly state which failures it is designed to survive.
Use private networking for replica-set traffic
MongoDB replica-set traffic should normally use private addresses rather than public internet paths.
On Raff, that means using VPC/private networking between MongoDB VMs where possible.
A sensible topology is:
public internet
↓
application edge
↓
application VM
↓ private VPC
MongoDB replica set
Avoid publicly exposing MongoDB port 27017 unless there is a specific, secured requirement.
Private networking does not replace authentication or TLS where encryption in transit is required. It reduces exposure; it does not remove the need for database security.
Use replica-set-aware connection strings
Applications should connect to the replica set, not hard-code only the current primary.
A replica-set-aware connection string allows the MongoDB driver to:
- discover members;
- identify the current primary;
- react to elections;
- reconnect after failover.
Conceptually:
mongodb://db1,db2,db3/app?replicaSet=rs0
The exact authentication, TLS, and hostname configuration depends on your deployment.
If the application only knows one fixed primary IP, automatic failover is incomplete even if MongoDB itself can elect a new primary.
Elections create a temporary write interruption
Replica sets provide automatic primary election, but failover is not instantaneous.
When the primary disappears:
- remaining members detect loss of primary;
- eligible voting members hold an election;
- one secondary becomes primary;
- drivers discover the new primary;
- writes resume.
During this window, writes can fail or block depending on client behavior and timeout configuration.
Applications should:
- use supported MongoDB drivers;
- set sensible server-selection timeouts;
- retry only operations that are safe to retry;
- surface sustained failover problems to monitoring.
High availability reduces downtime. It does not guarantee zero interrupted requests.
Write concern defines durability acknowledgement
Write concern determines how many data-bearing replica-set members must acknowledge a write before the client considers it successful.
For many replica-set configurations, w: "majority" is the default.
Conceptually:
write
→ primary
→ replicated to majority
→ acknowledgement returned
A weaker acknowledgement such as w: 1 can return sooner because only the primary must acknowledge the operation.
That can reduce latency but changes the durability trade-off.
Do not choose write concern globally from a benchmark alone. Match it to:
- data-loss tolerance;
- latency requirements;
- failure model;
- transaction semantics.
For important business writes, durability often matters more than shaving a small amount of acknowledgement latency.
Read preference is not a free scaling switch
MongoDB can route selected reads to secondaries depending on read preference.
That can help:
- reporting workloads;
- some analytics;
- selected read-heavy use cases;
- workload isolation.
But secondary reads can be stale relative to the primary.
Do not send all reads to secondaries simply because read traffic is high.
Ask:
- Can this request tolerate stale data?
- Does the query require read-your-writes behavior?
- Is the real bottleneck CPU, memory, indexes, storage, or query design?
- Would a larger primary or better index remove the need?
Use secondary reads when their consistency model matches the application.
Monitor replication lag
A secondary that is far behind can weaken failover confidence and read freshness.
Monitor:
- replication lag;
- member health;
- election frequency;
- oplog window;
- disk latency;
- CPU;
- memory;
- connection count;
- query latency.
A replica set can technically have all members online while one secondary is too far behind to be a good failover candidate.
Operationally, "up" and "healthy" are not the same thing.
Size the oplog for your recovery window
MongoDB uses the oplog to replicate changes.
The oplog must be large enough to cover the period a secondary may be disconnected or delayed before it can catch up without a full resynchronization.
A high-write workload consumes oplog history faster than a quiet workload.
Monitor the oplog window, not just its configured size.
If the oplog window is shorter than your realistic maintenance or outage duration, secondaries can fall too far behind.
Storage latency matters to replica health
MongoDB performance depends heavily on storage behavior.
Important storage signals include:
- write latency;
- fsync/journal latency;
- queue depth;
- available disk;
- disk growth;
- working-set fit in memory.
NVMe can help latency-sensitive workloads, but fast storage does not compensate for poor indexes or an oversized working set.
Before adding distributed complexity:
- inspect query plans;
- fix missing/inefficient indexes;
- measure memory pressure;
- measure storage latency;
- resize the VM/storage if appropriate.
Only then decide whether architectural scaling is required.
Vertical scaling should usually come before sharding
MongoDB supports both vertical and horizontal scaling.
Vertical scaling means increasing:
- CPU;
- RAM;
- storage performance/capacity.
Horizontal scaling through sharding splits data and workload across shards.
For small teams, vertical scaling plus replica sets is usually operationally simpler.
Consider sharding when measured evidence shows that one replica set cannot meet requirements because of:
- dataset size;
- write throughput;
- working-set limits;
- CPU saturation that cannot be addressed efficiently;
- storage throughput constraints;
- geographic/data-distribution requirements.
Do not shard because the application has "a lot of data" in the abstract.
Sharding changes the architecture substantially
A MongoDB sharded cluster includes:
application
↓
mongos query routers
↓
┌───────────────┬───────────────┐
│ shard A │ shard B │
│ replica set │ replica set │
└───────────────┴───────────────┘
↓
config server replica set
Each shard is itself a replica set.
Production sharding therefore adds:
- multiple replica sets;
- config servers;
- mongos routers;
- shard-key design;
- balancing;
- chunk/data distribution;
- more monitoring;
- more failure modes.
Sharding is not "replication with more servers." It is a separate distributed-data architecture.
The shard key is an architectural decision
A poor shard key can create:
- write hotspots;
- uneven data distribution;
- scatter-gather queries;
- unbalanced storage;
- difficult future migration.
A useful shard key should reflect:
- query patterns;
- cardinality;
- write distribution;
- locality requirements;
- growth pattern.
Do not select a shard key from a generic best-practices list without production-like query analysis.
Changing a shard key later is possible in modern MongoDB, but it remains a significant architectural operation.
Backups remain mandatory with replica sets and sharding
Replication copies current database state.
If an operator deletes data, the deletion can replicate.
If an application writes corrupt state, that state can replicate.
Therefore, production MongoDB still needs:
- backup schedule;
- retention policy;
- off-cluster backup location;
- restore testing;
- defined RPO;
- defined RTO.
For sharded clusters, backups must be consistent across the cluster architecture.
Do not assume a VM snapshot of one member is automatically a valid database backup.
A dedicated MongoDB backup guide remains a separate LC07/LC08 roadmap item.
Authentication and least privilege belong in the baseline
Production MongoDB should not run as an unauthenticated public database.
Use:
- MongoDB authentication;
- role-based access;
- separate application/admin identities;
- private networking;
- restricted firewall/security-group rules;
- TLS where required;
- secret rotation.
Application credentials should have only the database privileges they need.
Administrative credentials should not be embedded in the application.
Maintenance must preserve replica-set availability
Replica sets allow rolling maintenance when changes are compatible.
A safe operating model generally updates one member at a time:
- confirm replica health;
- update a secondary;
- wait for it to return healthy and caught up;
- repeat for other secondaries;
- step down or switch primary deliberately when needed;
- update the former primary;
- re-verify replication.
MongoDB recommends keeping replica-set members on the same major version outside rolling-upgrade windows.
Do not update every member simultaneously.
Production architecture examples
Small production workload
App VM(s)
↓ VPC
3-member MongoDB replica set
├─ primary
├─ secondary
└─ secondary
Good when:
- one replica set has enough capacity;
- automated failover is required;
- the team wants a straightforward operating model.
Read-heavy workload
App
↓
primary → writes / consistency-sensitive reads
secondaries → selected stale-tolerant reads
Use only when the application can tolerate the consistency trade-off.
Large horizontally scaled workload
App
↓
mongos
↓
multiple replica-set shards
+ config server replica set
Use only after measured requirements justify the operational cost.
Managed vs self-hosted MongoDB
Self-hosting gives the team control over:
- topology;
- VM size;
- storage;
- versions;
- networking;
- backup tooling;
- failover policy.
It also means the team owns:
- patching;
- replica health;
- elections;
- backup;
- restore testing;
- scaling;
- incidents.
A managed MongoDB service transfers more of those operational responsibilities to the provider.
Raff's public Managed Databases page currently shows MongoDB 7.0/8.0 as Rolling out, with replica sets and continuous backup listed as planned managed capabilities. Verify current availability before treating that managed option as production-ready.
For the hosting-model decision, use MongoDB Hosting: Self-Hosted vs Managed for Production.
MongoDB production checklist
Before production:
- use a replica set rather than a standalone node for HA-sensitive workloads;
- prefer three data-bearing members;
- distribute members across independent failure domains where possible;
- use private networking;
- use replica-set-aware connection strings;
- define write concern deliberately;
- monitor replication lag and oplog window;
- keep disk headroom;
- monitor storage latency;
- require authentication;
- restrict network exposure;
- back up independently of replication;
- test restores;
- document election/failover behavior;
- shard only after measuring the actual bottleneck.
Frequently asked questions
How many MongoDB replica-set members should production use?
MongoDB's standard production recommendation is a three-member replica set. Three data-bearing members provide redundancy and fault tolerance without unnecessary complexity.
Should I use an arbiter?
Use an arbiter only when constraints prevent another data-bearing member and you understand the reduced data redundancy. Three data-bearing members are generally preferable for production.
Does a MongoDB replica set replace backups?
No. Replication improves availability, but logical mistakes and destructive writes can replicate to every member. Keep independent backups and test restores.
When should I shard MongoDB?
Shard when measured dataset size, throughput, working-set, or distribution requirements exceed what a properly tuned and vertically scaled replica set can handle.
Can reads go to MongoDB secondaries?
Yes, depending on read preference, but secondary reads can be stale. Use them only when the application's consistency requirements permit it.
How should MongoDB nodes communicate in the cloud?
Use private networking where possible, restrict port exposure, require authentication, and use TLS when the security or compliance model requires encryption in transit.
Does Raff offer managed MongoDB?
Raff's current Managed Databases page lists MongoDB as Rolling out. Self-hosted MongoDB can run on Raff VMs today; verify managed MongoDB availability before planning a production migration to that service.
Sources
