K3s on Cloud VMs: Architecture, HA, Storage, and Operations

Key Takeaways

A single K3s server VM is a complete Kubernetes cluster. Add agent VMs for workload capacity, but do not confuse extra agents with control-plane HA. Embedded-etcd HA requires 3 or more server nodes; external-datastore HA can use 2 or more servers. Keep cluster-state backup, application-data backup, storage availability, and control-plane HA as separate controls. K3s reduces integration work, but self-managed teams still own upgrades, networking, datastore health, backups, and recovery.

Serdar Tekin
Serdar Tekin
Co-Founder & CTO
Updated 18 min read
Written for: Developers and small infrastructure teams designing and operating self-managed K3s clusters on cloud VMs
K3sKubernetesCloud VMHigh AvailabilityStorageDevOps
Last reviewed:
Guide
K3s on Cloud VMs: Architecture, HA, Storage, and Operations
rafftechnologies.com/learn

Run K3s on your own cloud VMs, from $4.99 a month.

  1. 1Pick a Linux imageUbuntu, Debian and more, with your SSH key added.
  2. 2Choose a sizeFrom $4.99 a month. Resize when your app grows.
  3. 3Connect over SSHYour server is up in about 55 seconds.

Ubuntu, Debian and other Linux distributions with full root and SSH access, up and running in about 55 seconds.

K3s is a fully compliant Kubernetes distribution that packages the control plane, container runtime, networking, DNS, ingress, load-balancing support, network policy, and local storage tooling into a smaller operating surface. On cloud VMs, that makes K3s a practical self-managed cluster model for teams that need Kubernetes scheduling without assembling every cluster component from scratch.

For most small teams, the architecture decision is straightforward: one K3s server VM is the smallest complete cluster; add agent VMs when you need more workload capacity; move to an HA control plane only when control-plane downtime no longer fits the workload's recovery target.

A useful progression is:

NeedK3s topology to evaluate
Smallest working cluster1 server VM
More application capacity1 server + agent VMs
HA with embedded datastore3+ server nodes with embedded etcd
HA with external datastore2+ server nodes + supported external datastore

The important decision is not whether K3s is “lightweight Kubernetes.” It is which K3s architecture your workload requires and which cluster responsibilities your team is prepared to own.

A production HA design changes the datastore, control-plane topology, network requirements, storage plan, backup process, and maintenance model. This guide owns that architectural overview. Detailed datastore design belongs in K3s High Availability: Embedded etcd vs External Database, while deeper networking, storage, and upgrade procedures belong in their dedicated guides.

K3s architecture has server and agent roles

K3s uses two main node roles.

  • Server nodes run the Kubernetes API and control-plane services. They also participate in the cluster datastore according to the selected topology.
  • Agent nodes run application workloads and connect back to the servers.

A basic multi-node layout looks like:

Text
K3s server API + control plane + datastore ↓ ┌─────────┼─────────┐ ↓ ↓ ↓ Agent 1 Agent 2 Agent 3 workloads workloads workloads

The current K3s quick-start documentation confirms that a single server node is already a fully functional Kubernetes cluster: it can run the datastore, control plane, kubelet, and container runtime needed to host workload Pods.

That means K3s supports a useful progression:

Text
Single VM ↓ One K3s server running workloads ↓ Add agents for workload capacity ↓ Add HA servers when control-plane availability justifies it

This is different from treating every production K3s deployment as a three-server cluster from day one.

A small internal application may tolerate one server with a tested recovery process. A customer-facing platform with a short recovery target may justify an HA control plane before workload scale alone would require it.

Single-server K3s is the smallest cluster model

A single-server installation is the lowest-complexity K3s architecture.

Text
Cloud VM ├── K3s server ├── Kubernetes API ├── control-plane components ├── default datastore ├── containerd └── application workloads

The default datastore for a simple single-server K3s cluster is SQLite. K3s also supports embedded etcd and external datastores, but SQLite keeps the smallest topology simple.

A single-server design can fit:

  • development environments;
  • CI or test clusters;
  • internal tools;
  • low-criticality production workloads;
  • small systems where fast rebuild and restore are acceptable;
  • teams learning Kubernetes operations before adopting HA.

The main limitation is clear: the server VM is both a control-plane and infrastructure failure domain.

If that VM fails, Kubernetes cannot schedule new workloads or serve its API until the server is recovered or rebuilt. Workloads on that same VM also disappear with the host.

Adding agent nodes gives more workload capacity, but it does not make a single-server control plane highly available.

Text
Single server ↓ Agent 1 Agent 2 Agent 3

If the server becomes unavailable, existing workloads on agents may continue running for a period, but cluster control and normal reconciliation are impaired. That architecture should therefore be chosen deliberately rather than described as HA merely because several VMs exist.

HA K3s changes the datastore and quorum model

K3s supports two primary HA datastore patterns:

HA modelServer requirementDatastoreMain trade-off
Embedded etcd3 or more server nodesetcd on K3s serversSelf-contained cluster, quorum and disk I/O matter
External datastore2 or more server nodesExternal etcd/MySQL/MariaDB/PostgreSQLSeparates datastore, adds external DB dependency

For embedded etcd, current K3s documentation requires three or more server nodes and recommends an odd number so etcd can maintain quorum efficiently.

A typical embedded-etcd topology is:

Text
Fixed registration address ↓ ┌────────────┼────────────┐ ↓ ↓ ↓ Server 1 Server 2 Server 3 control control control + etcd + etcd + etcd └────────────┬────────────┘ ↓ Agent nodes

Quorum determines whether the datastore can continue accepting changes. With three etcd members, two must remain available for quorum.

Fast, reliable storage matters because etcd is sensitive to disk latency. K3s explicitly warns that embedded etcd can suffer on slower storage.

An external-database HA design looks different:

Text
Fixed registration address ↓ ┌───────┴───────┐ ↓ ↓ Server 1 Server 2 control control └───────┬───────┘ ↓ External datastore PostgreSQL/MySQL/etc. ↓ Agent nodes

The external datastore can simplify some control-plane topology decisions, but it creates another production system that must itself be highly available, backed up, monitored, secured, and reachable.

Use the dedicated K3s High Availability guide when choosing between those two models. This connector guide keeps the distinction architectural.

A fixed registration address simplifies HA node joins

In an HA cluster, agents and additional server nodes need a stable endpoint for registration.

K3s supports using a fixed registration address in front of the server nodes. That address can be provided by a TCP load balancer, virtual IP, or another stable endpoint appropriate to the environment.

Text
Agents ↓ Stable registration endpoint ├── K3s Server 1 ├── K3s Server 2 └── K3s Server 3

After registration, K3s agents establish their normal connections to server nodes.

The registration endpoint solves control-plane reachability. It should not be confused with the application ingress path used by customer traffic.

That separation is important:

  • control-plane endpoint → Kubernetes API and node registration;
  • application ingress → HTTP/HTTPS or other workload traffic;
  • ServiceLB or another load-balancer implementation → Kubernetes LoadBalancer Services.

Do not use one vague “load balancer” label for all three responsibilities in the architecture diagram.

K3s packages the common cluster components

Current K3s documentation lists a batteries-included set of components including:

  • containerd / compatible CRI runtime support;
  • Flannel CNI networking;
  • CoreDNS;
  • Traefik ingress controller;
  • ServiceLB;
  • kube-router network-policy controller;
  • local-path-provisioner;
  • metrics-server;
  • Helm controller;
  • host utilities required by the distribution.

K3s v1.37, released in September 2026, also bundles Kubernetes Gateway API CRDs as a packaged system chart. That makes Gateway API resources easier to adopt on current K3s releases, but a controller is still required to implement Gateway traffic behavior.

The exact packaged set can evolve with releases, so production documentation should follow the current K3s release rather than an old component list copied from a tutorial.

These defaults reduce initial integration work, but they are not mandatory architectural choices. K3s allows operators to disable packaged components and replace them where the workload requires a different implementation.

A useful rule is:

Keep the default component until a documented requirement justifies replacing it.

Replacing Traefik, Flannel, ServiceLB, or local-path storage introduces a new operational dependency. That can be the correct decision, but it should solve a real compatibility, networking, security, storage, or platform requirement.

Cloud-VM networking should be private by default

K3s requires specific node-to-node and node-to-server communication.

The current K3s requirements document states that:

  • TCP 6443 must be reachable from nodes to the K3s server/API;
  • Flannel VXLAN commonly uses UDP 8472 between nodes;
  • metrics-server requires node reachability on TCP 10250;
  • embedded-etcd server nodes require TCP 2379–2380 between one another.

K3s explicitly warns that the Flannel VXLAN port must not be exposed to the public internet.

For cloud VMs, a safer baseline is therefore:

Text
Internet ↓ Public application ingress only ↓ Private VM network ├── server-to-server control traffic ├── agent-to-server API traffic ├── pod/network overlay traffic └── datastore traffic

Avoid assigning public exposure to every node merely because every VM has a public interface available.

Administrative SSH, Kubernetes API access, datastore access, and overlay networking should each have an explicit policy.

The deeper decisions around Flannel, ingress, public endpoints, private networks, and load balancing belong in K3s Networking, Ingress, and Load Balancing.

Ingress and ServiceLB solve different traffic problems

K3s includes Traefik by default as an ingress controller and ServiceLB as a controller for Kubernetes LoadBalancer Services.

Those are related but different layers.

LayerMain jobExample
Ingress controllerRoute HTTP/HTTPS into workloadsapp.example.com → web Service
Kubernetes ServiceStable service endpoint inside clusterweb Pods behind ClusterIP
ServiceLBImplement LoadBalancer Service behavior in K3s environmentsExpose selected Service on nodes
External infrastructure LBStable external or control-plane endpointAPI registration or app traffic

For a small VM-based cluster, Traefik may be enough for HTTP routing. A more complex production environment may replace it with another ingress or Gateway implementation.

The architectural question is not “which ingress is best?” It is where public traffic enters, how unhealthy backends are removed, where TLS terminates, and which node failure can interrupt traffic.

Those decisions become especially important once the cluster has multiple agents or separate control-plane nodes.

Local-path storage is convenient, not automatically highly available

K3s packages local-path-provisioner as a simple PersistentVolume implementation.

That is convenient because a workload can request persistent storage without first integrating a separate storage platform.

But local storage is tied to the node that owns the underlying path.

Text
Pod ↓ PVC ↓ Local-path volume ↓ Specific node disk

If the Pod is rescheduled to another node, Kubernetes scheduling constraints and the storage implementation determine whether that data remains usable.

Therefore, do not infer the following from the word “PersistentVolume”:

  • the data is replicated;
  • the data survives node loss;
  • the volume can move automatically to another VM;
  • the volume is backed up;
  • the application is highly available.

For stateful production workloads, choose storage based on the failure you need to survive.

Workload stateBetter direction to evaluate
Stateless appNo persistent volume
CacheRebuildable state where practical
User uploadsObject storage in many architectures
DatabaseManaged database or deliberate stateful storage + database backup
Filesystem-dependent appPersistent cluster storage with documented recovery

The dedicated K3s Persistent Storage and Backup Strategy owns that recovery design.

K3s server sizing must include workload capacity separately

K3s has lower resource requirements than many manually assembled Kubernetes distributions, but “lightweight” does not mean resource-free.

Current K3s requirements list 2 CPU cores and 2 GB RAM as the minimum for a server node before meaningful workload capacity is added. SSD-backed disks are recommended because datastore performance matters.

That minimum is a bootstrap floor, not a production sizing recommendation.

A single-server K3s VM may need capacity for:

Text
Kubernetes control plane + datastore + container runtime + system Pods + ingress + monitoring + application Pods + headroom for rescheduling and upgrades

An agent-only VM removes most control-plane overhead, but it still needs system resources and headroom for workload spikes.

At Raff, we size K3s control capacity and application capacity as separate budgets. If a workload needs 4 vCPU of application capacity, putting it on a 4-vCPU server node leaves no sensible allowance for the cluster services that also run there.

That is why small clusters should reserve headroom even when average CPU utilization looks low.

Server nodes can run workloads, but role separation can improve operations

By default, K3s server nodes are also schedulable nodes. They can run normal application Pods.

For a small cluster, that is efficient:

Text
3 K3s servers = control plane + etcd + workloads

For larger or more sensitive clusters, teams may prefer to keep application workloads primarily on agent nodes:

Text
3 K3s servers = control plane + etcd N agents = application workloads

Role separation can help when:

  • application spikes should not compete with etcd or API-server resources;
  • control-plane maintenance should be easier to reason about;
  • security boundaries differ;
  • workload node pools need different sizes;
  • stateful workloads require dedicated placement.

It also costs more because the control-plane VMs consume capacity even when they are not serving application traffic.

Choose the separation because it reduces a specific operational risk, not because every diagram of Kubernetes shows dedicated control-plane nodes.

Storage, HA, and backup are three different controls

K3s production designs often confuse availability with recovery.

A useful distinction is:

ControlPrimary purposeDoes not replace
Multiple K3s serversControl-plane availabilityDatastore backup
etcd quorumContinue cluster-state writes through eligible failuresHistorical recovery
Replicated workload storageStorage availabilityBackup after deletion/corruption
Database replicationDatabase availabilityPITR / database backup
K3s datastore snapshotCluster-state recoveryApplication-data backup
Application backupBusiness-state recoveryControl-plane rebuild plan

An HA cluster can faithfully preserve a bad configuration, an accidental namespace deletion, or corrupted application data.

Backups preserve a recoverable past. HA preserves service through selected present-time failures.

The cluster recovery plan should therefore include both:

  1. cluster state — K3s datastore and configuration;
  2. application state — databases, PVC data, object data, secrets, and external dependencies.

For broader Kubernetes recovery layers, use Kubernetes Backup and Disaster Recovery Strategy.

Operations ownership is the main cost of self-managed K3s

K3s reduces the amount of cluster assembly. It does not make the cluster self-operating.

A production owner still needs to manage:

  • K3s version lifecycle;
  • Kubernetes version compatibility;
  • server and agent OS patching;
  • certificates and tokens;
  • firewall rules;
  • cluster datastore health;
  • node capacity;
  • CNI health;
  • ingress health;
  • storage health;
  • backups and restore tests;
  • workload resource requests and limits;
  • node drains and maintenance;
  • incident response.

That ownership is what separates self-managed K3s from managed Kubernetes.

The existing K3s vs Kubernetes guide owns the platform-choice comparison. The practical point here is simpler: K3s reduces integration surface, but the team still operates the control plane.

If operating that control plane does not create product value, managed Kubernetes may be the more appropriate boundary.

Upgrades should be treated as cluster maintenance

K3s tracks Kubernetes releases through K3s-specific builds. As of September 2026, K3s v1.37 is available, while older supported release lines continue to receive patch releases. Production upgrade planning should follow the official K3s release notes and supported-version path rather than treating one version number in an architecture guide as permanently current.

Production upgrades should not be treated like updating a normal stateless package on every VM simultaneously.

A safer architecture has:

  • a documented supported-version path;
  • a current datastore backup;
  • enough capacity to drain or restart nodes safely;
  • workload disruption budgets appropriate to the application;
  • a tested rollback or recovery path;
  • staged server and agent upgrades;
  • post-upgrade workload validation.

For HA clusters, quorum must remain intact while server nodes are maintained.

Version boundaries can also change rollback behavior. K3s v1.36 moved the bundled etcd line to 3.6, and K3s v1.37 moved embedded etcd to 3.7. Treat a major datastore transition as a recovery-planning event: take and validate the required snapshots, read the release-specific upgrade notes, and do not assume an in-place binary downgrade is safe.

The dedicated K3s Upgrade and Rollback Strategy owns the exact maintenance model so this connector guide does not become a procedural upgrade tutorial.

The decision framework matches topology to workload risk

Workload situationK3s architecture to evaluateMain trade-off
Development/CISingle serverLowest cost and complexity
Small low-criticality productionSingle server + tested recoveryOne control-plane failure domain
More compute, control-plane downtime acceptableSingle server + agentsAdds workload capacity, not HA control plane
Production needs control-plane HA3+ servers with embedded etcdQuorum, disk and maintenance complexity
Team already operates reliable DB infrastructureHA servers + external datastoreExternal dependency and DB operations
Stateful workloads dominateDeliberate storage + backup architectureStorage may be harder than compute
Team wants Kubernetes but not control-plane ownershipManaged KubernetesLess control-plane work, different service boundary

There is no universal “production K3s topology.”

The architecture should follow failure tolerance, recovery objectives, workload state, team skill, and maintenance capacity.

A three-server embedded-etcd cluster is not automatically better than one server if nobody can maintain quorum or restore etcd. A single server is not automatically unsafe if the workload is replaceable and the recovery window is acceptable.

The correct design is the smallest topology whose failure and recovery behavior the team can explain before an incident.

Raff VMs provide the self-managed K3s boundary

K3s can run on standard Linux cloud VMs, which makes Raff VM the natural Raff infrastructure boundary for a self-managed deployment.

A staged Raff topology can look like:

Text
Stage 1 Raff VM └── single K3s server Stage 2 Raff VM server ├── Raff VM agent └── Raff VM agent Stage 3 3 Raff VM K3s servers + agent VMs as required + private node communication + externalized application data where appropriate

Keep cluster communication on private networking wherever possible and expose only the endpoints the architecture actually requires.

Use Raff VM when your team deliberately wants to own K3s servers, agents, upgrades, datastore recovery, networking, and cluster maintenance.

Use Raff Kubernetes when the Kubernetes API and workload model are valuable but self-managing the control plane is not.

At Raff, the useful boundary is responsibility rather than branding: self-managed K3s gives the team more cluster ownership; managed Kubernetes removes some of that ownership.

Deploy Linux VMs for a self-managed K3s cluster when your team wants direct control of the Kubernetes control plane, datastore, networking, and upgrade lifecycle.

14-day money-back guarantee

Deploy Your Server

Production architecture checklist

Before calling a K3s-on-VM design production-ready, confirm:

  • Single-server or HA topology is explicitly chosen.
  • Server and agent roles are documented.
  • Control-plane and workload capacity are sized separately.
  • Server disks use appropriate low-latency storage.
  • API port 6443 is restricted to required sources.
  • Flannel overlay traffic stays on trusted/private networks.
  • Embedded-etcd ports are restricted to server nodes when used.
  • A fixed registration endpoint exists for HA where required.
  • Ingress and application traffic entry are documented separately from the API endpoint.
  • Persistent workloads have an explicit storage class and failure model.
  • Cluster-state backup is separate from application-data backup.
  • Secrets and node tokens have controlled recovery paths.
  • Monitoring covers server, datastore, node, and workload health.
  • Upgrade order and maintenance capacity are documented.
  • Datastore and application restores have been tested.
  • One team or person owns after-hours cluster incidents.

K3s on cloud VMs is a self-managed Kubernetes architecture

K3s reduces Kubernetes packaging and integration work, but the architecture still has the same core production questions: control-plane availability, datastore quorum, node networking, workload placement, ingress, persistent storage, backups, upgrades, and recovery.

Start with one server when the workload can tolerate that failure boundary. Add agents when compute capacity is the problem. Move to three or more embedded-etcd servers, or an external-datastore HA design, when control-plane availability justifies the extra operating surface.

Most importantly, keep availability, persistence, and recovery separate. A cluster that survives one node failure is not automatically a cluster that can recover deleted application data or corrupted state.

Continue with K3s High Availability: Embedded etcd vs External Database for datastore topology, then K3s Networking, Ingress, and Load Balancing for the VM network boundary.

Sources

Was this article helpful?
Serdar Tekin
Serdar Tekin
Co-Founder & CTO

Co-Founder & CTO at Raff, focused on cloud infrastructure, systems architecture, security, reliability, and production operations.

Run K3s on your own cloud VMs, from $4.99 a month.

Ubuntu, Debian and other Linux distributions with full root and SSH access, up and running in about 55 seconds.

14-day money-back guarantee

Frequently asked questions

What is the basic K3s architecture?
K3s uses server nodes for the Kubernetes API, control plane, and datastore, while optional agent nodes run application workloads. A single server can also run workloads and form a complete one-node cluster.
Can K3s run on one cloud VM?
Yes. A single K3s server is a fully functional Kubernetes cluster and can run both control-plane components and application workloads. It is not a highly available control plane.
How many K3s servers are needed for high availability?
For embedded-etcd HA, current K3s guidance requires 3 or more server nodes and recommends an odd number to maintain etcd quorum efficiently.
Does adding K3s agent nodes make the control plane highly available?
No. Agents add workload capacity. If there is still only one server node, that server remains the control-plane failure domain.
What datastore does K3s use?
A simple single-server cluster uses embedded SQLite by default. K3s also supports embedded etcd and supported external datastores such as etcd, MySQL, MariaDB, and PostgreSQL.
Is K3s local-path storage highly available?
No. Local-path provisioning provides convenient persistent storage tied to node-local disk. It should not be assumed to provide replication, node-failure survival, or backup.
Which K3s ports should stay private?
The API requires TCP 6443 for cluster nodes, Flannel VXLAN commonly uses UDP 8472 between nodes, metrics-server uses TCP 10250, and embedded etcd uses TCP 2379–2380 between servers. K3s explicitly warns not to expose VXLAN port 8472 publicly.
When should I use Raff VMs instead of managed Kubernetes for K3s?
Use Raff VMs when your team deliberately wants to self-manage K3s servers, agents, datastore, networking, upgrades, backups, and recovery. Use managed Kubernetes when control-plane operation is not work your team needs to own.