Secrets management is the lifecycle used to create, store, deliver, rotate, revoke, and audit credentials such as database passwords, API keys, signing keys, service tokens, SSH private keys, and TLS private keys. For most small cloud teams, environment variables are useful for runtime delivery, a managed secret store is the practical next step when central control matters, and a Vault-style system fits when dynamic credentials and stronger policy boundaries justify extra operational ownership.
Application secrets remain a workload responsibility on Raff Technologies. Raff can restrict infrastructure access with IAM, project scopes, API-key roles, expiry, audit logs, firewalls, and private networking, but your application still needs a deliberate secrets workflow.
Use this decision table as the starting point:
| Situation | Practical starting point |
|---|---|
| One small app, few secrets, limited operators | Controlled runtime environment variables |
| Several services or environments, central inventory needed | Managed secret store |
| Dynamic or short-lived credentials, strict policy boundaries | Vault-style system |
| Kubernetes workloads | Kubernetes Secrets with encryption and RBAC, or an external store |
| CI/CD automation | CI/CD secret store or an external secrets manager |
The important distinction is simple: an environment variable can deliver a secret, but it does not manage the secret's lifecycle.
Secrets management is a lifecycle, not a storage location
A secret is any value that grants identity, trust, access, or privileged capability. Common examples include database credentials, API keys, OAuth client secrets, signing keys, SSH private keys, TLS private keys, webhook secrets, deployment tokens, registry credentials, and backup credentials.
Normal configuration and secrets should not be treated the same way:
APP_ENV=production # configuration PUBLIC_BASE_URL=... # configuration DATABASE_PASSWORD=... # secret PAYMENT_API_KEY=... # secret
A mature workflow covers more than storage:
Create -> Store -> Deliver -> Use -> Rotate -> Revoke -> Audit -> Recover
This is where small systems usually become difficult. One app may start with three credentials and a local .env file. Add staging, CI/CD, workers, contractors, backups, containers, and multiple services, and the same values are soon copied into places nobody owns.
The most common failure patterns are predictable:
- secrets committed to Git or embedded in images;
- production credentials reused in staging;
- one shared credential used by several services;
- values pasted into chat, tickets, screenshots, or documentation;
- CI/CD logs printing sensitive values;
- no inventory showing which service uses which credential;
- rotation that depends on one person remembering every consumer;
- backups or exports creating uncontrolled plaintext copies.
Secrets management therefore becomes an ownership and revocation problem as much as a storage problem.
Environment variables are useful delivery, but weak lifecycle control
The Twelve-Factor App recommends storing deploy-specific configuration in environment variables rather than hard-coding it in the application. That model remains useful because environment variables are language-agnostic and easy to inject at deployment time.
They work well when the application is small, the number of secrets is limited, production access is restricted, and manual rotation is still predictable.
But environment variables do not automatically provide:
- a central inventory;
- per-secret access policy;
- a record of who viewed or changed a value;
- expiry;
- revocation;
- rotation coordination;
- proof that old values were removed;
- protection from accidental logging or crash-dump exposure.
OWASP specifically warns that environment variables can be exposed to processes, logs, or system dumps, and recommends avoiding them for secrets when stronger delivery methods are available.
Use environment variables more safely by keeping the source of truth elsewhere when possible, separating development, staging, and production values, excluding local secret files from version control, and preventing application or CI logs from printing secrets.
A useful pattern is:
Secret source of truth | v Deployment system | v Environment variable or mounted file | v Application runtime
The runtime may still consume an environment variable. The important improvement is that the credential is centrally owned and can be replaced without searching through laptops and repositories.
Managed secret stores are the practical middle ground
A managed secret store centralizes sensitive values and normally integrates access control, encryption, API retrieval, and auditability without requiring the team to operate the secrets service itself.
This model fits teams that have outgrown copied .env files but do not need a dedicated secrets platform.
Choose a managed store when several of these are true:
- secrets exist across multiple services or environments;
- developers need one authoritative inventory;
- production access should be restricted by identity or workload;
- rotation needs to be repeatable;
- audit records matter;
- CI/CD needs a controlled source for deployment credentials;
- operating a dedicated Vault cluster would add more responsibility than value.
A managed store does not remove design work. You still need to decide which identity can read which secret, whether applications fetch at runtime or receive values at deploy time, how failures behave, and how old credentials are revoked.
The advantage is operational: centralization reduces secret sprawl without forcing a small team to own another critical platform.
Vault-style systems fit dynamic and policy-heavy environments
Vault-style systems become useful when secret management itself is a platform concern.
HashiCorp Vault, for example, supports secrets engines that can store values or generate credentials on demand. Dynamic credentials can carry leases and be revoked when the lease ends or is explicitly revoked.
That model is valuable when:
- many services need different access boundaries;
- long-lived shared credentials create unacceptable blast radius;
- database or infrastructure credentials can be generated dynamically;
- short-lived access is part of the security model;
- audit and revocation need to be centralized;
- several environments or clouds need a consistent policy layer.
The trade-off is ownership. A self-operated vault becomes critical infrastructure: authentication methods, policies, storage, availability, recovery, upgrades, audit logs, monitoring, backups, and break-glass access all need an owner.
The decision is therefore not "Is Vault more advanced?" It is "Will dynamic credentials and centralized policy reduce more risk than the platform adds in operational responsibility?"
Containers, Kubernetes, and CI/CD create different leak paths
Containerized and automated deployments do not eliminate secrets. They change where secrets can leak.
Docker images should never become secret storage
Do not bake sensitive values into Dockerfiles, image layers, build arguments that persist in build history, committed Compose files, labels, or image metadata.
Docker documents a stronger encrypted secrets workflow for Swarm services: secrets are encrypted in transit and at rest in the swarm and exposed only to services granted access. Docker also states that this mechanism is not available to standalone containers.
That distinction matters. A secrets: declaration in a local Compose workflow should not be assumed to provide the same security properties as Swarm secret management.
Kubernetes Secrets require explicit hardening
Kubernetes Secrets keep sensitive data out of ordinary application configuration, but the Kubernetes documentation states that Secret objects are stored unencrypted in etcd by default unless encryption at rest is configured.
For production clusters:
- enable encryption at rest for Secret data;
- use least-privilege role-based access control (RBAC);
- restrict who can list, watch, or read Secrets;
- remember that a user who can create Pods may gain indirect access to namespace Secrets;
- consider external secret stores when central lifecycle control is needed;
- avoid assuming base64 encoding provides encryption.
Kubernetes also recommends short-lived Secrets where practical and careful control over Secret access.
CI/CD credentials are production credentials
Deployment pipelines often hold registry tokens, SSH keys, cloud credentials, database migration credentials, signing keys, and webhook secrets.
Keep them in the CI/CD platform's protected secret store or an external manager, separate credentials by environment, restrict who can change production pipelines, and prefer short-lived or narrowly scoped machine credentials when the underlying service supports them.
A pipeline that can deploy production code or change infrastructure is part of the production security boundary.
Rotation and least privilege determine whether the system is operable
Rotation is useful only when the team can perform it without guessing which services will break.
For each production secret, record at least:
Name Owner Environment Consumers Source of truth Revocation method Rotation trigger or policy Verification step Recovery or rollback path
Do not choose an arbitrary rotation interval simply because it looks disciplined. Rotation should reflect the credential type, provider capability, exposure risk, internal policy, and whether short-lived credentials are available.
Rotate immediately after confirmed or suspected exposure. Also rotate when access ownership changes, a provider or policy requires it, or the credential reaches its defined lifetime.
When the system supports overlapping credentials, a low-risk rotation pattern is:
- create the replacement credential;
- grant it the required scope;
- deploy consumers using the replacement;
- verify normal traffic and jobs;
- revoke the old credential;
- monitor errors and update the inventory.
If the provider does not support overlapping credentials, design and test a provider-specific cutover rather than assuming the two-key pattern will work.
Least privilege is the companion control. Separate credentials by service, environment, and automation boundary instead of sharing one production administrator credential everywhere.
api-service -> app_db_user worker-service -> worker_db_user reporting -> readonly_db_user backup-job -> backup-specific credential
This makes revocation and investigation much more predictable.
A practical maturity path keeps complexity proportional
Most small teams do not need the final architecture on day one.
| Stage | Practical control | Move forward when... |
|---|---|---|
| 1. Disciplined delivery | Separate env vars or mounted files, no secrets in Git/images/logs | Copying and ownership become hard to track |
| 2. Centralized management | Managed secret store, access policy, inventory, rotation runbooks | Dynamic credentials or cross-service policy become valuable |
| 3. Dedicated secrets platform | Vault-style policies, leases, dynamic credentials, centralized revocation | The team can operate the platform reliably |
A 3-stage maturity path is often safer than deploying a complex secret platform before anyone owns its recovery procedure.
The trigger to move is operational pain or risk: more services, more environments, more people, more automation, more audit requirements, or a rotation process that has become fragile.
Raff IAM reduces infrastructure credential blast radius
Raff's current IAM model is a useful example of the same design principle at the infrastructure layer. API keys can hold a role, be scoped to projects, carry an optional expiry, and every API call is recorded in the audit log.
That means a CI key can be separated from a human administrator account and limited to the project and actions it actually needs. It also means infrastructure automation does not need one permanent account-wide token.
Raff IAM is not an application secrets manager, however. A database password, payment API key, JWT signing key, or third-party service token still needs its own application-level storage and rotation workflow.
For workloads on Raff, pair secrets management with other boundaries:
- Raff IAM for account, role, project, and API-key access;
- Cloud Security for security groups, audit visibility, and platform security controls;
- VPC for private service-to-service paths;
- Cloud Security Fundamentals for the wider identity, network, patching, recovery, and incident model;
- Raff API Keys for Automation for infrastructure automation credentials;
- API Rate Limiting for controlling application-layer request abuse.
Private networking and firewalls reduce reachability; scoped credentials reduce what a compromised service can do. Neither control should be expected to replace the other.
Production secrets need a short operational checklist
Before a production launch or architecture review, confirm:
- secrets are not committed to source control;
- container images and build logs do not contain secrets;
- development, staging, and production use separate credentials;
- every important production secret has an owner;
- each service receives only the credentials it needs;
- CI/CD credentials are scoped and protected;
- logging and error reporting redact sensitive fields;
- Kubernetes Secret encryption and RBAC are configured if Kubernetes is used;
- rotation and emergency revocation have documented verification steps;
- backups do not create uncontrolled plaintext secret exports;
- exposed credentials are rotated, not merely deleted from the place they leaked;
- old credentials are revoked after a successful cutover.
If a secret leaks, treat the value as compromised: revoke or rotate it, deploy the replacement, search for other copies, review access logs, and document what allowed the exposure.