Orchestration & Kubernetes

ConfigMap vs Secret — and the Honest Limit of a Secret

Configuration and credentials both come from outside the image, and Kubernetes offers two objects for them. The difference is smaller than the names suggest: a Secret is base64-encoded, not encrypted, and readable by anyone with namespace access.

The question this answers

Infrastructure question

Where does environment-specific configuration come from, and does putting a credential in a Secret object actually protect it?

Application requirement

One api:v7 image must run in staging and production with different database hosts, feature flags and log levels — and with a production database password that must not be in the image, the repository, or readable by every engineer with cluster access.

What it provides

Configuration and credentials injected at pod start rather than baked into the image, so one artifact is promoted unchanged across environments — plus, for Secrets, a separate object kind that can be permissioned, audited and encrypted differently.

Application Requirement→Infrastructure Requirement→Compute→Network→Storage→Identity→Deployment→Scaling→Reliability→Observability→Security→Cost→Trade-offs

Two objects, one purpose, one real difference

Both objects are key-value maps that a pod can consume as environment variables or as mounted files. The mechanics are nearly identical. The difference is what the platform is willing to do *around* them: Secrets are a distinct kind, so RBAC can grant read access to ConfigMaps without granting it to Secrets; they can be encrypted at rest in the datastore if you enable it; their values are omitted from most log output; and access to them is auditable as a distinct event.

The choice between environment variables and mounted files matters more than most teams think. Environment variables are read once at process start, so changing a ConfigMap does not change a running pod — you must roll the Deployment for it to take effect. Mounted files are updated in place after a propagation delay, but only if your application re-reads them, which most applications do not. Neither mechanism is live reload, and assuming otherwise produces the memorable incident where a config change appears to do nothing for three days and then takes effect during an unrelated deploy.

A useful rule: annotate the pod template with a hash of the config it depends on. Then a config change alters the template, which triggers a rollout automatically, which makes the change atomic and visible in the deployment history rather than invisible and deferred.

PropertyConfigMapSecret
Stored in the cluster datastoreYes, plaintextYes — base64, and plaintext unless encryption at rest is enabled
Encrypted by the object type itselfNoNo. Base64 is an encoding, not encryption
Separate RBAC surfaceYesYes — this is the main practical benefit
Redacted in typical tooling outputNoUsually, which is a convenience and not a control
Consumable as env varsYes, read once at process startYes, read once at process start
Consumable as mounted filesYes, updated in place after a delayYes, updated in place after a delay
Size limitAbout 1 MiB — it is not a file storeAbout 1 MiB
Rotation without a restartOnly if the app re-reads the fileOnly if the app re-reads the file
Suitable for a production database password on its ownNoOnly with encryption at rest, tight RBAC and audit — see below
What actually differs — and what does not

The honest limit: base64 is not a security control

This is the part that must not be softened. A Kubernetes Secret object is not, by itself, secret management. Its value is base64-encoded, which is a transport encoding with no key and no protection. Anyone who can read Secrets in the namespace can decode it in one command. By default, in a cluster where encryption at rest has not been explicitly configured, it is also sitting in plaintext in the datastore and in the datastore's backups.

That does not make Secrets useless — it makes them a *starting point* with three specific obligations. One: enable encryption at rest for Secrets in the datastore, ideally with keys held in a managed key service so the cluster cannot decrypt its own backups. Two: treat get secrets as a privileged permission and grant it deliberately, which means auditing which roles have it today, because default roles and convenience bindings hand it out generously. Three: understand that a secret consumed as an environment variable is visible to anything that can read the process environment, appears in crash dumps, and is frequently logged by well-meaning error handlers.

The real answer for high-value credentials is an external secret manager with short-lived, rotatable credentials, injected at pod start via a workload identity rather than stored in the cluster at all. Secrets in Infrastructure covers the lifecycle, and Roles vs Static Keys covers why a credential that expires in an hour is worth far more than one that is merely well hidden.

The permission that actually protects the credential is RBAC, not the object kind
Role "developer", bound cluster-wide during onboardinghumandangerous
on All namespaces, including production
Allowed
  • get / list / watch pods
  • get / list ConfigMaps
  • get / list **Secrets**
  • create pods/exec
  • port-forward
Actually needed
  • get / list / watch pods in team namespaces
  • read logs
  • get / list ConfigMaps

Blast radius: Every credential in the cluster: production database passwords, third-party API keys, TLS private keys. pods/exec additionally grants a shell inside any workload and access to its service-account token, so this role can act as any application in the cluster. Nothing about storing those credentials in Secret objects rather than ConfigMaps changed any of this.

What consuming them looks like, and how to get it less wrong

Kubernetes· Kubernetes 1.29. Whether Secrets are encrypted at rest depends entirely on cluster configuration; several managed offerings enable it by default and several do not.

The manifest below shows the two consumption modes side by side, with the annotation trick that makes config changes trigger a rollout. Note the deliberate split: non-sensitive settings come from a ConfigMap as environment variables, and the credential comes from a mounted file rather than an environment variable — because a file is not inherited by child processes, does not appear in /proc/<pid>/environ, and is far less likely to end up in a log line.

The last block is what a mature setup actually looks like: no long-lived credential in the cluster at all. The pod authenticates with its workload identity, an agent fetches a short-lived credential from an external secret manager, and the value in the cluster is a reference rather than a password. That is more moving parts, and for a hobby project it is not worth it — which is exactly the kind of judgment No Cargo-Cult Infrastructure asks for.

1apiVersion: apps/v1
2kind: Deployment
3metadata: { name: api }
4spec:
5  template:
6    metadata:
7      annotations:
8        # changes the pod template when config changes -> triggers a rollout -> change is atomic
9        checksum/config: "sha256:41c9..."
10    spec:
11      containers:
12        - name: api
13          envFrom:
14            - configMapRef: { name: api-config }     # non-sensitive: log level, feature flags, hosts
15          volumeMounts:
16            - name: db-credential                     # sensitive: a FILE, not an env var
17              mountPath: /run/secrets
18              readOnly: true
19      volumes:
20        - name: db-credential
21          secret: { secretName: api-db, defaultMode: 0400 }
22 
23# What the "Secret" actually contains, and why that is not protection:
24#   data:
25#     password: c3VwZXItc2VjcmV0     <-- base64. `base64 -d` reveals it. No key involved.
26#
27# Better, where the credential is worth it: nothing sensitive is stored in the cluster.
28#   1. The pod authenticates as its workload identity (no stored key).
29#   2. An agent or CSI driver fetches a short-lived credential from an external manager.
30#   3. The credential expires in minutes; rotation is the platform's job, not a runbook step.
Consumption modes, from adequate to good. Illustration, not a template.

Key points

  • ConfigMaps and Secrets are the same mechanism; the real difference is a separate RBAC surface, separate audit and the option of encryption at rest.
  • A Secret is base64-encoded, not encrypted. Anyone who can read Secrets in the namespace can decode it instantly.
  • Encryption at rest for Secrets is a cluster configuration decision, not a property of the object kind — verify it rather than assuming it.
  • Environment variables are read once at process start, so a config change does nothing until the pod is rolled; annotate the template with a config hash so it rolls automatically.
  • For high-value credentials, the right answer is short-lived credentials from an external manager fetched via workload identity, not better-hidden long-lived ones.

The loop, answered

Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.

How it works
  • • Both objects are stored in the cluster datastore and referenced by name from a pod spec.
  • • At pod start, the node agent fetches the referenced objects and injects them as environment variables or as files in a mounted volume.
  • • Environment variables are materialized once into the process; nothing updates them afterwards.
  • • Mounted values are refreshed by the node agent after a propagation delay, but only take effect if the application re-reads the file.
  • • RBAC on the Secret kind decides who may read the values through the API; encryption at rest, if enabled, decides whether the datastore holds plaintext.
What you still own
  • • Verifying that encryption at rest is actually enabled in your cluster, and that the key is managed outside it.
  • • Auditing who holds get secrets and pods/exec, both of which are effectively credential access — see Least Privilege in Infrastructure.
  • • Rotation: changing a Secret does not change a running pod, so rotation is a rollout unless the application re-reads its files.
  • • Keeping credentials out of the repository even in encrypted form unless you have deliberately adopted a sealed-secret workflow with a managed key.
  • • Datastore backups, which contain every Secret and therefore inherit the same classification as the credentials themselves.
How it fails
  • • A referenced ConfigMap or Secret that does not exist: pods fail with CreateContainerConfigError and never start — a total outage from a missing key.
  • • A config change that appears to do nothing because env vars were read at start, then takes effect days later during an unrelated deploy.
  • • A credential leaked into logs by an error handler that dumps the environment on an unhandled exception.
  • • Rotation performed in the secret manager but not rolled out to pods, so the application authenticates with a revoked credential and fails all at once.
  • • A cluster backup restored into a less-secured environment, carrying every production credential with it.
How it scales
  • • The ~1 MiB object limit means these are not a file distribution mechanism; large configuration belongs in object storage with a reference here.
  • • Every pod start reads its config objects, so a mass restart creates a burst of Secret reads — visible in audit logs and worth understanding before it looks like an attack.
  • • Secret count and namespace count grow with teams, and access review becomes the binding constraint long before any technical limit does.
Security
  • • The trust boundary is RBAC plus datastore encryption, not the object kind. Storing something in a Secret changes nothing on its own.
  • • get secrets and pods/exec are both credential access; grant them as you would grant production database access.
  • • Prefer file mounts over environment variables for credentials: files are not inherited by child processes and are much less likely to be logged.
  • • Short-lived credentials obtained through workload identity reduce the value of theft far more than any storage improvement does — see Roles vs Static Keys and Human vs Workload Identity.
  • • Every pod in a namespace can, by default, mount that namespace's Secrets if its spec references them; namespace boundaries are the practical isolation unit.
Cost shape
  • • The objects themselves are free; the cost is operational — rotation, review and the incident response when a long-lived credential leaks.
  • • An external secret manager adds a small per-secret and per-request charge, which is trivially worth it for anything production-grade.
  • • The largest cost is the one nobody budgets: rotating a credential that turned out to be embedded in six systems nobody documented.
What to watch
  • • Audit-log events for Secret reads, by identity — the signal that tells you who is actually accessing credentials.
  • • Pods stuck in CreateContainerConfigError, which is always a missing or misnamed config reference.
  • • Credential age and time-to-rotate as tracked metrics, because long-lived credentials decay silently — see Secrets in Infrastructure.
  • • The signal that lies: "the Secret exists and the value is correct". It says nothing about who can read it, whether it is encrypted at rest, or whether the running pod has picked it up.
Simpler alternatives
  • • An external secret manager with short-lived credentials injected at pod start — the correct answer for anything valuable.
  • • Workload identity with no stored credential at all, where the platform supports it: the credential you never store cannot leak.
  • • Plain environment variables from your deployment platform, for a small system with one environment — a PaaS config panel is a legitimate secret store at that scale.
  • • For non-sensitive configuration, a plain ConfigMap or even a file in the image, when the value genuinely does not differ between environments.
What adopting this costs
  • • Buys one image promoted unchanged across environments; costs an external dependency at pod start that can prevent the pod from ever starting.
  • • Buys a separate permission and audit surface for credentials; costs the false confidence that the object kind is itself a protection.
  • • Buys simplicity by keeping credentials in the cluster; costs a permanent obligation to encrypt, restrict and audit the datastore and its backups.

What people believe, and what is true

Claim

Secrets are encrypted, that is the point of the object.

Reality

They are base64-encoded. Encryption at rest is a separate cluster configuration that must be enabled, and even then it protects the datastore, not the reader with RBAC access.

Claim

Updating a ConfigMap updates the running application.

Reality

Environment variables are read once at start. Mounted files update after a delay but only matter if the application re-reads them. Roll the Deployment.

Claim

Putting a password in a Secret keeps it out of the repository, so we are fine.

Reality

It is now in the datastore and its backups, readable by anyone with get secrets and by anyone who can exec into a pod that mounts it.

Apply it

⚡ Leak Secret🔍 Access Denied, and the Policy Says Allow
← Ingress and Gateway: Getting Traffic InStateful Workloads: Databases Are Not Stateless APIs →