SecretsGENERALCLOUD-SPECIFICPLATFORM-SPECIFIC

Workload Identity

The primitive that removes the first secret: the platform attests what a workload is, that attestation is exchanged for a short-lived credential, and no static key exists anywhere for an attacker to find.

The question, the obvious approach, and why it breaks

Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.

The production question

If secrets live in a manager, what credential does the workload use to authenticate to the manager?

The problem

Every secret-management design terminates in a bootstrap question. Something must prove who the caller is, and if that proof is itself a stored secret then the whole structure rests on the one credential nobody rotates.

What teams do first

Create a service account, generate a long-lived key for it, and store that key as an environment variable so the workload can authenticate to the secret manager.

How it breaks

That key is the master credential, and it is stored the way the design was trying to avoid — as a static value in a deployment specification.

How it breaks in production
  • That key is the master credential, and it is stored the way the design was trying to avoid — as a static value in a deployment specification.
  • It does not expire, so a disclosure from a year ago is still exploitable today, and nothing in the system will tell you (Short-Lived Credentials).
  • It is portable. A static key works from anywhere: a laptop, a compromised CI job, another cloud account. There is nothing in it that ties use to the workload it was issued for.
  • It gets shared. One key is created for a service and reused by a second service, then a script, then a person debugging — and it stops being possible to say who is using it.
  • Rotating it means coordinating every consumer at once, which is why it is deferred, which is why these keys are routinely years old (Rotation That Applications Survive).
CodeBuildTestArtifactReleaseDeployRunObserveOperateIncidentRecoverLearnImprove

What is actually happening

Underneath the tooling, which is the part that survives a change of tool.

  • The underlying primitive is attested identity exchanged for a short-lived credential, and it is worth understanding before any product name is attached, because every provider implements the same three-step shape.
  • Step one: the platform — the orchestrator, the hypervisor, the CI system — knows what a workload is, because it created it. It issues a short-lived, cryptographically signed assertion of that fact: this process is workload W, in project P, running as service account S. The workload cannot forge it; the platform issues it and the workload merely reads it.
  • Step two: the workload presents that assertion to an authorisation service, which verifies the signature against the platform's published keys and evaluates a trust policy — the rules saying which assertions are acceptable, expressed as conditions on the claims inside them.
  • Step three: if the policy accepts, the workload receives a short-lived credential — minutes to an hour — scoped to what that identity may do. It expires on its own; there is nothing durable to steal, and nothing to rotate (Identity and Access Management (IAM)).
  • Three properties fall out. Nothing static exists to leak. The credential is bound to the workload, so it is not portable in the way a key is. And expiry is automatic, so revocation happens by the passage of time rather than by a process someone must remember to run.
  • Named products implement this: cloud providers offer it for their compute services, Kubernetes projects service account tokens that can be federated to a cloud identity, and CI systems issue signed tokens that cloud providers accept through a trust policy. The primitive is the same; the claim names and the policy syntax are not (Human vs Workload Identity).

The primitive, before any product name

GENERALThe shape is the same whether the platform is a cloud compute service, a Kubernetes cluster projecting a service account token, a CI system issuing a signed job token, or a service mesh issuing workload certificates. What differs is the claim set and the policy language — and those differences are precisely what does not survive being copied between providers.

Three steps, one property. The workload never holds anything durable, and the credential it does hold expires whether or not anyone remembers to revoke it.

Note where the trust decision is made: in the policy, not in the workload. The workload presents a claim it cannot forge, and something else decides whether that claim is acceptable.

issuesreadable bypresents assertionevaluates claimsif acceptedauthenticates, then expiresPlatform knows what it startedSigned assertion short-lived, claims insideWorkloadAuthorisation service verifies signatureTrust policy conditions on claimsShort-lived credential scoped, auto-expiringSecret store / API
UserLLMAgentToolDataDecisionHumanGuardrail

A static key against an attested identity

Both designs authenticate a workload to a secret store. The difference is what exists in the world afterwards, and that is the whole argument.

Two ways for a service to reach the secret store
Long-lived service account key
create service account
generate key  ->  AKIA... / .json

store it:
  deployment spec (base64)
  CI variable
  a laptop, for testing
  a wiki page, once

properties
  no expiry
  works from anywhere
  shared without a trace
  rotation = coordinate every
    consumer at once, so it is
    deferred indefinitely
  audit shows the key, never
    who used it
Platform-attested identity
workload starts
  platform projects a signed
  assertion:
    sub: system:serviceaccount:
         payments:checkout
    aud: sts.provider
    exp: +10m

exchange under a trust policy
  requires: this issuer
         AND this namespace
         AND this service account
  ->  credential valid 1h,
      scoped to 2 secrets

properties
  nothing durable to steal
  bound to the workload
  expires by itself
  audit shows the identity

The left column's risk is not that the key is weak — it is that a durable, portable, shareable value exists at all, and that its blast radius is unbounded in time. The right column has no such object: the only thing to steal expires within the hour and does not work from anywhere else. The remaining risk moves entirely into the trust policy, which is a reviewable, versionable artifact rather than a string in a deployment specification (Roles vs Static Keys).

The trust policy is where this goes wrong

Once the primitive is in place, essentially every failure is a policy that accepts more than it should. These are configuration errors with quiet symptoms: everything works, and it also works for someone it should not.

TriggerSymptomCauseResponse
Policy binds only on the issuerNothing — it works, and it works for other workloads on the same platform tooThe condition set omits the claims that identify this specific workloadRequire a condition on the workload-identifying claims; fail the policy check when they are absent (Policy as Code)
Policy accepts any branch or any repositoryA fork or an unprotected branch can obtain production credentialsA wildcard in the subject condition, often added to make the initial setup workPin repository, and pin branch or environment; verify by attempting the exchange from elsewhere (Protected Branches)
Credential fetched once at startup, never refreshedAuthentication fails abruptly after the credential lifetime, with no deploy nearbyThe application treats a short-lived credential as a static oneRefresh ahead of expiry and re-authenticate on a rejection; test by shortening the lifetime in a lower environment
Clock skew on the nodeIntermittent authentication failures that look like permission errorsAssertion validity windows are time-bound and the node clock has drifted (Clock Synchronisation)Monitor clock offset; treat validation failures and authorisation denials as distinct in logs
Identity shared across several workloadsAudit cannot attribute an action; least privilege is impossibleOne identity created for convenience during setup and then reusedOne identity per workload, provisioned automatically so the convenient path is also the correct one
Static key left in place as a fallbackThe migration is reported complete and the old risk is unchangedThe key was kept "just in case" and nobody removed itVerify the new path under load and during a full restart, then delete the key at the issuer and confirm (Access Review)

How to do it properly

Most important first.

  • Use platform-native workload identity wherever the workload runs on a platform that offers it. This removes the bootstrap secret rather than protecting it.
  • Write the trust policy narrowly. Bind on every claim that identifies the specific workload — service account, namespace, repository, branch or environment — not just on the issuer (Anatomy of a Policy).
  • Give each workload its own identity. Shared identities destroy the audit trail and make least privilege impossible (Least Privilege in Production).
  • Handle credential expiry in the application: refresh before expiry and re-authenticate on a 401, rather than assuming a credential obtained at boot lasts forever (Build-Time and Runtime Configuration).
  • Where a workload runs somewhere without native identity, federate from wherever an attestation does exist rather than falling back to a static key.
  • Audit for static keys as a standing check: every long-lived key that still exists is a candidate for replacement, and its age is the risk (Access Review).
  • Keep a break-glass path that does not depend on the identity system, and make its use noisy (Break-Glass Access).

How much can this affect

Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.

Blast radius if this is wrongEveryone
One testEveryone
What contains it

Contained well in the intended case: credentials expire on their own and are scoped per workload. Contained by nothing in the misconfigured case — an over-broad trust policy is a standing grant of production access to whoever satisfies it, and expiry does not help because a new credential can always be minted.

What can go wrong

Failure modes, including of the mitigation
  • A trust policy bound only on the issuer, so any workload from that platform — including someone else's — can assume the role. This is the failure that matters most and it is a configuration error, not a protocol weakness.
  • A policy that accepts any repository or any branch, so a fork or an unprotected branch can obtain production credentials (Secrets in CI).
  • Clock skew causing assertion validation to fail, presenting as intermittent authentication errors that look like a permissions problem (Clock Synchronisation).
  • An application that fetches a credential once and never refreshes, which works for the credential's lifetime and then fails abruptly.
  • Identity federation configured but a static key left in place as a fallback, so the removal never actually happened.
  • Over-broad permissions attached to the identity, which means the exchange worked perfectly and granted too much anyway.
  • The identity provider becoming a single point of failure for every workload's ability to start (When Secrets Fail).
Misreads this invites
  • "Workload identity means no secrets." It removes the bootstrap secret and the static keys. Application secrets still exist; they are now fetched under an identity instead of being stored beside the code.
  • "The token is short-lived, so a misconfigured trust policy is low risk." A policy that accepts the wrong workloads can be exchanged for a fresh credential on demand, indefinitely. Lifetime bounds a stolen credential, not a broken policy.
  • "It is a cloud feature." The primitive — platform-attested identity exchanged for a short-lived credential — long predates the products, and the same shape appears in orchestrators, CI systems and service meshes. Learn the shape; the products are instances of it.
  • "We federated CI to production, so CI is secure." You have made a production credential obtainable by whatever the CI trust policy accepts. If that includes pull requests from forks, you have made production credentials obtainable by anyone (CI Security).

Operating it

Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.

How you know it worked
  • An inventory of long-lived keys exists and is shrinking; ideally it is empty for production workloads.
  • Audit logs show which workload identity performed which action, distinguishably — not one shared identity for a whole environment (Audit Logs for Privileged Actions).
  • Credentials observed in use have short expiry times, verified by inspection rather than by policy documentation.
  • A deliberate test from outside the intended workload — a different branch, a fork, another namespace — is denied by the trust policy.
How you get back
  • Migrating to workload identity is reversible while the old static key still exists, which is exactly why teams leave it in place and never finish the migration.
  • Finish it: verify the new path works under load and during a restart, then delete the key at the issuer. A key that exists is a key that can be used.
  • A broken trust policy is rolled back by restoring the previous policy, which is fast when policies are versioned as code and slow when they were edited in a console (Infrastructure as Code).
What to automate, and what stays human
  • Automate identity issuance as part of workload provisioning, so a new service gets an identity by default rather than by request (Golden Paths).
  • Automate detection of long-lived keys and their age, reported continuously.
  • Automate trust-policy review as policy-as-code: a policy without a workload-specific condition should fail the check (Policy as Code).
  • Do not automate widening a trust policy. Broadening who may assume an identity is the change that most deserves a human and a recorded reason.
What this costs
  • Workload identity ties you closely to the platform's identity model, which is a real portability cost.
  • Short-lived credentials require refresh logic in every client, and libraries vary in how well they handle it — this is where the work actually lands.
  • Trust policies are harder to reason about than a key, and a misconfigured policy is a quieter failure than a leaked key because everything appears to work.
  • It adds a dependency on the identity provider to the startup path of every workload.

Where this applies

This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.

  • GENERALThe three-step primitive — platform attestation, policy evaluation, short-lived credential — is what to learn, and it holds across clouds, orchestrators, CI systems and service meshes. Everything below it is product detail.
  • CLOUD-SPECIFICImplementations differ in ways that break a copied configuration: which claim identifies the workload, whether the trust policy can express conditions on repository, branch, namespace or environment, default credential lifetimes and whether they are adjustable, and how federation from a non-native platform is configured. A trust policy translated between providers without re-reading claim semantics is the specific mistake that leaves a role assumable too broadly (The IAM Model).
  • PLATFORM-SPECIFICOn a platform with no attestation mechanism — bare metal, an unmanaged VM, a developer laptop — there is nothing to attest, and some bootstrap credential is unavoidable. There the goal shifts to making that one credential short-lived, narrowly scoped, monitored and rotated, rather than pretending it can be eliminated.

Where the depth lives

This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.

Domains that do not exist yet
  • Distributed Systems — why an assertion with a validity window makes clock agreement a security property rather than a convenience.