The Container Registry

A registry looks like storage and behaves like a control point. On one side, a small number of build identities write; on the other, every node in the fleet reads, continuously, including during incidents when new capacity is being created. That asymmetry drives the design: writes are rare, privileged and auditable; reads are constant, high-volume and must not fail.

Which registry you use matters less than where it sits. A registry inside the provider network is reached over private connectivity and does not bill through the NAT meter; a public registry is reached over the internet, bills on every uncached pull, and imposes rate limits that surface as a stuck rollout on exactly the day you scale out hardest. The common production pattern is a private registry as the source of truth, with third-party images mirrored into it rather than pulled from upstream at deploy time — which also removes an external dependency from your critical path.

The registry is therefore a reliability dependency of the same rank as your load balancer. If it is unavailable, running containers keep running, but no new capacity can start and no deploy can proceed. That is a survivable outage right up until it coincides with a traffic spike.

A tag is a mutable label attached to a manifest — a pointer, like a git branch. A digest is the SHA-256 of the manifest itself, so app@sha256:6b1f… names one specific set of bytes and can never name anything else. Everything about verifiable deployment follows from that distinction.

The classic finding is image: app:latest in a production manifest. It reads as "the current version" and means "whatever that pointer happens to reference at the moment each node pulls". During a rollout across a hundred nodes, if a build pushes mid-rollout, some nodes get one artifact and some get another, under one name. Rollback is undefined, because the previous bytes no longer have a name. And an incident timeline cannot state what was running, because the evidence was overwritten.

The fix has three parts and none of them is difficult. Tag immutably — the commit SHA or a semantic version — so a tag is never reused. Turn on registry-side tag immutability so a reused tag is rejected rather than silently accepted. Deploy by digest, recording it in the pipeline (The Container Build Pipeline). A human-readable tag is still useful; it just is not the thing the deployment references.

Registry permissions are usually granted once, broadly, and never revisited, which is how a single leaked token ends up able to overwrite every service's images. The useful decomposition is three identities with three different jobs: the build identity pushes new tags to one repository; the node identity pulls, and nothing else; the human identity reads metadata and, rarely and deliberately, deletes.

The pull side is the one people forget to narrow, because pull feels harmless. It is not, in two directions. A node identity with broad pull access can fetch every team's images, and images routinely contain enough application detail to plan an attack. And a node identity with *push* access — granted by accident, or by attaching one over-broad role to the whole node pool — means anything that compromises a single container can replace the images the entire fleet runs.

This is the same argument as Least Privilege in Infrastructure and connects directly to The Infrastructure Supply Chain: the registry is the last checkpoint before code executes in production. Signing artifacts and verifying signatures at admission is what turns "we trust the registry" into something a control can actually enforce.