Artifact Plus Configuration
A running service is an immutable artifact combined with environment-specific configuration — and the configuration half is the one nobody versions, tests or reviews.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
What belongs in the artifact, what belongs in configuration, and why does that line decide so much?
The same bytes must run in five places and behave differently in each, so something has to vary — and whatever varies is the least-tested input in the system, because by definition no environment exercises another environment's values.
Keep a settings file per environment in the repository. config/production.yaml, config/staging.yaml. It is versioned, reviewed and right there next to the code.
It couples configuration changes to a rebuild. Changing one timeout means a new artifact, a full pipeline run, and a redeploy — so people stop doing it through the pipeline and start editing in the console instead.
- It couples configuration changes to a rebuild. Changing one timeout means a new artifact, a full pipeline run, and a redeploy — so people stop doing it through the pipeline and start editing in the console instead.
- It pulls secrets toward the repository, because once configuration lives in files next to code, the database password wants to live there too (What Counts as a Secret, and Where It Must Not Be).
- It makes the artifact environment-specific, which breaks build-once-promote-many: the thing tested in staging is no longer the thing that ships (Build Once, Deploy Many).
- It hides which values are actually in effect. Files, environment variables, command-line flags, platform defaults and a remote store all overlay each other, and the file in the repository may be losing every time.
- Nothing validates it. A typo in a key name is a silent no-op that leaves a default in place, and defaults are usually chosen for development.
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- A running service is artifact + configuration. The artifact is immutable, addressable and identical everywhere. Configuration is everything that legitimately differs between two places running that artifact: endpoints, credentials references, limits, timeouts, feature state, region, log level.
- The line between them is a design decision with consequences. Anything on the artifact side is versioned, reviewed, tested and rolled back with the artifact. Anything on the configuration side gets whatever discipline you deliberately build (A Config Change Is a Production Change).
- Configuration arrives through a precedence chain — built-in defaults, then a file, then environment variables, then command-line flags, then possibly a remote store — and the effective value is the last writer. Most confusion about configuration is confusion about that chain.
- Configuration is also read at different times: some at build, some at process start, some on every request. When a value takes effect is as important as what it is (Build-Time and Runtime Configuration).
- Secret references belong in configuration; secret values do not. Configuration says "the database password lives at this path"; the runtime fetches it under its own identity (Workload Identity).
Where the line goes
Getting this line wrong is what produces environment-specific artifacts on one side and unmanageable sprawl on the other. The test is simple: does this value differ between two places running the same code, and if it changes, must the code change with it?
| Value | Belongs to | Why | Change costs |
|---|---|---|---|
| Business logic constants (tax rules, retry policy shape) | Artifact | Changing them is a code change that needs tests | A build and a deploy |
| Database endpoint | Configuration | Differs per environment by definition | A restart, or a pool refresh |
| Credential value | Neither — a reference only | Values in configuration end up in logs, dumps and repositories | A fetch under workload identity (What Counts as a Secret, and Where It Must Not Be) |
| Timeouts and pool sizes | Configuration | Tuned per environment against real capacity | A restart; a wrong value saturates (The Connection Budget) |
| Log level | Configuration, ideally runtime-changeable | Needs to change during an incident without a deploy | Immediate if dynamic, a restart if not |
| Feature state | Configuration, runtime | Decoupling release from deploy is the whole point (Feature Flags: Deploy Is Not Release) | Immediate, which is also the risk |
| Region and zone identity | Configuration, from the platform | The platform knows; hardcoding it breaks failover (Region Failover) | A restart in a new location |
| Public asset base URL for a browser bundle | Artifact, at build time | It is compiled into the bundle whether you like it or not | A rebuild (Build-Time and Runtime Configuration) |
The precedence chain, and why a change appears to do nothing
Almost every "I changed the config and nothing happened" is a precedence question. The effective value comes from the last layer that set it, and the layer someone edited may not be that one.
Make the chain explicit, in one place, and make the effective-value dump report which layer each value came from. That single feature removes an entire category of wasted incident time.
What a good configuration surface looks like
The shape below does three things at once: it declares the schema, it fails at startup rather than at first use, and it produces a redacted dump an operator can read during an incident.
1const schema = {2 DATABASE_URL: { type: 'url', required: true, secretRef: true },3 DB_POOL_MAX: { type: 'int', required: true, min: 1, max: 200 },4 PAYMENTS_TIMEOUT_MS: { type: 'int', required: true, min: 50, max: 30_000 },5 LOG_LEVEL: { type: 'enum', values: ['debug', 'info', 'warn', 'error'], default: 'info' },6 ALLOW_INSECURE_TLS: { type: 'bool', default: false, forbidIn: ['production'] },7} as const8 9// Resolved once, at startup, before the server binds a port.10const cfg = loadAndValidate(schema, process.env) // throws, listing every failure11 12// Operator endpoint: what is this process actually running with?13app.get('/debug/config', requireOperator, () => redact(cfg, schema))Four properties matter here and none of them are about the library. Constraints are ranges rather than presence checks, so a pool size of 5000 is rejected rather than accepted and then exhausted. forbidIn encodes that some values must never be true in production. secretRef marks what the dump must redact. And validation happens before the port is bound, so an invalid process never receives traffic (Probes: Readiness, Liveness and Startup).
How to do it properly
Most important first.
- Define the configuration schema once, in code, with types, required-ness and constraints — and derive every environment's values from that one schema (Validate at Startup, Fail Clearly).
- Supply values from outside the artifact: environment variables and mounted files for most systems, a remote store where dynamic change is genuinely needed.
- Keep secret values out entirely. Configuration holds a reference; the process resolves it at startup under its workload identity (Secret Managers and What They Actually Give You).
- Make defaults safe for production rather than convenient for development. A missing value should not silently enable debug logging or disable TLS verification.
- Expose the effective configuration — with secrets redacted — on an operator endpoint, so "what is this process actually running with" is answerable in seconds (Dashboards an Operator Can Act On).
- Version configuration and record which version was in effect with which artifact, so an incident can correlate against both halves (The Release Manifest).
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
Frequently nothing. Configuration is commonly applied to every instance at once, without a canary and without staged rollout, so a bad value reaches all traffic in one step — the honest answer for most teams is that nothing contains it (A Config Change Is a Production Change).
What can go wrong
- A key that exists in one environment only, so a code path is exercised nowhere else (Configuration Drift).
- A typo in a variable name that produces no error, leaves a development default in place, and is found weeks later.
- A precedence chain nobody understands, so a change is made in the wrong layer and appears to do nothing.
- Configuration values logged at startup with the credentials included, which puts secrets into every log sink and every log-forwarding vendor (Secrets in Logs).
- A remote configuration store that becomes a hard startup dependency, so its outage prevents every service from starting — including the ones needed to fix it.
- Types lost in transit: every environment variable is a string, so
false,0andnobecome truthy in a language that does not check.
- "Configuration is not code, so it does not need review." Configuration decides behaviour exactly as code does, and it reaches production faster with fewer checks (A Config Change Is a Production Change).
- "Twelve-factor says use environment variables, so everything is an environment variable." Environment variables are a good default for flat, small, startup-time values. They are poor for structured data, poor for values that must change at runtime, and they leak through process listings and crash dumps (The Anatomy of a Process).
- "If a value is missing it will use a sensible default." It will use the default the author chose, usually for their laptop. That is where debug endpoints and permissive CORS get into production.
- "Config in Git means config is safe." It means it is versioned. It also means every developer, every CI job and every fork of the repository can read whatever ended up in it (Secrets Are Not Configuration).
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- You can print the effective configuration of a running process, with secrets redacted, and it matches what you believe is deployed.
- A missing or invalid required value stops the process at startup with a message naming the key (Validate at Startup, Fail Loudly).
- The same artifact digest runs in every environment, and only configuration differs (Promotion Between Environments).
- The last incident review could name the configuration version in effect, not just the artifact version.
- Reverting the artifact does not revert configuration. If both changed, reverting one leaves a combination that has run nowhere — this is a common way a rollback makes things worse (Rollback: Only Useful If It Is Actually Safe).
- Configuration held as versioned data can be reverted like code, provided you know which version was in effect. Configuration edited in a console usually cannot, because the previous value was not recorded.
- Values read once at startup need a restart to revert, so the rollback of a config change is a rolling restart with all the disruption that implies (Graceful Shutdown).
- Automate schema validation, type coercion and the effective-configuration dump. All of it is mechanical.
- Automate the diff of configuration keys between environments and attach it to promotion (Environment Drift).
- Do not automate the decision about which layer a value belongs in. Whether something is a build constant, a startup value or a runtime-tunable is a design judgement with a rollback story attached.
- Externalised configuration is more flexible and less visible — the repository no longer tells you what production is running.
- A remote store enables change without redeploy and introduces a runtime dependency on the store's availability.
- A strict schema catches errors early and adds friction to adding a key, which pushes people toward reusing existing keys for new meanings.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- GENERALThe artifact/configuration split holds on every platform. What differs is the mechanism: environment variables and mounted files on containers, parameter stores on managed platforms, and process arguments or unit files on virtual machines — with different size limits, different type handling and different visibility to other processes on the host.
- PLATFORM-SPECIFICServerless platforms typically expose configuration only as environment variables set at deploy time, so "change config without redeploy" is not available without an external store. Container platforms can mount configuration as files that update in place, which changes the rollback story from redeploy to file write (ConfigMap vs Secret — and the Honest Limit of a Secret).
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
- — Testing & Reliability Engineering — testing a configuration schema itself, so an invalid combination fails a build rather than a deployment.