The question this answers
What changes when the decision to run new code and the decision to expose new behaviour stop being the same event?
A large change must reach production in small, low-risk increments, and the moment it becomes visible to users must be reversible in seconds by someone who is not able to run a deploy.
Two independently controllable operations: deployment, which puts code on servers, and release, which decides which users execute which branch of it — with the second reversible without a rollout.
Two operations that were only ever coupled by accident
Historically deploying and releasing were the same event because there was no mechanism to separate them: the code that shipped was the code that ran. A flag inserts a runtime decision — a conditional whose value comes from outside the artifact — and once that exists, the two operations come apart. The new code path can be in production for three weeks, executed by nobody, and then enabled for internal users, then for 1%, then for everyone, by someone changing a value in a console.
The consequences are larger than they first look. A half-finished feature can merge to the main branch behind a disabled flag, which removes the long-lived branch and the merge that goes with it. A rollback stops requiring a deploy, which means it stops requiring the deploy pipeline to be working — during an incident that is a meaningful property. And the person who decides to expose a feature no longer has to be the person who can deploy, which is usually the correct division of labour and is impossible when the two are the same act.
It also changes what a deploy is for. If most deploys carry no user-visible change, deploying often becomes low-risk and boring, which is the healthiest state a delivery pipeline can be in. The risk moves to the flag flip, where it is smaller, faster to reverse, and separately observable.
// The new pricing engine ships and goes live in the same instant.
// If it is wrong: roll back the fleet, wait for the rollout, hope
// the pipeline is healthy. Minutes, and it needs a deploy to work.
export async function quote(order: Order): Promise<Quote> {
return newPricingEngine.quote(order)
}export async function quote(order: Order): Promise<Quote> {
// Evaluated per request against an externally controlled value.
// Deploy on Tuesday, enable for staff on Wednesday, 1% on Thursday.
if (await flags.enabled('pricing-engine-v2', { tenantId: order.tenantId })) {
return newPricingEngine.quote(order)
}
return legacyPricingEngine.quote(order)
}
// Two paths in one artifact means BOTH must work. The old one is not
// dead code while the flag can still be turned off — it is a live
// branch that must keep passing tests and keep being maintained.The decoupled version makes exposure a runtime decision, reversible in seconds without a deploy. The price is that the artifact now contains two live code paths, and every combination of enabled flags is a configuration that could reach production.
What it costs, and the debt nobody budgets for
The first cost is combinatorial. Ten independent boolean flags describe 1024 possible configurations, and production runs some of them. You cannot test 1024 configurations, so in practice you test a handful and hope the interactions are benign. They usually are; the exceptions are memorable, because a bug that only appears when flags A and F are both on is nearly impossible to reproduce from a report.
The second is flag debt. A flag that has been at 100% for eight months is not a feature flag; it is a conditional wrapped around production behaviour, plus a dead branch that still has to compile, still appears in coverage, and still confuses the next reader. Flags need expiry the way branches need deletion: an owner, a removal date, and an alert when they outlive it. The removal is a code change and it will not happen unless someone schedules it.
The third is the one that causes incidents. The flag system is a production dependency on the request path. If flag evaluation requires a network call and that service is slow, your service is slow. If it is unavailable and your client has no cached default, your service fails. Evaluate locally against a cached ruleset, define a default for every flag that is correct when the system is unreachable, and treat the flag service's availability as part of your own. A team that adds flags to reduce deployment risk and then makes every request depend on a third-party API has moved the risk rather than reduced it.
| Kind | Lifetime | Who flips it | Main risk |
|---|---|---|---|
| Release toggle | Days to weeks | The engineering team | Never removed; becomes permanent branching. |
| Experiment / A-B | The length of the experiment | Product or data | Left running after the result is known. |
| Operational kill switch | Permanent by design | On-call | Untested. A kill switch nobody has ever used is a hypothesis. |
| Permission / entitlement | Permanent | The product | Not really a flag — it is authorisation, and belongs in the authorisation model. |
| Circuit-breaker style toggle | Permanent | Automation | Flapping, and hiding a dependency problem instead of surfacing it. |
Where flags fit next to the rollout strategies
Flags and deployment strategies solve adjacent problems and are frequently confused. A rollout strategy manages the risk that *the new build* is broken — a bad artifact, a failing dependency, a memory regression. A flag manages the risk that *the new behaviour* is wrong — a pricing rule that is subtly incorrect, a UI change that hurts conversion. They compose: deploy the artifact with a canary to prove the build is healthy, then flip the flag progressively to prove the behaviour is right.
A flag flip is also faster and cheaper to reverse than any rollout. There is nothing to reschedule, no capacity to provision, no readiness to wait for; the next request takes the other branch. That makes flags the correct control for anything you genuinely expect might need reverting, and it makes them a poor substitute for a rollout strategy, since a flag cannot save you from a build that crashes on startup.
One flag deserves its own mention: the kill switch on a non-critical dependency. Being able to turn off recommendations, or personalisation, or an enrichment call, and keep the core flow serving, is a graceful-degradation control that has nothing to do with releasing features and is worth having for its own sake. Test it, on a schedule, in production — an untested kill switch is a hypothesis, not a control.
1flags:2 - key: pricing-engine-v23 kind: release-toggle4 owner: team-billing # a person answers for this, not "the platform"5 created: 2026-08-046 expires: 2026-10-01 # alerts when it outlives this; removal is a code change7 default: false # what the client uses if the flag service is unreachable8 evaluation: local-cache # never a blocking network call on the request path9 rules:10 - { match: { staff: true }, value: true }11 - { match: { tenantTier: beta }, value: true }12 - { percentage: 5, value: true } # progressive exposure, independent of the deploy13 14 - key: recommendations-enabled15 kind: kill-switch16 owner: team-discovery17 expires: never # permanent by design, and exercised quarterly18 default: true19 note: >20 Turning this off drops the recommendation carousel and keeps checkout21 serving. Tested in production on the first Tuesday of each quarter,22 because a kill switch nobody has pulled is an assumption.Key points
- Deployment puts code on servers; release decides who executes it. Separating them makes exposure reversible in seconds without a rollout.
- It lets unfinished work merge behind a disabled flag, which removes long-lived branches and makes deploys boring.
- The costs are real: a combinatorial configuration space, flag debt, and a flag service that is now on the request path.
- Every flag needs an owner, an expiry and a default that is correct when the flag system is unreachable.
- Flags manage behaviour risk; rollout strategies manage build risk. Use both — a flag cannot save you from a build that crashes on startup.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • A flag definition lives outside the artifact, in a configuration service or a distributed key-value store.
- • The application evaluates the flag per request against a locally cached ruleset, with a context — user, tenant, region.
- • Rulesets are pushed or polled into the client cache, so evaluation is a local operation with no network call in the request path.
- • Changing a rule takes effect within the cache refresh interval, typically seconds, with no deploy.
- • A default value in the client decides behaviour when no ruleset is available, which is what makes the dependency non-fatal.
- • You own the flag inventory: what exists, who owns it, when it expires, and what is still live past its date.
- • You own removal, which is a code change that competes with feature work and loses unless it is scheduled.
- • You own the flag service as a production dependency, including its availability and its cache behaviour.
- • You own the audit trail of flag changes — who enabled what, for whom, when. A flag flip is a production change.
- • You own the exercise schedule for kill switches, because an untested one is not a control.
- • A blocking network call for flag evaluation on the request path: the flag service degrades and takes your latency with it.
- • No safe default, so an unreachable flag service produces an unhandled exception rather than the old behaviour.
- • Flag debt: dozens of permanently-on flags, each with a dead branch that still has to compile and still confuses readers.
- • An interaction bug between two flags, reproducible only in the exact combination the reporting user had.
- • A flag flipped in production by someone who did not realise it was equivalent to a deploy, with no audit trail and no rollback plan.
- • A permission or entitlement implemented as a flag, so authorisation logic lives in a system with no authorisation model of its own.
- • A kill switch that has never been exercised and does not work when it is finally needed.
- • Flag count scales with team size and never decreases on its own; without an expiry mechanism the inventory grows monotonically.
- • The tested fraction of the configuration space shrinks exponentially as flags are added — this is the cost that is easiest to ignore and hardest to reverse.
- • Evaluation cost is negligible when local and unbounded when remote, which is why local evaluation is not an optimisation but a design requirement.
- • Ruleset propagation delay sets how fast a flip takes effect, and it bounds how fast a kill switch can save you.
- • A flag flip is a production change. It needs authentication, authorisation and an audit trail exactly as a deploy does — and it usually has weaker controls, which is the gap.
- • Do not implement authorisation as a feature flag. A flag system evaluates rules; it does not enforce access control, and a client-evaluated flag is trivially observable.
- • Flag context often carries user identifiers, which makes the flag service a personal-data processor and a place tenant identifiers can leak between contexts.
- • A kill switch on an external dependency is a security control too: it is how you cut off a compromised third-party integration in seconds.
- • The service cost is usually per-seat or per-evaluation and modest; self-hosting trades it for operational work.
- • The dominant cost is carrying two code paths: both must work, both must be tested, and one of them is dead weight that nobody is scheduled to remove.
- • The saving is real and hard to price: flags convert a class of incident from "roll the fleet back" into "flip a value", and reduce the time term in error rate times traffic times time.
- • Metrics split by flag state, or you cannot tell whether the new path is the one causing the regression.
- • Flag inventory with age and owner, alerting on anything past its expiry date.
- • Flag change events in the same audit stream as deploys, since both are production changes.
- • Flag service availability and evaluation latency, treated as a dependency of your own service.
- • The signal that lies: overall service health during a 1% flag rollout. The affected population is too small to move an aggregate, exactly as with Canary: Let 5% of Traffic Find the Bug.
- • A rollout strategy alone. For a small team shipping small changes, canary or rolling with a fast rollback covers most of the same risk without a second control plane.
- • Trunk-based development with small changes, which reduces the need for flags by reducing the size of what is being released.
- • Branch by abstraction for large refactors: an interface with two implementations selected at deploy time rather than per request. Less flexible, far less runtime machinery.
- • A configuration value in the existing configuration mechanism, for a single toggle. A flag *platform* is worth adopting when you have dozens, not when you have three.
- • Entitlements in the authorisation model for anything that decides who is allowed to do what, rather than what is currently switched on.
- • Buys second-level reversibility of behaviour without a deploy; costs a runtime dependency on the request path.
- • Buys the ability to merge unfinished work; costs two live code paths and a combinatorial test surface.
- • Buys a division of labour between deploying and releasing; costs a second production control plane that is often less governed than the first.
What people believe, and what is true
Feature flags replace deployment strategies.
They address different risks. A flag cannot help with a build that fails to start, and a canary cannot help with a pricing rule that is subtly wrong.
A flag is a small change, so it needs less care than a deploy.
A flag flip changes production behaviour instantly for the targeted population. It deserves the same authentication, audit trail and rollback thinking a deploy gets.
We will remove the flag after the rollout.
Not unless it is scheduled and owned. Flag debt accumulates because removal is unglamorous work competing with features, and losing.