Performance Signals
Green flags show measurement-first reasoning. Red flags reveal fixes chosen before diagnoses, and comfort taken from numbers that do not mean what they appear to.
Green flags
Establishes a baseline before changing anything.
Without a baseline there is no way to tell improvement from noise, or from the traffic dropping because it is Sunday.
Starts from the user-visible symptom, not from a resource dashboard.
Dashboards show what is easy to measure; the symptom tells you what actually matters. Starting from CPU means finding CPU explanations.
States a falsifiable hypothesis before opening the tool.
Writing down "if it is the pool, I expect DB CPU to be flat while pool wait is high" makes you able to be wrong quickly instead of confirming a story.
Distinguishes average from tail behaviour without being asked.
p50 and p99 fail for different reasons; a candidate who only quotes averages will optimize the wrong half of the system.
Recognizes queueing as the reason systems slow down before they fail.
The knee in the utilisation curve explains most "it was fine yesterday" incidents, and nothing else explains them as well.
Separates utilisation from saturation.
A resource at 80% with nothing waiting is healthy; at 80% with a deep queue it is the bottleneck. Only one of those readings predicts latency.
Finds the critical path before optimizing anything.
Work parallel to the critical path can be made infinitely fast without moving user latency by a millisecond.
Asks whether the system is waiting or working before choosing a tool.
CPU-bound and I/O-bound problems have disjoint fix sets; getting this backwards wastes the whole investigation.
Refuses to accept correlation as root cause.
Two metrics rising together usually share a cause. Acting on the correlation fixes nothing and leaves everyone confident it is handled.
Measures again after the change, against the metric that motivated it.
An optimization that was never validated is a belief. Half of them turn out to have moved nothing users can feel.
Leaves behind an alert, an SLO or a CI benchmark.
Performance regresses silently by default; without a guard the same incident returns in two quarters with a new cast.
Expects the bottleneck to move and predicts where.
Relieving one constraint always exposes the next. Predicting it in advance is what separates a plan from a whack-a-mole loop.
Weighs the cost of the fix, not just the latency win.
Faster is not free; tripling spend to shave 20 ms is a decision someone should make deliberately rather than discover on the invoice.
States what a number depends on: workload, runtime, hardware, engine.
Performance claims without conditions are folklore. "It is 3× faster" means nothing without the workload it was measured on.
Red flags
'It's slow — let's add Redis.'
A fix chosen before a diagnosis. If the bottleneck is a lock, a pool or a third party, a cache adds a moving part and changes nothing.
'Just add more servers.'
Works only if the resource you are adding is the constrained one. Against a saturated pool, a hot key or someone else's API it buys a bigger bill.
'CPU is below 100%, so CPU is fine.'
Utilisation is an average over an interval and says nothing about queueing. A resource at 70% with a deep run queue is already hurting users.
'Average latency is good, so performance is good.'
The mean is dominated by the majority; the complaints come from the tail. Averages have ended more investigations than they have started.
'The database is slow — add an index.'
An index is a fix for a specific plan problem. Add one without reading the plan and you may slow writes, bloat storage and leave the actual cause untouched.
'The benchmark says it is faster, ship it.'
Benchmarks measure what they measure. Without warm-up, realistic payloads, variance and a matching environment, the number describes the benchmark, not the system.
'More threads means more throughput.'
Past the point where a resource saturates, extra threads add context switching and queueing, and usually make latency worse while throughput stays flat.
'Retries improve reliability.'
Without backoff, jitter and a budget, retries multiply load exactly when the dependency is least able to take it — turning a slowdown into an outage.
'A 95% cache hit rate is good.'
It depends entirely on what the 5% costs and which keys they are. A 95% hit rate where every miss is a 2-second query is a system waiting to fall over.
'Tracing everything fixes observability.'
Unsampled tracing at scale is unaffordable, and naive sampling drops exactly the slow requests you needed. Sampling strategy *is* the design.
'Add user_id as a label so we can slice by customer.'
Cardinality is multiplicative and unbounded. That label can take down the metrics backend for everyone; per-user detail belongs in traces and logs.
'It times out — raise the timeout.'
That converts fast failures into slow ones and holds capacity longer, so a dependency slowdown becomes your outage. Timeouts are capacity decisions.
'This function is 40% faster now.'
A real improvement to something that may not be on the critical path. Without an end-to-end measurement it is an unvalidated claim about user experience.
'It is fast on my machine.'
Local has warm caches, one user, no network, a tiny dataset and a fast laptop. Every condition that makes production slow is absent by construction.