Logs, metrics, traces: what does each answer that the others cannot?
“Explain the difference between logs, metrics and traces, and for a "checkout is slow" incident, which one you would open first and why.”
What this tests
- The three signals as answers to three different questions
- Cardinality and cost: why metrics are cheap and logs are not
- Correlation ids and trace ids as the glue
- Alerting on symptoms with metrics, then diagnosing with traces and logs
Answers by level
Read the beginner answer first and notice what is missing.
Metrics answer "how much, how often": counters, gauges and histograms aggregated over time — request rate, error rate, p99 latency (the RED signals per endpoint). Cheap to store, good for alerting and dashboards, but they lose the individual request. Logs answer "what happened": discrete events with context, structured as JSON so they can be queried, expensive at volume and useless without a correlation id. Traces answer "where did this request spend its time": one request as a tree of spans across services with durations, which is the only signal that shows a 140 ms database call hiding behind a 180 ms service call.
For "checkout is slow": metrics first, to see when it started and whether it is all requests or p99 only (a histogram tells you that). Then a trace of a slow request to see which hop owns the time. Then logs from that span, using the trace id, to see why. Grepping logs first is slow and reveals nothing about latency distribution — see Logs, Metrics and Traces.
Green flags · Red flags
- Maps each signal to its question in one sentence
- Understands cardinality limits on metric labels
- Knows p99 needs histograms, not averaged percentiles
- Uses the trace id to move from metric to trace to logs
- Alerts on symptoms, diagnoses with traces and logs
- "We log everything, so we have observability."
- Puts user ids in metric labels
- Averages p99 across instances
- Would start a latency investigation by grepping logs