advanced
Every Forty Seconds
Read the evidence before you read the options. The signals are presented the way a dashboard would present them — nothing is labelled with the answer.
The report
A trading dashboard service has "latency spikes". Users describe it as the page freezing briefly and then catching up. It has been happening since a release ten days ago that added a new market-data enrichment step. Average latency looks fine and nobody can reproduce it on demand.
Market data service — one hourILLUSTRATIVE
| Signal | Value | What it tells you |
|---|---|---|
| p50 latency | 32 ms | The typical request is fast. |
| p99 latency | 1.4 s (was 190 ms before the release) | The tail is seven times worse than before the release. |
| Slow request distribution | bursts every 38–45 s | Slow requests arrive in clusters at a regular interval rather than randomly. |
| Affected instances during a burst | one instance at a time | Each burst is confined to a single instance, and different instances take turns. |
| CPU | sawtooth 45% → 92% → 45% | CPU rises and falls on a regular cycle. |
| Heap after collection | 1.1 GB, stable across cycles | The live set returns to the same level after each collection. |
| Allocation rate | 1.4 GB/s (was 180 MB/s before the release) | The service allocates roughly eight times as much memory per second as before. |
| Time in collector | 14% of wall clock | One seventh of available time is spent on memory management. |
Allocation profile, 60 s sample (bytes allocated, self)
71.2% marketdata/enrich.decorateQuote
| └─ 68.9% encoding/json.Unmarshal → map[string]interface{}
11.4% net/http.readRequest
6.8% marketdata/serialize.Response
4.1% database/sql.(*Rows).Scan
6.5% (everything else)
allocations/sec: 1.41 GB live set after GC: 1.10 GB (stable)What is the constraint?