What Should Autoscaling Scale On?
You are configuring autoscaling for a web service. What signal do you scale on, and where do you set the threshold?
Practical scenario
An API scales on CPU with a target of 85%. Under morning load its p99 latency hits four seconds while CPU stays at 40%, because every request waits on a saturated database connection pool. Instances take 100 seconds to become healthy.
What it tests
Answer it out loud first. The guide below stays closed until you ask for it, on purpose.
- Whether the candidate picks a signal that reflects the actual bottleneck
- Whether they leave headroom for the time capacity takes to arrive
- Whether they understand flapping, cooldowns and the limits of scaling out
Reading a strong answer is not the same as producing one. Commit to what you would say, then compare.