The autoscaler is working perfectly and the database is dying
An API scales on CPU at a 70% target. During the morning peak, p99 latency goes from 120ms to 4 seconds, database connections are exhausted, and the autoscaler adds nothing because CPU never exceeds 45%. An engineer lowers the CPU target to 40%. The fleet doubles, and the outage gets worse.
The fix that looks right
Increasing the connection pool size per instance to stop the pool exhaustion errors. The errors disappear immediately, which is enormously convincing. What has happened is that the queue that was visible in the application — bounded, measurable, with a clear error — has been moved into the database, where it appears as lock contention and context switching that degrades every client of that database, including the ones that were healthy. The pool is a bulkhead; enlarging it removes the bulkhead. The corresponding trap on the other side is raising the server's connection limit, which trades a connection error for a memory exhaustion event on the database host.