IntermediateCloud Networking← All labs

The API Cannot Reach the Database

The report, in their words

The checkout API started returning 500s at 09:14. Every request logs connect ETIMEDOUT 10.0.3.41:5432 after exactly 30 seconds. Nothing was deployed this morning — the last release was Thursday. The database console shows the instance available, CPU near idle, and zero connections from the API.

The infrastructure under investigation

Exposure is shown, because half of these failures are a reachability question.

The path a query takes, and everything on it that can drop a packet.PROVIDER-NEUTRAL

Customerspublic— checkout traffic over 443

Region

VPC 10.0.0.0/16

Public subnet 10.0.1.0/24public

Load balancerpublic— the public entry point is the design, not the finding

NAT gatewaypublic— outbound only

App subnet 10.0.2.0/24private

Checkout API (2 instances)private

Bastion hostinternal— the control that proves the path works

Data subnet 10.0.3.0/24private

Database security groupinternal— inbound 5432 from one referenced group

PostgreSQL 10.0.3.41private

Private DNS zoneinternal

Customers→Load balancer· 443crosses boundary

Load balancer→Checkout API (2 instances)· 8080

Checkout API (2 instances)→Private DNS zone· resolve db.internal

Checkout API (2 instances)→Database security group· 5432 — hangs, then times out

Bastion host→Database security group· 5432 — connects in 4ms

Database security group→PostgreSQL 10.0.3.41

Checkout API (2 instances)→NAT gateway· outbound

Pull evidence

One item at a time, and nothing here tells you which one matters. Deciding what is worth looking at is most of the diagnosis.

Does the connection hang, or is it refused?

Can anything else in the same subnet reach the database?

Inbound rules on the database security group

Name resolution from an API instance

Route table for the app subnet

Network ACLs on the app and data subnets

Database credentials in the application config

Connection pool configuration and behaviour

0 of 8 inspected. You are not required to open all of them — a real investigation is judged on how few you needed.

What is your diagnosis?

Commit to one. Nothing below is shown until you do.

Guessing wrong and being told exactly why is the point of this page. Reading the answer first is not practice.