Application ArchitecturedesignExpert
Nine services, one database, one deploy train
Symptoms
- Releases are a four-hour coordinated event every second Tuesday in which all nine services deploy together; the last one that shipped a single service alone (November) broke pricing.
- A column rename on
customersbroke four services at once; the migration was rolled back and rescheduled for the next train. - Any one service being down returns 503 for checkout; last quarter’s availability was 98.7% against a 99.9% target.
- Checkout p99 is 1.9 s across 11 synchronous hops; a developer needs all nine services running to test a change to any of them.
# one checkout request (all hops synchronous and required)
client → gw → checkout-svc → customer-svc ─────────────────────────► shop.customers
→ cart-svc ──────────────────────────────► shop.carts
→ pricing-svc ─────────────► shop.products, shop.customers ← reads customer tier directly
→ inventory-svc ─────────────────────────► shop.stock
→ payment-svc ───────────────────────────► shop.payments, shop.orders ← writes the orders table
→ order-svc ─────────────────────────────► shop.orders
→ notification-svc → email provider (sync, 400 ms)
# database 'shop': one Postgres · 63 tables · 9 services connect as the same role 'app'
# tables written by ≥ 2 services: orders (3), customers (2) · read by ≥ 3 services: customers (7), products (6), orders (5)
# shared package shop-common@3.41: DTOs mirroring table rows; every service pins the exact version
# availability arithmetic: 11 required hops at 99.9 % each → 0.999^11 ≈ 98.9 %
# release train v3.41: 9 services, 4 h 10 min, 1 rollback · v3.40: 3 h 55 minInvestigate
Inspect areas in any order (0/7 inspected). When you think you know the root cause, commit to it.
Infrastructure cost and cluster health
Database ownership
Communication style
The shared shop-common package
Team structure
Network latency
Test suite