Architecture Cheat Sheet

“Problem says X → think Y.” The sentence an engineer says in a design review, and the mechanism it should trigger. Click a row to open the lesson.

Structure

Small team, one product, domain still changing->Monolith with a layered structure; scale as copies behind an LB We can name the subdomains and who owns each->Modular monolith: module public APIs, private tables, in-process events Teams block each other on deploys, weekly->Independent deployables — only where data can be owned outright One module needs 20× the CPU of the rest->Extract that one service behind its existing module boundary Nine services, one database, one release train->Distributed monolith — consolidate or finish the split; do not add a tenth Business logic in controllers, SQL in the domain->Dependency direction: presentation → application → domain; infrastructure behind interfaces Cannot unit-test the domain without a database->Ports and adapters: swap the database adapter for an in-memory one Five layers of interfaces for a CRUD screen->Drop the abstraction; Clean Architecture pays off only where boundaries matter

Communication

Two services must both succeed or neither->Saga with compensation (not 2PC)Caller needs the answer to respond->Synchronous call with a timeout shorter than your own Work can happen later and is retryable->Queue + workers; idempotent handler; DLQ Several systems care that an order was created->Publish `OrderCreated` to a topic; each consumer group subscribes User needs "accepted" now, the rest can follow->Sync command returning 202 + status; async consequences with a `pending` state Events for the same order arrive out of order->Partition key = order id; ordering holds per partition only Consumer must reprocess last week->Log-based broker with retention; rewind the offset State written, event never published (crash in between)->Outbox table in the same transaction + relay

APIs

Public API, cacheable reads, versions that live for years->REST with ETags and an `Idempotency-Key` on writes Every screen needs a different shape; five round trips per page->GraphQL with batched resolvers, or a BFF per client Hot internal path, thousands of calls/s, typed contracts->gRPC with deadlines that propagate Server must push to the browser, one way->Server-Sent Events with `Last-Event-ID` resume Chat, collaboration, bidirectional and low-latency->WebSockets + a pub/sub backplane to reach sockets on other nodes A partner must learn about our events->Webhooks: signed, retried with backoff, event id for dedup Auth, rate limits and routing repeated in every service->API gateway for edge concerns only Every team's deploy touches the gateway->Business logic and aggregation leaked into the edge — move them back

Scaling

One server at 90% CPU->Vertical first (4 → 32 CPU); horizontal when the ceiling or the failure domain bites Traffic exceeds one instance->Load balancer + N stateless instances + health checks Users logged out randomly after autoscaling->Sessions in process memory — externalise to Redis Same user should hit the same instance for cache affinity->Consistent-hashing LB, not sticky sessions Database reads dominate and the primary is saturated->Cache in front, then read replicas; writes still go to one primary Writes exceed one primary->Partition by a key you always have; accept cross-shard queries get hard Users in Europe see 300 ms for static assets->CDN edge; versioned URLs; origin shield

Caching

Same keys read 1,000× more than written->Cache-aside in Redis with a TTL; invalidate on write All keys expire at midnight, thousands of identical recomputes->Jittered TTLs; lock or early recompute on miss One hot key pins one cache shard->Replicate the hot key, add an in-process layer, or split the key Repeated lookups for ids that do not exist->Negative caching with a short TTL Browser refetches unchanged resources->`Cache-Control` + ETag conditional requests Cache restart takes the database down->Warm-up, request coalescing, and a database sized for a cold cache

Async

One client retries and we charge twice->Idempotency key stored with the response Message delivered twice after a rebalance->Processed-message table or upsert; at-least-once + idempotency is the honest exactly-once Video transcoding in the request path->Job queue, worker pool, status endpoint or push for progress A job that fails is retried instantly forever->Exponential backoff + jitter, max attempts, dead-letter queue Producer 100k/s, consumer 20k/s->Backpressure: bound the buffer, shed or slow the producer, scale consumers Queue depth is flat but the oldest message is 3 hours old->Alert on oldest-message age, not depth; Little's law

Data consistency

One transaction across three services->It does not exist — saga, outbox, compensation Payment failed after inventory was reserved->Compensating step: release inventory, cancel order; semantic undo, not rollback Confirmation email cannot be un-sent->Order saga steps so irreversible actions come last Reads need a shape the write model cannot give without twelve joins->CQRS: projected read model; start with CQRS-lite UI reads the read model right after a command and shows nothing->Return the write-side state or show `pending`; projection lag is real "How did the balance get here" must be answered exactly->Event sourcing: append events, replay state, snapshot

Reliability

p99 fine, p50 fine, one dependency slow takes everything down->Bulkhead + timeout per dependency Retries at three layers finished off a slow provider->Retry budget, jitter, retries only at one layer; 3 × 3 × 3 = 27×Dependency is down; every call waits for its timeout->Circuit breaker: fail fast when open, probe when half-open One tenant sends 10× everyone else->Token bucket per tenant at the gateway; 429 + `Retry-After`Rate limit lets 2× through at the window boundary->Sliding window counter or token bucket, not fixed window "We need 99.99%"->52 minutes per year; serial dependencies multiply — check the arithmetic first Releases and reliability fight every week->Error budget: ship while the budget lasts, freeze when it is spent

Distributed

Adding a cache node remaps almost every key->Consistent hashing: only K/N keys move One node owns 45% of the keyspace->Virtual nodes (100–200 per physical node)Traffic sent to instances the autoscaler killed->Registry with short heartbeat TTL; deregister on shutdown During a partition, should we accept writes?->Choose per operation: refuse (consistent) or accept and reconcile (available)Read must see the latest write across replicas->Quorum: W + R > N A timeout — is the node dead or slow?->You cannot know; a timeout is a partition. Design for both

Observability

Where did the 800 ms go across five services?->Distributed trace with propagated `traceparent`One order call → 38 sequential database spans->N+1 across a service boundary; batch or join Alert fired on CPU, users were fine->Alert on symptoms (SLO burn), not causes Average latency looks fine, users complain->Histograms and p99; averages hide the tail Metrics bill exploded after adding `user_id` as a label->Cardinality: ids belong in logs and traces, not metric labels Cannot find the log lines for one request->Correlation id in every structured log line