API Gateway
One edge component that authenticates, rate-limits, routes, logs and reshapes every inbound request so the services behind it do not each reinvent those concerns — valuable exactly as long as it stays an edge and does not absorb the business logic, aggregation and orchestration that turn it back into the monolith.
With a dozen services, every one of them would otherwise implement TLS, token validation, rate limiting, request logging and CORS — differently, and with different bugs. A gateway does the edge concerns once, in one place, and gives clients one stable address while the services behind it are split, merged and renamed.
The pipeline at the edge
A gateway is a reverse proxy with a fixed pipeline. Authentication: validate the JWT or session, reject early, forward the verified identity as a header so services never parse tokens. Rate limiting: per client, per token, per route — a token bucket in Redis, 429 with Retry-After when exceeded (Rate Limiting). Routing: /orders/* to the Order Service, /users/* to User; path rewriting; canary weights (5% of traffic to the new version). Logging and metrics: one access log line per request with a request id, latency and status, which becomes the RED metrics for the whole edge (Logs, Metrics and Traces). Transformation: protocol translation (REST in, gRPC out), header normalisation, response compression, and stripping internal fields from responses.
Each stage is stateless or backed by a shared store, which is what lets the gateway itself scale horizontally. The whole pipeline should add single-digit milliseconds; a gateway that adds 40 ms is doing something that is not an edge concern.
What belongs in it, and what must not
The test for any candidate feature: does it need to know what the request means? TLS, auth, limits, routing, logging, compression and CORS work on *any* request without understanding orders or users. Those are edge concerns and they belong in the gateway. The moment a rule needs the domain — "premium customers get 3 free returns", "an order over €1,000 needs a fraud check first" — it is business logic, and putting it in the gateway means the rule lives outside the service that owns the data, in a component every team deploys through.
Three things creep in and each is a step toward the gateway becoming the monolith. Business rules (above). Orchestration: the gateway calls Inventory, then Payment, then Shipping in sequence with compensation on failure — that is a saga (Saga Pattern) and it belongs in a service that owns the workflow, not in the proxy. Data aggregation that hides service boundaries: /checkout-page on the gateway calls five services and merges their responses, so the gateway now depends on five schemas, every service change touches it, and its latency is the sum of the slowest paths. The result is a component that every team must change for every feature, deployed by nobody in particular, with a p99 that doubles as it accretes logic — the exact scenario in the challenge gateway-became-the-monolith.
The legitimate home for client-specific aggregation is a Backend for Frontend (BFF): a thin service per client type — one for the web app, one for the mobile app, one for partners — owned by that client’s team, which calls the internal services and composes the shape that client needs. A BFF is allowed to know the domain because it is an application, not infrastructure; it is deployed by the team that changes it; and there can be several, so no one of them becomes the bottleneck. The gateway sits in front of the BFFs and stays dumb.
| Concern | Gateway | BFF | Service |
|---|---|---|---|
| TLS, auth, rate limits, CORS, access logs | Yes | No | No (trust the header) |
| Routing, canary weights, protocol translation | Yes | No | No |
| Compose one screen from 4 services | No | Yes | No |
| Business rules (pricing, eligibility) | Never | No | Yes |
| Orchestrate a multi-step workflow | Never | No | Yes (saga owner) |
| Cache a public GET | Yes (by URL + Vary) | Sometimes | Rarely |
The single point of failure, and how it is not one
Every request goes through the gateway, so a dead gateway is a dead product. It is made highly available the same way any stateless tier is: several instances behind a network load balancer or DNS with health checks; no local state — rate-limit counters and auth caches live in Redis or are tolerated as per-instance approximations; rolling deploys with connection draining so a restart drops no requests; and multi-zone placement so a zone outage takes out one third of capacity, not all of it. Managed gateways (cloud API gateways, a CDN’s edge workers) push this further by running at the provider’s edge in dozens of regions.
Its dependencies are the real risk. If the gateway validates every token against an auth service synchronously, the auth service’s outage is now a total outage; cache validated tokens locally with a short TTL and verify JWT signatures offline. If Redis for rate limiting is down, fail open (allow the request, log the miss) rather than fail closed — a rate limiter that blocks all traffic when its store is unreachable has inverted its purpose. And put timeouts on every upstream call; a gateway with no timeouts turns one slow service into an exhausted connection pool for everyone (Reliability Patterns).
1async function rateLimit(req: Req, next: () => Promise<Res>): Promise<Res> {2 const key = `rl:${req.clientId}:${Math.floor(Date.now() / 1000)}`3 try {4 const count = await redis.multi().incr(key).expire(key, 2).exec() // fixed 1 s window5 if (Number(count[0]) > LIMIT_PER_SEC) return tooManyRequests({ retryAfterSec: 1 })6 } catch (err) {7 metrics.increment('gateway.ratelimit.store_error') // fail open: an outage of Redis must not be an outage of us8 }9 return next()10}Key points
- A gateway does the edge concerns once: TLS, auth, rate limiting, routing, logging, transformation. Single-digit milliseconds added.
- Test for admission: does the feature need to know what the request means? If yes, it is application logic and does not belong here.
- Business rules, orchestration and cross-service aggregation in the gateway turn it into a monolith every team deploys through.
- Client-specific composition belongs in a BFF per client type, owned by that client’s team.
- High availability is the stateless recipe: N instances, shared or approximate counters, draining deploys, multi-zone — and fail open when Redis is down.
One request through the gateway
How data moves through it
One request or event, hop by hop.
- 1Client → Gateway: TLS terminated; the raw request is parsed and given a request id.
- 2Gateway → Auth stage: the JWT signature is verified offline and the identity attached as
X-User-Id; invalid tokens get401here. - 3Gateway → Redis: the rate-limit counter for this client is incremented with a TTL; over the limit returns
429. - 4Gateway → Service: the route matches
/orders/*, the request is forwarded with the identity header and a deadline; the access log line is written on response. - 5Service → Gateway → Client: the response is compressed, internal headers stripped, and returned with the request id for correlation.
When to use — and when not
- More than a handful of services that would otherwise each implement auth, limits and logging.
- Public or partner APIs that need one stable hostname while the services behind it change.
- Protocol translation at the edge: REST or GraphQL for clients, gRPC inside.
- A single Monolithic Architecture — a reverse proxy or the framework’s middleware already does everything a gateway would.
- Purely internal service-to-service traffic; a service mesh or client libraries handle that, and a gateway hop adds latency for nothing.
- As the place for orchestration or aggregation; that is a BFF or a service, not an edge.
Tradeoffs
One more tier to run and one extra hop, in exchange for edge concerns implemented once. Its dependencies (auth service, Redis) decide whether it is a resilience gain or a new outage source.
How it fails
- The gateway accretes aggregation and rules; every feature touches it, deploys queue behind it, and its p99 doubles from 8 ms to 16 ms while nobody owns the code.
- Synchronous token validation against an auth service; an auth outage becomes a total outage.
- Rate limiter fails closed when Redis is unreachable and blocks 100% of traffic.
- No upstream timeouts; one slow Payment Service holds every gateway connection and unrelated routes start returning 503.
- Local rate-limit counters on N instances behind round robin let every client exceed the limit N-fold.
How it scales
- Horizontally, as a stateless tier: add instances behind an L4 balancer; shared counters in Redis or accept per-instance approximation.
- The bottleneck is usually per-request work that is not an edge concern (body transformation, aggregation); keeping the pipeline thin keeps throughput at tens of thousands of requests per second per instance.
- Rate-limit and auth-cache traffic to Redis scales with request rate; batch or sample the counters if Redis becomes the hot spot (Consistent Hashing for sharding it).
- Managed edge gateways move the tier into the provider’s network for global scale and DDoS absorption.
How it interacts with databases, queues, caches, APIs and external systems
- Cache: Redis for rate-limit counters and short-TTL token validation results; optionally a response cache for public GETs.
- Services: forwarded to over HTTP/1.1, HTTP/2 or gRPC with timeouts and retries only on idempotent methods.
- Auth provider: consulted on cache miss or for revocation lists; never on the hot path for every request.
- Observability: emits one access log per request, RED metrics per route, and starts the trace with a
traceparentheader (Distributed Tracing). - External clients: the only surface partners see; versioning and deprecation headers are enforced here.