API Gateways: Routing, Auth Offloading, Rate Limiting & Trace Initiation
An API Gateway is not a reverse proxy — it is the system's single policy enforcement point for routing, auth offloading, rate limiting, circuit breaking, and trace initiation. Teams that skip it re-implement all of these incorrectly in every downstream service. This article builds a complete gateway understanding from routing mechanics through distributed rate limiting to OpenTelemetry root span injection.
API Architecture & System Resilience
API Gateways: Routing, Auth Offloading, Rate Limiting & Trace Initiation
Senior engineers don't just wire services together — they design the boundary: the contract, the trust model, the failure envelope, and the signal pipeline that proves it's working. The API Gateway is that boundary made concrete. Most teams discover they need one after they have already implemented JWT validation in seven services (each slightly differently), added rate limiting middleware to five of them (with incompatible counters), and tried to correlate an incident across logs from twelve services with no shared request identifier. By then, the gateway is a refactor, not a design. This article builds it correctly — from first principles.
Series positioning: This is Part 2 of the API Architecture & System Resilience series. It builds on the RESTful design vocabulary from Part 1 and establishes the enforcement layer that all subsequent articles depend on — Part 3 (Service Mesh), Part 5 (Auth Architecture), and Part 6 (Observability) all assume a gateway as the entry point.
1. Gateway vs BFF vs Reverse Proxy vs Service Mesh
These four patterns are frequently conflated. They solve different problems at different layers of the stack.
| Pattern | Owns What | Protocol-Aware? | Auth Enforcement | Observability Role |
|---|---|---|---|---|
| Reverse Proxy (NGINX, HAProxy) | TLS termination, load balancing, static routing | HTTP method only | None | Access logs |
| API Gateway (Kong, AWS APIGW, Apigee) | Request policy enforcement (auth, rate-limit, routing, transform, trace) | Full HTTP + gRPC | JWT/OAuth2 validation | Root span initiation |
| BFF (Backend-for-Frontend) | Client-shaped data aggregation | Full HTTP + gRPC → JSON | Delegates to gateway | Downstream span |
| Service Mesh (Istio, Linkerd) | East-west (service-to-service) traffic, mTLS, telemetry | L4/L7 (sidecar) | mTLS workload identity | Automatic span per hop |
A BFF is not a gateway. A BFF owns client contracts and aggregates data for a specific client type (desktop, mobile). A gateway owns cross-cutting policy: who is allowed in, how fast, and where the request goes. A BFF sits behind the gateway. Merging them creates a service that is too large to own by a single team and too coupled to evolve independently.
2. Dynamic Routing & Service Registry Integration
3. JWT Authentication Offloading
3.1 The Distributed Validation Anti-Pattern
3.2 Gateway-Level JWT Validation

4. Distributed Rate Limiting
4.1 The Per-Instance Counter Failure
4.2 Redis-Backed Distributed Rate Limiting
Always set the Retry-After header on 429 responses. Clients that do not receive a Retry-After value will retry immediately, converting a rate-limit event into a thundering herd. See Part 7: Retry Engineering for how clients should consume this header correctly.
5. Global Circuit Breaking at the Gateway
The gateway circuit breaker is a system-level backstop. Individual services may have their own circuit breakers for inter-service calls. The gateway breaker protects the entire system from a client-visible cascade when a service starts failing — even if the service has no internal circuit breaker at all. Defense in depth: both layers should exist; the gateway layer is the last resort.
6. Protocol Translation: REST → gRPC
7. OpenTelemetry Trace ID Initiation
Return the X-Trace-Id header in all gateway responses. Client-facing error messages can say "Contact support with trace ID: abc123" — support engineers can then pull the full distributed trace in Jaeger or Grafana Tempo in seconds without needing to reproduce the issue.
8. API Composition & Fan-Out

Summary
| Concern | Gateway Rule |
|---|---|
| Routing | DNS-based dynamic routing via k8s service registry — never static IPs |
| Auth | JWT validated at gateway perimeter; services receive X-User-Id + X-Roles headers |
| Rate Limiting | Redis-backed distributed counter — per-user or per-IP key; always set Retry-After |
| Circuit Breaking | Gateway-level breaker per service; returns 503 fallback when OPEN |
| Protocol Translation | External REST → Internal gRPC; public contract is decoupled from internal protocol |
| Trace Initiation | Gateway generates root span; injects traceparent into all upstream requests |
| Composition | Fan-out to multiple services in parallel; surface partial failures as null fields |
What's Next
In Part 3, we go deeper into the east-west traffic layer — Part 3: Service Mesh & mTLS examines how Istio's Envoy sidecar provides automatic mTLS, workload identity, and declarative traffic management between every service pair, without any application code changes.
This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.