API Gateway vs Load Balancer vs Reverse Proxy

How It Works

These three concepts sit on a spectrum of network intelligence. At one end, a load balancer makes a simple decision: which backend instance gets this connection? In the middle, a reverse proxy understands the protocol and can terminate TLS, cache responses, and route by URL. At the far end, an API gateway understands the application — it validates auth tokens, enforces rate limits, transforms requests, and exposes developer-facing API management.

The confusion exists because modern tools blur these boundaries. NGINX started as a reverse proxy but handles load balancing. AWS ALB is called a load balancer but routes by HTTP path. Kong is an API gateway but also proxies and load-balances. Understanding the conceptual layers helps engineers make architectural decisions even when a single product handles multiple roles.

Load Balancer: Distributing Traffic

A load balancer's primary job is simple: given N backend instances, decide which one handles each request. This is the fundamental building block of horizontal scaling.

L4 (Transport Layer) Load Balancers operate at TCP/UDP. They see IP addresses and port numbers, nothing else. They cannot inspect HTTP headers, URL paths, or cookies. What they lack in intelligence, they make up in speed — an L4 LB can handle millions of connections per second because it never parses the payload.

L4 Load Balancer decision:
  Input:  TCP SYN to 10.0.0.1:443
  Output: Forward to 10.0.1.{5,6,7}:8080 (round-robin)
  
  Cannot: Route by URL path, inspect headers, cache responses

L7 (Application Layer) Load Balancers understand HTTP. They parse the request, read headers, and make routing decisions based on content. AWS ALB, HAProxy in HTTP mode, and NGINX in upstream configuration are L7 load balancers.

L7 Load Balancer decision:
  Input:  GET /api/v2/users Host: api.example.com
  Output: Route to users-service-v2 pool (based on path prefix)
  
  Can: Route by path, host, header, cookie, query parameter

Load balancing algorithms determine distribution:

Algorithm	How It Works	Best For
Round Robin	Rotate through backends sequentially	Equal-capacity backends
Least Connections	Send to the backend with fewest active connections	Variable request duration
Weighted	Assign proportional traffic by backend weight	Mixed-capacity backends, canary deploys
Consistent Hashing	Hash a key (IP, cookie) to a fixed backend	Session affinity, caching layers
Random with Two Choices	Pick two random backends, choose the less loaded one	Large pools where least-conn is expensive

Reverse Proxy: Mediating Connections

A reverse proxy sits between clients and the backend servers. Clients talk to the proxy, and the proxy talks to backends. Unlike a forward proxy (which hides clients from servers), a reverse proxy hides servers from clients. The client never knows the backend topology.

Key reverse proxy capabilities:

TLS Termination: The proxy holds the SSL certificate and decrypts HTTPS traffic. Backends receive plaintext HTTP, simplifying their configuration and reducing CPU load. This is the single most common reason to deploy a reverse proxy.

# NGINX TLS termination
server {
    listen 443 ssl;
    ssl_certificate     /etc/nginx/cert.pem;
    ssl_certificate_key /etc/nginx/key.pem;
    
    location / {
        proxy_pass http://backend:8080;  # plaintext to backend
    }
}

Response Caching: The proxy caches responses from backends, serving repeat requests without hitting the backend at all. For read-heavy APIs, this can reduce backend load by 80-90%.

Compression: The proxy compresses responses (gzip, brotli) before sending to clients, reducing bandwidth without backend code changes.

Request Routing: Based on URL path, host header, or other request attributes, the proxy routes to different backend pools. /api/* goes to the API servers, /static/* goes to the CDN origin, /admin/* goes to the admin service.

Connection Pooling: Clients open many short-lived connections to the proxy. The proxy maintains a smaller pool of persistent connections to backends, reducing connection establishment overhead.

API Gateway: Managing APIs

An API gateway is a reverse proxy with application-level intelligence. It understands not just HTTP, but the API contracts — who is allowed to call what, how fast, and with what transformations.

Core API gateway functions:

Authentication and Authorization: The gateway validates auth tokens (JWT, API keys, OAuth) before the request reaches any backend. If a token is expired or invalid, the backend never sees the request. This centralizes auth logic instead of reimplementing it in every service.

# Kong API Gateway — JWT authentication plugin
plugins:
  - name: jwt
    config:
      claims_to_verify:
        - exp
      header_names:
        - Authorization
  - name: acl
    config:
      allow:
        - premium-users

Rate Limiting: The gateway enforces request quotas per API key, user, IP, or any custom attribute. This protects backends from abuse, implements tiered pricing (free tier: 100 req/min, paid: 10,000 req/min), and prevents a single client from consuming all capacity.

Request/Response Transformation: The gateway can modify requests before forwarding (add headers, rename fields, merge responses from multiple backends) and modify responses before returning to clients. This is how teams maintain backward compatibility — the gateway translates old API formats to new backend formats.

API Versioning: Route /api/v1/users to the legacy service and /api/v2/users to the new service, managed at the gateway without client changes.

Developer Portal and Analytics: API gateways often include developer-facing features: API documentation, API key issuance, usage dashboards, and deprecation notices.

The Comparison Table

This table shows where responsibilities actually live. Notice the overlap — these are not discrete categories.

Feature	Load Balancer	Reverse Proxy	API Gateway
Traffic distribution	Yes (primary purpose)	Yes	Yes
Health checks	Yes	Yes	Yes
TLS termination	L7 only	Yes (primary purpose)	Yes
Response caching	No	Yes	Sometimes
Compression	No	Yes	Sometimes
Request routing (path/host)	L7 only	Yes	Yes
Authentication	No	Basic (client certs)	Yes (primary purpose)
Rate limiting	No	Basic (per-IP)	Yes (per-key, per-user)
Request transformation	No	Minimal (headers)	Yes (body, headers, path)
API versioning	No	No	Yes
Developer portal	No	No	Yes
L4 (TCP/UDP) support	Yes	Limited	No
Protocol translation	No	No	Yes (REST→gRPC, etc.)

How They Compose in Real Architectures

In a production system, all three layers often appear. Here is a typical cloud-native setup:

Internet
  │
  ▼
┌──────────────────────────┐
│  Cloud Load Balancer (L4) │  ← Distributes across AZs
│  (AWS NLB, GCP TCP LB)   │     Handles millions of connections
└──────────┬───────────────┘
           │
  ┌────────▼────────┐
  │  Reverse Proxy   │  ← TLS termination, caching, compression
  │  (NGINX, Envoy)  │     Static asset serving
  └────────┬────────┘
           │
  ┌────────▼────────┐
  │  API Gateway     │  ← Auth, rate limiting, API versioning
  │  (Kong, custom)  │     Request transformation
  └────────┬────────┘
           │
    ┌──────┼──────┐
    ▼      ▼      ▼
  Svc A  Svc B  Svc C    ← Business logic

However, this three-layer stack is not always necessary. Here are common simplifications:

Small to Medium (< 50 services): A single product like NGINX Plus or Envoy handles all three roles. Configure load balancing, TLS termination, and basic rate limiting in one place.

Cloud-Native: AWS ALB + API Gateway. The ALB handles L7 load balancing, TLS, and path-based routing. API Gateway adds authentication, throttling, and Lambda integration. No self-managed reverse proxy needed.

Kubernetes: An Ingress Controller (NGINX Ingress, Traefik, Envoy-based) acts as reverse proxy and load balancer. Add Kong Ingress Controller or Ambassador when API gateway features are needed. The service mesh handles service-to-service load balancing.

Common Architecture Decisions

When to separate layers vs. combine them:

Separate when: the architecture needs independent scaling (the gateway CPU-bottlenecks on auth token validation while the LB is idle), independent teams manage different layers, or compliance requires a dedicated WAF/API gateway appliance.

Combine when: optimizing for simplicity and operational cost, traffic volume does not require independent scaling, or the team is early-stage and a single NGINX config covers everything.

When an API gateway is needed vs. just a reverse proxy:

If the requirements are TLS termination, caching, and basic load balancing, a reverse proxy (NGINX, Caddy) is sufficient. The moment the system needs per-user authentication, per-key rate limiting, request transformation, or API analytics, an API gateway is necessary.

# Quick test: Does the reverse proxy handle what's needed?
# If any of these are required, add a gateway:
# - Validate JWTs and reject expired tokens
# - Rate limit by API key (not just by IP)
# - Transform request bodies (rename fields, merge payloads)
# - Provide API key self-service to developers
# - Track usage per consumer for billing

# If only these are needed, a reverse proxy is enough:
# - TLS termination
# - Static file serving and caching
# - Path-based routing to different backends
# - Gzip compression
# - Basic IP-based rate limiting

The microservices trap: Do not put a public-facing API gateway in the path of every internal service-to-service call. The gateway is for north-south traffic (external clients to internal services). For east-west traffic (service-to-service), use a service mesh or simple client-side load balancing. An API gateway in the east-west path adds latency and creates a bottleneck.