Internal API Gateway

Gateway vs Service Mesh for Internal Traffic

This is the most common question platform teams face. An API gateway sits at the edge of your service network and handles request routing, authentication, rate limiting, and protocol translation. A service mesh operates at the transport layer between every service.

Use a gateway when you need centralized policy enforcement, API versioning, and a developer portal. Use a mesh when you need per-service mTLS, fine-grained traffic splitting, and observability across every service-to-service call. Many organizations run both: a gateway for API management and a mesh for transport security. Shopify and Airbnb use this dual pattern.

Kong vs Envoy vs AWS API Gateway

Kong is built on NGINX and OpenResty. It's the easiest to operate and has the largest plugin ecosystem (100+ plugins). The open source version handles most use cases. Kong Gateway Enterprise adds RBAC, developer portals, and analytics at $35k+/year. Latency overhead is typically under 1ms without plugins.

Envoy is a proxy, not a gateway out of the box. You need a control plane like Envoy Gateway, Ambassador (now Emissary-Ingress), or Gloo Edge. Envoy itself is extremely performant and configurable but has a steep learning curve. It's the right choice if you're already running Istio or need advanced traffic management.

AWS API Gateway is fully managed but designed primarily for external APIs. For internal east-west traffic, the HTTP API type works at $1/million requests. The main limitation is that it adds 10-30ms latency per call, which compounds in deep service call chains.

Rate Limiting and Circuit Breaking

Rate limiting at the gateway is your first line of defense against cascading failures. Set per-service rate limits based on expected traffic patterns plus a 2-3x buffer. A payments service that handles 500 requests per second should be limited at 1,500 rps at the gateway. Anything above that gets a 429 response before it reaches the service.

Circuit breaking works alongside rate limiting. When a downstream service starts returning 500 errors, the gateway trips the circuit and returns a fallback response. This prevents one unhealthy service from consuming all gateway resources and degrading other routes. Kong's circuit breaker plugin and Envoy's outlier detection both handle this.

Developer Portal Integration

The gateway becomes a platform product when you add a developer portal. Teams should be able to browse available internal APIs, read documentation, generate API keys, and test endpoints without asking another team. Kong Developer Portal and Backstage both support this pattern.

The portal should show real-time metrics: request volume, error rates, and latency percentiles for each API. When a team sees their API getting 10,000 requests per minute from a single consumer, they can address it proactively instead of discovering it during an outage.

API Lifecycle Management

Internal APIs need versioning just like external ones. The gateway makes this easy. Route /v1/users to the old service and /v2/users to the new service. Run both simultaneously until all consumers migrate. The gateway tracks which consumers still call v1, making deprecation a data-driven decision.

Mandate that every internal API registers with the gateway. No direct service-to-service calls that bypass the gateway. This gives platform teams visibility into the full API dependency graph. When a team needs to deprecate a service, they can see exactly who depends on it.