System Design: URL Shortener (10B Short URLs, 100K Redirects/sec)
Goal. Build a URL shortener for 10 billion stored URLs, 100K redirect lookups/sec, and 1K creates/sec. Custom aliases, expiration, click analytics, and cache-hit redirect latency under 5ms p99 server-side.
TL;DR. This is a read-heavy key-value lookup with an async analytics sidecar.
Each region mints IDs from its own counter with zero cross-region coordination, shuffled and Base62-encoded into 7-character codes (62⁷ ≈ 2⁴² IDs).
ScyllaDB stores the mappings, replicated active-active across three regions. The read path is CDN → local LRU → Valkey → Scylla, with each layer absorbing what the previous one missed.
Kafka + Flink + ClickHouse run analytics asynchronously, off the critical path.
1. Final Architecture
🔒 Premium section
2. Problem Statement
A URL shortener is an afternoon prototype and a months-long production system. Three things make it hard at scale:
- Unique IDs with no coordination. At 1K creates/sec across regions, a single auto-increment row is a bottleneck. Random IDs need a DB read per write. Hash truncation collides at 42 bits (the birthday paradox makes collisions likely around 10B rows).
- Redirect latency under skewed load. 100K/sec of reads, a Zipf-shaped hot set, and a sub-5ms p99 target on cache hit. You can't touch the DB on the hot path.
- Analytics without coupling. Every redirect produces a click event. Writing those synchronously adds 10-50ms to every 302. They have to go through a queue.
Scale numbers.
- 10B URLs stored
- 1K creates/sec (~86M/day)
- 100K redirects/sec (~8.6B/day), distributed across regions
- 100:1 read-to-write ratio
- 200 B average long URL; 7-character short code
3. Functional Requirements
| ID | Requirement | Priority |
|---|---|---|
| FR-01 | Create a short URL from a long URL, returning a unique 7-character code | P0 |
| FR-02 | Redirect short URL to original long URL via HTTP 301/302 | P0 |
| FR-03 | Support custom aliases (user-chosen short codes) | P0 |
| FR-04 | URL expiration: optional TTL (1 day, 7 days, 30 days, 1 year, never) | P0 |
| FR-05 | Click analytics: total clicks, clicks over time, geographic distribution | P1 |
| FR-06 | Referrer and device tracking per click | P1 |
| FR-07 | Bulk URL creation via API (up to 1000 URLs per request) | P1 |
| FR-08 | URL deletion by owner | P1 |
| FR-09 | API key authentication for URL creation | P0 |
| FR-10 | Rate limiting per API key (100 creates/min default) | P0 |
| FR-11 | QR code generation for any short URL | P2 |
| FR-12 | Link preview metadata (title, description, image from target page) | P2 |
4. Non-Functional Requirements
| ID | Requirement | Target |
|---|---|---|
| NFR-01a | Redirect latency, cache-hit path (server-side) | p50 < 2ms / p99 < 5ms |
| NFR-01b | Redirect latency, cache-miss path (server-side) | p50 < 8ms / p99 < 15ms |
| NFR-01c | Redirect latency, end-to-end (client-observed, intra-region) | p50 5–10ms / p99 20–40ms |
| NFR-02 | Create latency (p50 / p99) | < 20ms / < 50ms |
| NFR-03 | Redirect throughput | 100K/sec sustained globally |
| NFR-04 | Create throughput | 1K/sec sustained (10K burst) |
| NFR-05 | Availability | 99.99% (52 min downtime/year) |
| NFR-06 | URL durability | Zero data loss for non-expired URLs |
| NFR-07 | Data retention | Expired URLs purged automatically via Scylla TTL; active URLs stored indefinitely |
| NFR-08 | Analytics freshness | < 5 second lag from click to dashboard |
| NFR-09 | Short code length | 7 characters (Base62 = 3.5 trillion combinations) |
- NFR-01a/b are server-side. Request arriving at the Redirect Service to response leaving it, and they're what we page on.
- NFR-01c is end-to-end. Dominated by network round-trip, not server work (intra-region ~10-20ms, cross-region 40-150ms).
CDN edge caching, not origin speed, is what keeps user-perceived time low worldwide.
5. Design Assumptions
🔒 Premium section
6. High-Level Architecture
🔒 Premium section
7. Back-of-the-Envelope
🔒 Premium section
8. Data Model
🔒 Premium section
9. API Design
🔒 Premium section
10. ID Generation
🔒 Premium section
11. Caching
🔒 Premium section
12. Click Analytics Pipeline
🔒 Premium section
13. Multi-Region Writes
🔒 Premium section
14. Custom Alias & Expiration
🔒 Premium section
15. Security & Abuse
🔒 Premium section
16. Failure Scenarios
🔒 Premium section
17. Operational Playbook
🔒 Premium section
18. SLOs and Error Budgets
🔒 Premium section
19. Appendix
🔒 Premium section
Explore the Technologies
🔒 Premium section
