System Design: LeetCode (Code Sandbox, Container Isolation, Real-Time Contests)
Goal: Design an online judge platform like LeetCode that handles 50 million code submissions per day across 20+ programming languages.
- Execute untrusted user code safely in Firecracker microVMs with hardware-level KVM isolation.
- Support real-time contests with live leaderboards, Elo-based rating, and penalty calculation.
- Target sub-5-second end-to-end execution latency, 20K concurrent submissions, and 15M registered users.
1. Final Architecture
🔒 Premium section
2. Problem Statement
An online judge sounds deceptively simple. Someone writes code.
The system runs it. Check if the output matches. Done.
The reality? It's running arbitrary, untrusted code from millions of strangers. That code could be a fork bomb, a Bitcoin miner, a kernel exploit, or an infinite loop that allocates 64GB of memory.
And the system needs to run 50 million of these per day. It has to return the right answer in under 5 seconds. It has to do that across 20 different programming languages, each with its own compiler, runtime, and memory model.
Problem 1: Running untrusted code without getting owned.
Someone submits os.system("rm -rf /") in Python, or while(1) fork() in C++. Or something subtler: code that reads /proc/self/mountinfo to fingerprint the container and tries known escape exploits. A default Docker container won't stop any of this.
Kernel-level isolation is mandatory: intercept every syscall, block network access, and kill anything that exceeds resource limits. One sandbox escape = access to test cases, other users' code, or production infrastructure.
Problem 2: 50M submissions/day means thousands of containers running simultaneously.
580 submissions/second average, 2K peak. Each takes 2-10 seconds, which means 1,200 to 5,800 containers in parallel. Each needs its own isolated filesystem, resource limits, and language runtime.
Fresh Docker containers per submission is a non-starter (3-5s cold start). A warm pool of pre-spawned containers is the only workable option.
Problem 3: Contest fairness requires deterministic execution and cheat prevention.
Same code, different machines: 48ms vs 72ms due to noisy neighbors. Ranking by raw execution time is unfair, timing must be normalized.
Scale numbers:
- 15M registered users, 3M monthly active users
- 50M code submissions per day (~580/sec avg, 2,000/sec peak)
- 20K concurrent submissions during peak contest windows
- 20+ supported programming languages
- 4,000+ problems with 10-50 test cases each
- Average execution time: 3 seconds (compilation + running all test cases)
- Weekly contests: 100K participants, 4-5 problems, 90-minute window
What NOT to do:
exec()/eval()on the app server. That's RCE on production.- Default Docker security. Shared kernel, no syscall filtering. Escapes are well-documented (CVE-2019-5736, CVE-2020-15257).
- Network access in containers. Code could fetch solutions, exfiltrate data, or attack internal services.
- Raw execution time for ranking. Noisy neighbors cause 2x variance. Normalize or use dedicated hardware.
- Running all test cases after first failure. Wrong output on case 2 of 50? Stop. Only run all for Accepted.
- Test cases inside the container image. Users could read expected outputs and hardcode answers.
- Single queue for all submissions. Contest and practice traffic compete. Separate queues with priority.
- Solutions and test cases in the same DB. Different sizes, different access patterns, different write rates.
- Monolithic judge. API, execution, leaderboard, problem management scale completely differently.
The sandbox isolation layer is where most of the complexity lives. Getting it wrong means either a security breach or unacceptable performance. §11.1 breaks down the tradeoffs.
3. Functional Requirements
| ID | Requirement | Priority |
|---|---|---|
| FR-01 | Execute user-submitted code in a sandboxed environment with strict resource limits | P0 |
| FR-02 | Support 20+ programming languages (Python, Java, C++, Go, Rust, JavaScript, C, C#, Ruby, Kotlin, Swift, TypeScript, Scala, PHP, Haskell, Dart, Elixir, Erlang, Racket) | P0 |
| FR-03 | Run submitted code against ordered test cases and return a verdict (Accepted, Wrong Answer, TLE, MLE, Runtime Error, Compilation Error) | P0 |
| FR-04 | Display execution time and memory usage for each submission | P0 |
| FR-05 | Support contests with timed problem sets and real-time leaderboards | P0 |
| FR-06 | Support penalty time calculation (time to solve + penalty per wrong attempt) | P0 |
| FR-07 | Provide a problem bank with descriptions, constraints, examples, and hidden test cases | P0 |
| FR-08 | Show submission history per user per problem | P0 |
| FR-10 | Support "Run Code" (test against visible examples only, fast debug loop) separate from "Submit" (full hidden test suite) | P0 |
| FR-11 | Rate-limit submissions per user (5/minute for practice, 10/minute during contests) | P0 |
| FR-12 | Push real-time verdict updates to users via WebSocket | P1 |
| FR-13 | Support problem difficulty tagging and topic categorization | P1 |
| FR-14 | Track user statistics (problems solved, acceptance rate, contest Elo rating) | P1 |
| FR-15 | Support editorial solutions and community discussions per problem | P2 |
| FR-16 | Distinguish TLE (user's algorithm too slow) from Timeout (server overloaded, auto-retry) | P0 |
| FR-17 | Premium priority queue: 3-10x faster judging for premium subscribers (practice only, not contests) | P1 |
| FR-18 | Post-contest Elo rating computation with absence penalty | P1 |
4. Non-Functional Requirements
| ID | Requirement | Target |
|---|---|---|
| NFR-01 | End-to-end submission latency (submit to verdict) | < 5 seconds (p95), < 10 seconds (p99) |
| NFR-02 | Execution throughput | 50M submissions/day, 2,000/sec peak |
| NFR-03 | Concurrent submissions | 20,000 during peak contest windows |
| NFR-04 | microVM snapshot restore time | < 50ms (with warm pool: < 5ms claim) |
| NFR-05 | Availability | 99.95% (26 min downtime/month, non-contest), 99.99% during contests |
| NFR-06 | Sandbox escape rate | 0 (any escape is a critical security incident) |
| NFR-07 | Leaderboard update latency | < 1 second from verdict to leaderboard update |
| NFR-08 | WebSocket message delivery | < 500ms from verdict to client notification |
| NFR-10 | Horizontal scalability | Linear scale-out by adding execution workers |
| NFR-11 | Data retention | Submissions: 2 years. Test cases: indefinite. Contest results: indefinite. |
| NFR-12 | Language image update | < 4 hours from new language/version release to production availability |
5. High-Level Approach & Technology Selection
🔒 Premium section
6. Design Assumptions
🔒 Premium section
7. High-Level Architecture
🔒 Premium section
8. Back-of-the-Envelope Estimation
🔒 Premium section
9. Data Model
🔒 Premium section
10. API Design
🔒 Premium section
11. Deep Dives
🔒 Premium section
12. Identify Bottlenecks
🔒 Premium section
13. Failure Scenarios
🔒 Premium section
14. Deployment Strategy
🔒 Premium section
15. Observability
🔒 Premium section
16. Security
🔒 Premium section
17. SLOs and Error Budgets
🔒 Premium section
18. Operational Playbook
🔒 Premium section
19. Appendix
🔒 Premium section
Explore the Technologies
🔒 Premium section