System Design: Online Auction (50K Bids/sec, Effectively-Once Settlement, Anti-Sniping)
Goal
A real-time online auction platform.
Scale:
- 10M active listings
- 50K bids/sec at peak, 1.7K bids/sec average
- 1M concurrent WebSocket watchers
- Sub-200ms regional p99 bid confirmation and broadcast, 99.99% availability (cross-region readers see +100 ms)
Features:
- English, Dutch, and sealed-bid auction types
- Proxy (auto) bidding
- Anti-sniping extension
- Effectively-once settlement converging on a single committed winner
Architecture at a glance
🔒 Premium section
1. Problem Statement
An auction platform sounds simple until the last ten seconds of a popular listing.
A rare sneaker auction is ending. 500 users are watching.
In the final 30 seconds, 50 bids arrive within a 2-second window. Each must be validated against a price that is changing bid by bid, processed in strict arrival order, and broadcast to all 500 watchers within 200 ms.
If two users bid $105 when the current price is $100, only one commits and the other is told "outbid, the price is now $105." Not both. Not neither.
That is the central challenge: strongly serialized writes per auction combined with low-latency fan-out to thousands of readers, all while settlement at auction end is idempotent in the face of crashes.
Four problems drive the design.
Concurrent bids on the same auction. Two users click "Bid $105" at the same instant when the current price is $100. Without concurrency control, both bids pass the "$105 > $100" check and both get accepted.
The fix is optimistic concurrency: every bid carries the price it expected to see, and acceptance is conditional on that value still being current. Under Valkey's single-threaded execution, an atomic Lua script gives per-auction serialization for free.
Bid sniping. A user places a bid in the final second, leaving no one time to respond. Some platforms accept this as legitimate strategy.
Fairer-outcome platforms (eBay Live, Catawiki) extend the auction by a short window when a bid lands near the end. Anti-sniping is configurable per auction.
Settlement must be effectively-once. When time runs out, the system must converge on a single committed winner after retries settle, and the payment provider must end up with a single captured charge once it acks.
A settlement job that crashes between "write SOLD" and "call Stripe" must restart without double-charging. The solution is a fencing token plus target-side idempotency. Standard pattern for any job that terminates with an external side effect.
Real-time broadcast at scale. 1M concurrent WebSocket connections across a fleet of stateless gateway pods. Every accepted bid must reach every watcher of that auction within 200 ms.
Polling is not an option. The fan-out path is Valkey Pub/Sub, with each gateway pod only subscribing to channels for auctions its users care about.
Scale targets.
- 10M active listings at any time
- 50K bids/sec at peak, 1.7K bids/sec average (30× ratio driven by evening prime-time)
- 1M concurrent WebSocket watchers
- Average auction duration 7 days; minimum 1 hour; maximum 30 days
- Bid confirmation and broadcast latency: <200 ms p99
- 99.99% availability for bid processing
2. Functional Requirements
| ID | Requirement | Priority |
|---|---|---|
| FR-01 | Create auction listings: title, description, images, starting price, reserve price, bid increment, start and end times, auction type | P0 |
| FR-02 | Place bids on active auctions with real-time validation against current highest | P0 |
| FR-03 | Real-time bid updates pushed to watchers over WebSocket within 200 ms | P0 |
| FR-04 | Anti-sniping: extend auction end time by a configurable amount when a bid arrives within the final window | P0 |
| FR-05 | Effectively-once settlement: winner determination, reserve check, payment capture | P0 |
| FR-06 | English auction: ascending bids, highest wins | P0 |
| FR-07 | Dutch auction: price drops on a schedule, first to accept wins | P1 |
| FR-08 | Sealed-bid auction: blind bids, revealed at close, highest wins | P1 |
| FR-09 | Proxy bidding: user sets a max, system auto-bids the minimum increment on their behalf | P1 |
| FR-10 | Watchlist: users subscribe to auctions and receive notifications on key events | P1 |
| FR-11 | Bid history: full audit trail of bids per auction | P0 |
| FR-12 | Reserve price: sale only completes if final bid meets the seller's hidden minimum | P0 |
| FR-13 | Search and browse by category, price range, ending soon, newly listed | P1 |
| FR-14 | Bid retraction within policy window | P2 |
3. Non-Functional Requirements
| ID | Requirement | Target |
|---|---|---|
| NFR-01 | Bid processing throughput | 50K bids/sec peak, 1.7K average |
| NFR-02 | Bid confirmation latency (regional p50 / p99) | 60 ms / 200 ms (cross-region readers see +100 ms) |
| NFR-03 | Bid broadcast latency (regional p99, acceptance to watcher frame) | <200 ms |
| NFR-04 | Active concurrent auctions | 10M |
| NFR-05 | Concurrent WebSocket connections | 1M |
| NFR-06 | Bid processing availability | 99.99% (52 min/year) |
| NFR-07 | Settlement guarantee | Effectively-once (one SOLD row, one captured charge) |
| NFR-08 | Bid data durability | Zero loss once the API returns 202 |
| NFR-09 | Anti-sniping timer precision | <1 s drift |
| NFR-10 | Recovery Time Objective | <30 s for bid processor partition rebalance |
| NFR-11 | Recovery Point Objective | 0 for accepted bids |
| NFR-12 | Retention | Bids: hot 90 days in Postgres, archive to S3, drop after 2 years |
| NFR-13 | Geography | Multi-region active reads; bid writes pinned per-auction to a single region |
| NFR-14 | Search latency | <500 ms p99 |
[3.1] Traffic and workload assumptions
- Median bids per auction ~15; mean ~105. The distribution is long-tailed: most listings end quiet, a small fraction of hot listings pull the mean up sharply. Downstream math (§6.1) uses the mean.
- 3% of auctions end in any given hour during evening prime time.
- Hot auctions (top 0.01%) can take 100-500 bids/sec in the final minute.
- Payment provider (Stripe-equivalent) supports idempotency keys and 2xx/4xx responses within 1 s p99.
- Watchers per auction: average 30, hot auction up to 5K.
- Clients resolve a regional endpoint via DNS; the chosen region processes the bid (auction is pinned to its region).
4. End-to-End Architecture
🔒 Premium section
5. Technology Selection
🔒 Premium section
6. Back-of-the-Envelope
🔒 Premium section
7. Data Model
🔒 Premium section
8. API Design
🔒 Premium section
9. Settlement, Payouts, and Risk
🔒 Premium section
10. Bid Processing Model
🔒 Premium section
11. Timers and Anti-Sniping
🔒 Premium section
12. Hot Auctions and Fair Queueing
🔒 Premium section
13. Auction Search and Ranking
🔒 Premium section
14. Multi-Region
🔒 Premium section
15. Bottlenecks and Backpressure
🔒 Premium section
16. Retries, Fraud, and Recovery
🔒 Premium section
17. Failure Scenarios
🔒 Premium section
18. Operational Playbook
🔒 Premium section
19. SLOs and Error Budgets
🔒 Premium section
20. Security
🔒 Premium section
21. Key Takeaways
🔒 Premium section
22. Appendix
🔒 Premium section
Explore the Technologies
🔒 Premium section