System Design: Notification Platform, 100M Notifications/Second
Goal: Build a notification platform that pushes 100 million notifications per second across web (WebSocket/SSE), Android (FCM), and iOS (APNs). Support scheduling, broadcast, per-user preferences, and survive failures with at-least-once delivery.
Here's how to architect it, from ingestion to delivery, with scale math and failure scenarios.
1. Problem Statement
The goal: a notification platform that pushes 100 million notifications per second across web, Android, and iOS.
Design a notification system that can send 100 million notifications per second. Support web push (real-time via SSE/WebSocket), Android (FCM), iOS (APNs), scheduled notifications, and handle failures gracefully.
A few clarifications before diving in:
Scale clarification: 100M/sec is massive, on par with the biggest platforms in production today. I'll design for this as a sustained peak, not a burst. Every layer must be horizontally partitioned.
Assumptions I'm making:
- This is a multi-tenant platform (like OneSignal, Pusher, or Firebase itself, serving many apps).
- "Send" means accepted + dispatched. Actual delivery to the user's eyeball depends on device state, OS, and network.
- At-least-once delivery semantics. Exactly-once is impractical at this scale, so deduplicate where possible.
- Users can be on multiple devices simultaneously (phone + laptop + tablet).
- The system is a backend platform. It doesn't build client apps, it provides SDKs and APIs.
What NOT to do:
- Poll clients asking "any new notifications?" (this is push, not pull)
- Store notifications in a single PostgreSQL table and SELECT by user_id (fails at 1M/sec)
- Use Redis as the sole data store (volatile memory, no durability)
- Send FCM/APNs synchronously in the request path (one slow Apple response blocks everything)
- Broadcast every notification to every WebSocket pod (20B internal messages/sec of waste)
- Build one monolith that does ingestion + routing + delivery + scheduling
The trickiest problem in this design? A notification can arrive at any server, but the user's WebSocket lives on one specific pod out of hundreds. Section 9.3 covers how to solve this.
2. Functional Requirements
| ID | Requirement | Priority |
|---|---|---|
| FR-01 | Send real-time push notifications to web browsers via persistent connection (WebSocket/SSE) | P0 |
| FR-02 | Send push notifications to Android devices via FCM | P0 |
| FR-03 | Send push notifications to iOS devices via APNs | P0 |
| FR-04 | Support scheduled notifications (deliver at a future timestamp, recurring via cron) | P0 |
| FR-05 | Support broadcast notifications (send to all users, segments, topics) | P0 |
| FR-06 | Support targeted notifications (single user, device, or channel) | P0 |
| FR-07 | Track delivery status: sent, delivered, read, failed, retried | P1 |
| FR-08 | Support notification templates with variable interpolation | P1 |
| FR-09 | User-level preference management (opt-in/out per channel, quiet hours) | P1 |
| FR-10 | Multi-tenant isolation | P0 |
| FR-11 | Idempotent delivery (deduplication) | P0 |
| FR-12 | Notification cancellation before delivery | P1 |
| FR-13 | Priority levels (critical, high, normal, low) | P1 |
| FR-14 | Rate limiting per tenant, per user, per channel | P0 |
| FR-15 | Dead letter queue for permanently failed notifications | P1 |
3. Non-Functional Requirements
| ID | Requirement | Target |
|---|---|---|
| NFR-01 | Throughput | 100M notifications/sec sustained |
| NFR-02 | Latency (p50 / p99) | < 100ms / < 500ms for real-time channels |
| NFR-03 | Availability | 99.99% (52 min downtime/year) |
| NFR-04 | Durability | No notification loss once accepted (at-least-once) |
| NFR-05 | Horizontal scalability | Linear scale-out, no single bottleneck |
| NFR-06 | Data retention | Hot: 7 days, Warm: 90 days, Cold: 1 year |
| NFR-07 | Concurrent WebSocket/SSE connections | 100M peak (designed for 5x growth to 500M) |
| NFR-08 | Scheduled notification precision | ≤ 1 second drift |
| NFR-09 | Recovery Time Objective (RTO) | < 30 seconds |
| NFR-10 | Recovery Point Objective (RPO) | 0 (no data loss) |
| NFR-11 | Geographic distribution | Multi-region active-active |
| NFR-12 | Tenant isolation | Noisy neighbor protection at every layer |
4. High-Level Approach & Technology Selection
🔒 Premium section
5. High-Level Architecture
🔒 Premium section
6. Back-of-the-Envelope Estimation
🔒 Premium section
7. Data Model
🔒 Premium section
8. API Design
🔒 Premium section
9. Deep Dives
🔒 Premium section
10. Identify Bottlenecks
🔒 Premium section
11. Failure Scenarios
🔒 Premium section
12. Deployment Strategy
🔒 Premium section
13. Observability
🔒 Premium section
14. Security
🔒 Premium section
Explore the Technologies
🔒 Premium section
