The framework
1. Requirements (5 min, and never skip it)
- Functional: the 3–5 core use cases. Say explicitly what's out of scope.
- Non-functional: scale (DAU, QPS), latency targets (p99), availability (99.9% ≈ 43 min/month), consistency needs, durability, multi-tenancy, compliance and data residency.
2. Back-of-envelope
- 1M requests/day ≈ 12 QPS average; plan for peak at 2–5× average.
- Storage = objects/day × size × retention × replication factor.
- 1 day ≈ 86,400 s ≈ 10⁵ s (handy approximation).
3–5. API, data, boxes
Define endpoints or events, then entities and access patterns. Access patterns choose the database, not fashion. Then draw clients → edge/LB → services → caches → stores → async workers.
6. Deep dives
Hot keys, fan-out, the single points of failure, how it scales 10×, what happens when X is down, how you deploy without downtime, and how you observe it.
7. Trade-offs
State what you chose, what you gave up, and what you'd change at 100× scale.
Numbers worth memorising (orders of magnitude)
| Operation | ≈ Latency |
|---|---|
| L1 cache reference | 1 ns |
| Main memory reference | 100 ns |
| Read 1 MB sequentially from memory | ~10 µs (varies) |
| SSD random read | ~100 µs |
| Round trip within a datacenter | ~0.5 ms |
| Redis GET over network | ~0.5–1 ms |
| Round trip Mumbai ↔ US East | ~200 ms |
Availability math
| SLA | Downtime / month |
|---|---|
| 99% | ~7.3 h |
| 99.9% | ~43.8 min |
| 99.95% | ~21.9 min |
| 99.99% | ~4.4 min |
Components in series multiply: two 99.9% services give ~99.8%. Redundancy in parallel raises availability.
Cheatsheets
The whole topic on one page. Click to open full screen.
Sources & further learning
Videos, courses, docs and books I recommend for this topic.
Related topics
Design: URL Shortener
The classic warm-up — ID generation, read-heavy caching, redirects at scale and analytics without slowing the hot path.
Design: Distributed Rate Limiter
Protect APIs and tenants — algorithms (token bucket, sliding window), distributed counters in Redis, and where to enforce limits.
CAP & PACELC Theorems
During a network partition you choose consistency or availability; when there's no partition you still trade latency against consistency.
Caching Strategies
Cache-aside, read/write-through, write-behind; TTLs, invalidation, stampede protection and multi-layer caches (browser → CDN → app → DB).