Learn System Design in a Single Post: Complete Tutorial From Scaling and Caching to CAP and the Design Interview

System design is the study of how real, large systems are built: how a URL shortener scales to a billion lookups, how Twitter serves a timeline, how a chat app stores history, how a ride-hailing app matches riders to drivers. It’s less about any one technology and more about a set of building blocks and the tradeoffs between them — the language senior engineers speak when they design for scale. This single post teaches the whole subject in five stages, with hand-drawn diagrams and the reasoning behind each choice.

Learning Roadmap

System Design Roadmap

The roadmap moves from clarifying what you’re building (Stage 1), through the standard building blocks (Stage 2), to how data scales (Stage 3), the patterns that hold it together (Stage 4), and the interview process that ties it all together (Stage 5).

Stage 1 — Requirements

A design interview (or real design) starts with requirements, not boxes. Two kinds:

Functional — what the system does. “Users shorten URLs; anyone can resolve a short URL.”
Non-functional — scale, latency, availability, consistency, durability. “100M URLs, 10:1 read:write, <200ms p99 read, 99.9% uptime.”

The numbers that matter

Memorize these back-of-envelope numbers (from Jeff Dean); they let you reason about scale without a spreadsheet:

	Latency
L1 cache	0.5 ns
Main memory	100 ns
SSD random read	100 µs
1 Gbps network round-trip	0.5 ms
Datacenter round-trip	0.5–1 ms
HDD seek	10 ms
Cross-region network	30–100 ms

And the scale numbers: a single box handles ~1k QPS for simple work; a good MySQL primary ~10k writes/sec; Redis ~100k ops/sec; a single HTTP server ~1k–10k RPS depending on the work.

Read vs write ratio

This determines almost everything: a read-heavy system (Wikipedia, a URL shortener’s redirect path) screams “cache it”; a write-heavy system (telemetry, logs) screams “buffer + batch + queue.” State the ratio early.

Stage 2 — Building Blocks

Every large system is an assembly of a handful of standard components. Learn what each does and when to reach for it.

System Design Building Blocks

Load balancer

Distributes incoming traffic across multiple backend instances so no single box is overwhelmed and so a dead box is routed around. Layer 4 (TCP/UDP — e.g. HAProxy, AWS NLB) or layer 7 (HTTP — e.g. nginx, AWS ALB, which can route by host/path). Strategies: round-robin, least-connections, consistent hashing (for cache affinity).

Cache

A fast key-value store (Redis, Memcached) sitting in front of a slower store. Caches turn a 50ms DB read into a 0.5ms cache read — but introduce cache invalidation (when does stale data get refreshed?) and eviction policy (LRU/LFU/TTL). A cache hit rate of 80%+ typically slashes load and latency dramatically.

Pitfall: A cache that warms slowly after a cold start or a flush can overwhelm the DB behind it (cache stampede). Mitigate with request coalescing, jittered TTLs, or a “stale-while-revalidate” policy.

CDN

A content delivery network serves static assets (images, CSS, JS, video) from edge nodes near the user, so the request never hits your origin. Invalidating CDN content (when you deploy a new asset) is the hard part — use content-hashed filenames (app.3a9f.js) so new deploys are new URLs and never need invalidation.

Message queue

Decouples producers from consumers so a burst of writes doesn’t crash a slow consumer, and so consumers can be scaled independently. Kafka (durable log, replay, high throughput), RabbitMQ (rich routing, ack/nack), SQS (managed). Used for: async work (image processing), event fan-out, buffering writes.

Database

Durable storage. The first big choice is SQL (Postgres, MySQL — relational, ACID, joins, schemas) vs NoSQL (DynamoDB, Mongo, Cassandra — flexible schema, horizontal scale, tunable consistency). Pick SQL by default; reach for NoSQL when the access pattern is a simple key-value or when you need horizontal scale that SQL sharding can’t easily give.

Search index

Full-text and faceted search needs an inverted index — Elasticsearch/OpenSearch or the DB’s built-in full-text (Postgres tsvector). Don’t LIKE '%term%' in SQL at scale; it’s a full table scan.

Object storage

Files, images, backups that don’t belong in a database: S3, GCS, Azure Blob. Immutable, cheap, infinitely scalable. Serve via CDN.

API gateway

A single entry point that offloads cross-cutting concerns — auth, rate limiting, request routing, TLS termination, observability — so your services don’t repeat them. See the REST architecture for what sits behind it.

Stage 3 — Data at Scale

Replication

Copy data so reads scale and the system survives a failure. Two patterns:

Primary-replica — one writable primary; replicas serve reads. Writes go to the primary and propagate asynchronously (eventual consistency) or synchronously (strong, slower).
Multi-primary — several writable nodes; conflict resolution is the hard part (last-write-wins, CRDTs, app-level merge). Avoid unless you must.

Sharding (partitioning)

Split one logical database into many physical shards by a shard key — e.g. user_id % N. Each shard holds a slice of the data, so reads/writes scale horizontally. The hard parts: hot shards (one key gets all the traffic — celebrity accounts), cross-shard joins (don’t), and rebalancing when you add a shard (consistent hashing minimizes movement).

Pitfall: Choosing the wrong shard key is the most expensive system-design mistake. A key that fans reads evenly but groups related data together is ideal; a key that puts all of one tenant’s data on one shard while another tenant is cold creates hot shards. Pick the shard key before you have data; re-sharding in production is painful.

CAP theorem

In a distributed system under a network partition (P, which is unavoidable), you can guarantee either Consistency (C) or Availability (A), not both:

Tradeoffs: CAP, Consistency, Latency

CP — refuse reads/writes during a partition to stay consistent (e.g. HBase, a bank ledger). Users see errors but never stale data.
AP — keep serving (possibly stale) data during a partition (e.g. DynamoDB, Cassandra). Users always get an answer, but it may be behind.

Pitfall: “Pick two of three” is a simplification. In practice you tune the C-A balance per operation: a payment is CP; a “view count” is AP. The partition isn’t a checkbox — it’s the normal mode of a distributed system.

Consistency models

Strong / linearizable — every read sees the latest write; expensive (synchronous coordination), low availability under partitions.
Eventual — given no new writes, all replicas converge; fast, high availability, but reads can be stale.
Causal / read-your-writes / session — middle grounds a user-facing app often wants (“I see my own write immediately”).

Pick the weakest consistency the user can tolerate — it buys availability and latency.

Stage 4 — Patterns

The reusable solutions that hold a scaled system together:

Rate limiting — token bucket or leaky bucket per user/IP; protects the system and enforces fair use. Do it at the edge (API gateway).
Circuit breaker — stop calling a failing downstream service after N failures, then probe periodically; prevents cascading failures. (The no-mistakes and TREK posts cover resilience patterns.)
Idempotency — make retries safe. An idempotency key on writes means a double-send (network retry) doesn’t double-charge. Essential for payments.
CQRS — separate the write model (optimized for validation/consistency) from the read model (denormalized for fast queries). The write side emits events; the read side is a projection.
Event sourcing — store the events (state changes) as the source of truth; the current state is a fold over them. Enables replay, audit, and time travel, at the cost of complexity.
Saga — distributed transactions without two-phase commit: a sequence of local transactions with compensating actions on failure.
Bulkhead — isolate resources (thread pools, connections) per service so one slow dependency can’t exhaust the pool and starve everything else.

Stage 5 — The Interview + Tradeoffs

A system-design interview is a 45-minute simulation of a real design conversation. Structure your answer:

The Design Interview Process

Clarify requirements (5–8 min) — functional AND non-functional. Ask about scale, users, latency targets, consistency needs. Never start drawing before you know what you’re building.
Back-of-envelope estimate (3–5 min) — QPS, storage over 5 years, bandwidth, cache hit rate. Shows you can reason about scale.
Sketch the architecture (10–15 min) — draw the boxes: client → CDN/LB → app → cache → DB → queue. Keep it simple first; don’t reach for Kafka on step one.
Deep-dive the bottleneck (10–15 min) — pick the hardest part (usually the database: schema, sharding key, index strategy) and justify your choices. This is where you’re graded.
Tradeoffs + bottlenecks (5 min) — name the single point of failure, the CAP choice, the scaling plan for 10× growth. Acknowledging tradeoffs is the senior signal.

Communication is the grade

The interviewer isn’t grading your knowledge of Kafka — they’re grading how you think aloud, handle ambiguity, justify choices, and react to new constraints (“now make it multi-region”). Narrate your reasoning; say “I’m picking X because Y, trading off Z.” A confident, structured narrative with honest tradeoffs beats a perfect-but-silent sketch.

Quick-Start Checklist

Memorize the latency numbers — they’re the unit of back-of-envelope math.
Know every building block (LB, cache, CDN, queue, DB, search, object store, gateway) and when each applies.
State the read/write ratio early — it drives the design.
Pick the shard key before you have data — it’s the costliest decision to reverse.
Learn CAP cold and which side your system sits on (and that it can differ per operation).
Design for the tail (p99), not the mean — the mean hides the worst user experience.
Make writes idempotent — retries are inevitable; double-charge is not.
Structure the interview: clarify → estimate → sketch → deep-dive → tradeoffs.
Narrate tradeoffs out loud — “I’m choosing X, giving up Y, because Z.”
Practice on a known set — URL shortener, Twitter timeline, chat, rate limiter, web crawler, ticket booking. The patterns repeat.

Common Pitfalls

Jumping to boxes before requirements — you’ll design the wrong thing. Clarify first.
Over-engineering on step 3 — reaching for Kafka, sharding, and CQRS before you’ve drawn a load balancer. Start simple; add complexity only when a bottleneck demands it.
Forgetting the shard key — the #1 data-scaling mistake; pick it deliberately.
Caching without invalidation — a cache that serves stale data forever is a bug. Define the freshness contract.
Ignoring the single point of failure — one primary DB, one cache node, one region. Name them and address them.
Confusing availability with durability — a system can be “up” (available) yet lose data (durability). Backups + replication give durability; multi-AZ gives availability.
Two-phase commit at scale — it blocks and kills availability. Use sagas + idempotency instead.
Designing for the mean latency — the p99 user experience is what you ship.

Learn System Design in a Single Post: A Complete Tutorial From Scaling and Caching to CAP, Sharding, and the Design Interview

Learn System Design in a Single Post: Complete Tutorial From Scaling and Caching to CAP and the Design Interview

Learning Roadmap