WhatsApp

Understanding the Problem

Picture trying to reach a friend who keeps moving between coffee shops. You can shout as loud as you want in the shop you're standing in — it does nothing if they've already walked two doors down, unless something tells you exactly where they went. That's the real problem underneath 1:1 and small-group messaging: a message has to find whichever one specific machine, out of potentially hundreds of interchangeable ones, currently holds the recipient's live connection.

Users send text messages inside a conversation (1:1 or small group). Participants who are online should receive new messages within a couple hundred milliseconds; anyone should be able to scroll back through history regardless of whether they were online when a message was sent. With that in mind, here's exactly what we're on the hook to build.

Functional Requirements

Those are the functional guardrails — now the constraints that decide what "good" actually means for this design.

Non-Functional Requirements

• Message delivery latency under 200ms at p99 for an online recipient.
• Assume 50 million daily active users, averaging 40 messages/day each — roughly 2 billion messages/day, or about 23,000 messages/sec on average, with bursts several times higher at peak hours.
• Message delivery must be at-least-once — a dropped message is a worse outcome than a rare duplicate a client can dedupe by message ID (the actual trade-off behind that choice is worth defending explicitly, and gets its own treatment in Trade-offs below).

Two of those numbers — the message volume and the at-least-once guarantee — are what the rest of this design leans on hardest. Let's put real weight behind the volume first, since it decides how this gets stored.

Capacity Estimation

At 2 billion messages/day, and assuming roughly 200 bytes per stored message (a modest body plus IDs and timestamps, with database overhead), message history alone grows by about 400GB/day — on the order of 150TB a year. That volume, sustained indefinitely, is well past what a single database instance can hold or serve, which is why message storage has to be sharded — by conversationId, the same key that already determines pub/sub partitioning below, so a conversation's full history stays on one shard instead of being scattered across many. At 23,000 messages/sec average — climbing to roughly 70,000 at a typical 3x peak multiplier — the write path also needs a storage layer built for sustained appends — an append-heavy, horizontally-scalable store (e.g. Cassandra) rather than a single relational primary. If you haven't worked with Cassandra specifically, that's fine — name whichever wide-column or append-optimized store you know; nothing here demands a particular vendor.

Message volume isn't the only number capacity estimation needs — connections are long-lived, so what matters just as much is how many are open at once, not just how many messages flow through per second.

If roughly a fifth of the 50 million daily users are online at once, that's 10 million concurrent WebSocket connections a gateway fleet has to hold at the same time — the concrete number behind "a single machine can only hold so many" below, and the real reason gateways shard across many machines rather than just handling more messages per second.

With the storage shape settled, here's the one thing that actually needs to be persisted.

Core Entities

type Message = {
  id: string;
  conversationId: string;
  senderId: string;
  body: string;
  sentAt: Date;
};
type Conversation = { id: string; participantIds: string[] };

Two small types are all this design needs to persist — everything else below is about moving a Message from one client to another, not about what gets stored.

API Interface

GET  /conversations/:id/messages?before=...   -> Message[]

WS   /ws
  send:    { type: "message", conversationId, body }
  receive: { type: "message", message: Message }

With the contract pinned down, here's how a message actually gets from one client to another.

High-Level Design

Connections are long-lived, so a single machine can only hold so many — clients get sharded across many gateway processes behind a connection load balancer. That means the sender and a given recipient are very often connected to different gateway instances, which is the actual hard problem this design solves — exactly the coffee-shop problem from the top of this lesson, now with a concrete fix on the table: a message written on Gateway A has to reach a client connected to Gateway B.

A publish/subscribe layer (Redis Pub/Sub or Kafka, keyed by conversation ID) is what bridges that gap — every gateway subscribes to the conversations its connected clients care about, and publishes are broadcast to all of them. If you haven't worked with either product specifically, that's fine — the mechanism (publish once, fan out to every subscriber) is what matters here, not which one you'd reach for. This is the same high-level shape large real-time messaging systems like WhatsApp and Slack are built on: many sharded connection processes, unified by a fan-out layer underneath.

The database write happens before the publish, so message history is never missing a message that a client received live — the DB is the source of truth, and the pub/sub layer is purely a delivery accelerator for the online case.

Pub/sub is one specific answer to the routing problem, though — not the only one, and not an obviously correct one until you've actually weighed it against the alternatives. That's exactly what the trade-offs below do first.

Trade-offs

Consistent hashing's real upside is precision: Gateway A knows exactly which one gateway to talk to, so nothing is wasted on a broadcast. At least one large real-time messaging platform runs close to this shape in production, using a consistent hash ring to map each conversation to one owning server; when that server fails, a replacement is ready to take over in well under a minute. The cost is what happens when the ring itself changes — every user pinned to a server that goes down has to be reassigned, and in most implementations that means a forced reconnect. A single failure can trigger a wave of reconnections at exactly the moment the system is already degraded.

The pub/sub broadcast layer sidesteps that reconnection storm — no user is permanently pinned to any one gateway, so a crash only drops the connections it happened to be holding, and clients simply reconnect to whichever gateway is available next (the same reconnection path this lesson comes back to at Staff level). What it gives up is precision: a publish reaches every gateway subscribed to a conversation, whether or not that gateway is actually holding a participant, and at large enough group sizes that waste is real. At least one other large chat platform hit this exact cost scaling a similar broadcast model — naively notifying every member of a very large group, one at a time, could take the better part of a second to a couple of seconds for their largest groups, and they needed a dedicated batching layer just to keep fan-out fast.

Chosen: pub/sub broadcast. This design caps conversations at a small group size (see Follow-Up Questions below for what changes past that), not tens of thousands of participants, so the fan-out waste stays small — and avoiding the reconnection storm a hash-ring rebalance causes is worth more here than the precision consistent hashing buys.

With the core routing decision made, a few smaller decisions round out the design.

• WebSocket vs. long-polling: long-polling is simpler to deploy behind some older infrastructure but costs a full HTTP request/response cycle per poll, which doesn't hold up at 23,000 messages/sec of underlying traffic. Chosen: WebSocket, for a persistent connection with much lower per-message overhead.

Holding a connection open indefinitely raises a question long-polling's own request/response cycle answers for free, though: how does either side know the connection actually died, rather than just gone quiet? A TCP-level timeout alone can take minutes to notice a half-open connection, which is far too slow for a 200ms delivery target. Production systems close that gap with an application-level heartbeat instead — the gateway pings each client on a short interval (something on the order of every 15-30 seconds) and expects a pong back within a few seconds; missing enough of those closes the connection and kicks off the reconnection flow covered at Staff level below. It costs a small, constant stream of ping/pong traffic per connected client, which is real at hundreds of millions of connections but bounded and predictable — a trade worth making for detecting a dead connection in seconds instead of minutes.

• Per-conversation ordering vs. global ordering: a single global sequence number across every conversation would make system-wide debugging and audit tooling simple — "what happened, in what order, across the whole system" becomes one sorted scan instead of merging many independent per-conversation logs. But it needs every write, from every conversation, to coordinate through one shared source of truth, turning an otherwise embarrassingly parallel write path into a single bottleneck — and it buys an ordering guarantee no user can actually observe, since a user only ever sees one conversation at a time. Chosen: ordering guaranteed only within a conversation, which is both cheaper and matches what's actually user-visible; if audit tooling genuinely needed a global view later, that's a case for a separate log fed by every conversation's shard, not for coordinating the hot path itself.

• At-least-once + client dedup vs. exactly-once delivery: true exactly-once delivery over an unreliable network isn't actually achievable — if a sender never gets an acknowledgment back, it can't tell whether the message was lost or just the ack was, so it's always choosing between risking a duplicate and risking a drop. Guaranteeing something that behaves like exactly-once in practice means coordinating a distributed transaction (or an idempotency ledger) across the write, the publish, and the client's own processing — real coordination cost paid on every single message. Chosen: at-least-once, with the client deduping by the message id already defined in Core Entities — cheap on the hot path, since nothing needs to coordinate before sending an ack, and the dedup cost moves to the client, which only has to check one field against messages it's already rendered.

Final Design

Put every piece from above together and this is the shape it settles into:

Each piece here scales — or fails — on its own terms:

• Gateways scale horizontally purely to hold more concurrent WebSocket connections; none of them hold any state that survives a restart, which is exactly what lets the pub/sub broadcast model chosen above tolerate a gateway crash without losing anything durable.
• Pub/Sub is the one piece every gateway shares, and its only job is best-effort fan-out to whichever gateways are currently subscribed — it never needs to be durable, since the database write it follows already is.
• The database sits underneath everything as the actual source of truth — every message lands here before it's ever published, which is the one fact this whole design's delivery guarantee rests on.

Operations & Observability

A design that only handles the happy path of "message goes in, message comes out" isn't finished — you also need to know it's degrading before a user notices messages arriving late or not at all. Raising this without being asked is one of the clearest signals that separates a senior or staff-level answer from one that stops at "and then it works."

Watch message delivery p99 for online recipients first, tracked separately from any offline/history-pull latency — this is the number the 200ms NFR is actually about, and a creeping p99 usually points at the pub/sub layer or a gateway's own fan-out backing up, not at the database write, which already has to finish before the publish even starts.

Pair it with WebSocket reconnect rate. A steady low baseline is normal — phones sleep, wifi drops — but a spike usually means either a deploy cycling gateways too aggressively or a heartbeat interval that's too tight for real network conditions. Every reconnect carries a downstream cost worth watching on its own: reconnection-backlog size, how many messages the average reconnecting client has to pull from history through its last-seen cursor. A rising average is an early sign clients are staying disconnected longer than this design assumed, well before it ever shows up as a user complaint.

Finally, watch pub/sub fan-out cost per publish — how many gateways a typical publish actually reaches, versus how many are truly holding a participant. This is the direct cost side of the routing decision made above: it should stay small at this design's group-size cap, and a climbing number is the earliest signal that some conversation has outgrown the scale that choice assumed.

One operational wrinkle worth naming: a pub/sub layer that buffers per-subscriber (Redis Pub/Sub does this) can hit a slow-consumer limit and disconnect that subscriber outright rather than grow its memory without bound. A gateway that falls behind can lose its subscription this way — but because the database write always lands before the publish, nothing is actually lost, only delayed until that gateway reconnects and the client resyncs from history.

Level Expectations

Here's roughly how the bar moves as you climb this ladder, from a working design to one that survives production.

Mid-level: proposes the WebSocket-plus-database design and can explain why a single gateway instance alone isn't enough for delivery to work across all clients.

Senior: proactively introduces the pub/sub fan-out layer, can weigh it against a naive load balancer or a consistent-hash-pinned-gateway alternative by naming a real cost on each side rather than asserting pub/ sub is obviously correct, explains why the DB write has to happen before the publish, and names message delivery p99 and WebSocket reconnect rate as what they'd actually watch in production, not just what they'd build.

Staff: reasons about reconnection — what a client does after a dropped connection to avoid missing messages sent during the gap (e.g. a last-seen-message cursor reconciled against history on reconnect) — and names the application-level heartbeat mechanism a real system uses to detect a dead connection in the first place, not just what happens after. Can argue for at-least-once-plus-client-dedup over attempting exactly-once delivery by naming the actual coordination cost exactly-once would add to every message, and reasons about channel granularity at large group sizes (per-conversation vs. per-user pub/sub channels) as a concrete lever once a single group's fan-out cost outgrows this design's assumptions.

Follow-Up Questions

Try It Yourself