How chat apps and Netflix actually work

1 The 5-step design approach

Whenever you're handed a "design X" question, follow the same recipe. The first four steps are about understanding the problem (spend ~20–25 minutes here), and only then do you start the actual system design (~20–25 minutes). Today we'll use the recipe to design a messaging application.

2 Problem statement & reasoning by analogy

Our prompt is "Design a messaging application." That's huge and vague. The trick is to reason by analogy: find existing companies/systems that offer a similar product, which gives you an overview of the umbrella of different scopes you could pick from.

What not to build

Just as important: scope out the things that look similar but are a different problem entirely. Naming these shows you understand the boundary.

3 Functional requirements (the MVP)

A functional requirement is some feature/functionality you offer to the client (a user, or another internal system). You explicitly expose an API / write code for each one. The guiding sentence is: "Who is doing what, and based on that, what happens?"

Actors

• Users — the sender and the receiver.
• Organisation (only for enterprise software like Slack) — relationships like org ⇔ admin ⇔ HR.

Whenever you think of features, think of what things the actors can do in the app.

MVP feature 1 — A user can send messages to other users

sendMessage(
  sender_id:    uuid,
  recipient_id: uuid,
  message:      json
): ack | failure          # user-facing

Message status

Users should be able to see the status of the messages they've sent:

Rich messages

• React to messages with Emojis
• Videos / Images
• Gifs

MVP feature 2 — Receive & view past messages (most recent first)

Users should be able to receive messages and view the past messages in a conversation, most recent first. Users should only be able to view messages of a conversation they're a part of.

getMessages(
  user_id:          uuid,
  conversation_id:  uuid,        # identifies the conversation (Tarun+Sanjana, Harini+Pallavi, …)
  pagination_offset: int,
  pagination_limit:  int
): list of messages       # user-facing

Updating/deleting already-sent messages is extremely hard to do correctly, as is full profile management — so they're future scope, not MVP.

MVP feature 3 — View your conversations (most recent first)

getConversations(
  user_id:          uuid,
  pagination_offset: int,
  pagination_limit:  int
): list of conversation   # each: {conversation_id, name, unread count,
                          #        quick summary of the latest message}  -- user-facing

Future scope (v1+) — good to have, but not the MVP

• Notifications — in-app and push.
• Broadcast the same message to multiple people (batch processing).
• Groups — users can participate & send messages inside groups. A single group can potentially have 100,000+ users:

  sendMessageGroup(sender_id: uuid, group_id: uuid, message: json): ack | failure
  getMessagesGroup(user_id: uuid, group_id: uuid,
                   pagination_offset: int, pagination_limit: int): list of messages

• Multi-device support — extremely hard to do correctly (needs CRDTs).
• Group management — creating, adding people, permissions, group description.
• Contacts — suggest / load contacts from the user's phone.
• Message forwarding.
• Start / delete a conversation, and mark as spam — start a new conversation by selecting a recipient from contacts/friends.
• Syncing messages between the app cache & the cloud.
• End-to-end encryption.

4 Non-functional requirements (design goals)

A non-functional requirement is an additional constraint/behavior you want the system to adhere to — not a feature you offer the client. For a messaging app, three matter most: message order, idempotency, and the consistency / availability / latency trade-off.

4.1 Message order

Message ordering must be maintained: the recipient must see messages in the same order the sender sent them.

4.2 Idempotency

Messaging apps are typically mobile-first, and mobile users don't always have perfect internet. So these apps frequently deal with network failures. In a lot of scenarios, the frontend is coded to automatically retry sending if it fails — we don't want the user to re-type their whole message.

<input id="message" type="text" placeholder="send a message" onkeydown="sendMessage" />
<script>
    function sendMessage() {
        message = document.getElementById("message").text;
        sendWithRetries(message);
    }

    function sendWithRetries(message) {
        fetch("/backend-api-url/sendMessage", "POST", {
            message: message
        }).then(() => console.log("success"))
          .error(() => {
              // automatic retry after 1 second
              setTimeout(() => sendWithRetries(message), 1)
          })
    }
</script>

The fix is to make the operation idempotent.

Any WRITE (post/update/delete) API endpoint should be idempotent if your frontend can retry.

4.3 Consistency vs Availability vs Latency (PACELC)

Don't jump to an answer. Reason it out by asking the right questions first.

Q: What is "the data"? The messages that are being sent.

Q: What does eventual consistency (stale reads) mean here? Messages out of order on the recipient's side, or not being able to see the latest message. Two concrete cases:

Q: Can we afford stale reads here? No, we cannot. Q: What does data loss mean? Sender sent a message, the server ACK'd, but the message actually got lost by the backend. Q: Can we afford data loss? Very, very bad. No.

Our latency requirement: Low latency < 5 seconds.

5 Scale estimation

The scale will dictate our design choices. Let's estimate.

Users & messages per second

Assume total users = 2 billion (web/planet scale).

Assume average 10–20 messages / active-user / day. Taking 20:

Total messages/day  = 20 messages/user/day × 2 billion users
                    = 40 billion messages / day

Avg messages/second = 40 billion / day
                    = 4 × 10¹⁰ messages / (10⁵ seconds)
                    = 4 × 10⁵ messages / second
                    = 400,000 messages / second

Per-API load

Peak load

Assume peak = 5× average during global events (Covid, New Year): ≈ 2 million messages/second.

Storage over 20 years

What's the data in one message?

message: {
    sender_id:                            uuid (16b)
    receiver_id / group_id / conversation_id: uuid (16b)
    message_id:                           uuid (16b)
    text:                                 string (200b avg)
    when:                                 timestamp (8b)
    where:                                geolocation (16b)
    delivery_status:                      enum (1b)
    attachment_url:                       string (200b avg)
}

Average size ≈ 500 bytes/message:

Data/day  = 500 bytes/message × 40 billion messages/day
          = 20 trillion bytes/day
          = 20 TB / day

Data/20yr = 20 TB/day × (20 × 365 days)
          = 20 TB/day × ~10,000 days
          = 200 Petabytes

Read-heavy or write-heavy?

6 System design — idempotency & message order

Now we start the actual system design. Two NFRs get solved together on the client side: idempotency and ordering.

Client-generated message ids: UUID v7

Each message gets a unique message_id generated on the client. We use UUID v7:

• universally unique, decentralized, sparse, and roughly sortable (time-ordered)
• a widely accepted spec with popular implementations in every major language
• 128-bit, high performance

Idempotency via client-side id

Whenever the backend receives a sendMessage call, it stores the message only if the message_id isn't already in the DB. If it is already there, it ignores the request but still returns a success response (so the frontend stops retrying).

<input id="message" type="text" placeholder="send a message" onkeydown="sendMessage" />
<script>
    function sendMessage() {
        var content = document.getElementById("message").text;
        var message = {
            id: uuid_v7(),     // this id stays UNCHANGED across retries
            content: content,
            timestamp: new Date(),
        }
        sendWithRetries(message);
    }
    function sendWithRetries(message) {
        fetch("/backend-api-url/messages", "POST", {
            message: message
        }).then(() => console.log("success"))
          .error(() => {
              // automatic retry after 1 second
              setTimeout(() => sendWithRetries(message), 1)
          })
    }
</script>

Because the id is generated once (before any retry) and reused, a retried request carries the same id — so the backend recognizes the duplicate and refuses to store it twice.

Message order — why timestamps and backend queues fail

Solution: a message chain

Every new message contains the previous_message_id inside it. On the recipient's side, the app will not show a message until its previous message has also been received & shown. If a message appears out of order, the app shows waiting-for-messages until all previous messages arrive.

<input id="message" type="text" placeholder="send a message" onkeydown="sendMessage" />
<script>
    var previous_id = null;
    function sendMessage() {
        var content = document.getElementById("message");
        var id = uuid_v4()
        var message = {
            previous_id,         // id of the last message that I sent
            id: id,
            content: content,
            timestamp: new Date(),
        }
        previous_id = id;
        sendWithRetries(message);
    }
</script>

7 Sharding the data

We need sharding (Topic 5). The big question is the sharding key. Two candidates: user_id and conversation_id.

Option A — shard by user_id

All data (messages sent & received) of a particular user lives within the same shard.

Jinesh's shard:
  row_id  user_id  id  sender_id  recipient_id  message
  123     Jinesh   1   Jinesh     Anika         Hi

Anika's shard:
  row_id  user_id  id  sender_id  recipient_id  message
  321     Anika    1   Jinesh     Anika         Hi

Option B — shard by conversation_id

For groups, conversation_id = group_id. For 1-1 chats, conversation_id = a unique id assigned to a pair of users (just the pair (sender_id, receiver_id)).

The schema

users                    (id, name, age, avatar_url)
conversations            (id, title, description)
conversation_participants (conversation_id, user_id)
messages                 (id, sender_id, conversation_id, message)

8 The illusion of consistency

PACELC says: if we want consistency, we give up availability & latency. We do want consistency. Assume we're sharding by user_id — then every write must go to 2 shards (sender & receiver), and we must keep them consistent.

Option 1 — Two-Phase Commit (2PC)

Write to both shards atomically with 2PC. But it's extremely slow (low throughput, high latency) and gives low availability. Rejected.

Can we instead create an illusion of consistency?

Instead of writing atomically, write the two shards one by one. Running example:

Riya  – "bro can I copy your project submission?" → Aditya
(sender)                                            (recipient)

Attempt: write to Sender shard first

• Failure (low latency): return failure immediately; client can retry. Fine.
• Success: Can we return "success" to the sender now? No! The data isn't on the receiver shard yet — if the receiver does getMessages, they won't see it, even though the sender thinks it's sent. Inconsistency: "bro, I sent it" / "bro, you didn't!"
  • Then write to the Receiver shard. Success (low latency) → return success, done.
  • Failure (high latency): now we're in an inconsistent state (on sender shard, not receiver shard). Retry → sender waits until retry succeeds → very high latency. Rollback → return an error, but the system stays inconsistent until the rollback completes, so we must lock and block the sender's reads → high read latency for the sender.

The fix: write to Receiver shard first

• Failure (low latency): return failure immediately; client (sender) can retry.
• Success: the message has actually been sent — if the recipient fetches now, they see it. Return success immediately (without even writing the sender shard yet) → very low latency.
  • (Async) write to the Sender's shard. Success → done. Fails → just keep retrying until it succeeds. The sender isn't waiting (already ACK'd), so no added latency.

How this beats PACELC on availability & latency

Consistency vs Availability — a network partition:

Harini ----- "Hi" -----> Srinidhi
App Server 1 receives the request.
It CAN reach Srinidhi's shard, but a network partition blocks Harini's shard.

The app server just writes to Srinidhi's shard (recipient) and returns success to Harini. It also drops a message in a task queue to add this to Harini's shard asynchronously. Even during the partition, sendMessage & getMessages stay available → high availability. Srinidhi sees the message (written to her shard); Harini sees it via the frontend cache → (illusion of) immediate consistency. (If the write to Srinidhi's shard fails, return an error and let Harini retry.)

Consistency vs Latency: because we're not trying to write both shards atomically, we never wait for retry/rollback — we return success the moment the recipient shard is written → low latency.

9 Choosing the right database

Recall the load: writes 400,000/sec avg (2M peak), reads 800,000/sec avg (4M peak) — both read & write heavy, and no database is optimized for both.

What the ideal database needs

• High write throughput (400,000 writes/sec avg)
• No joins needed — we only "find all messages of Akshay (within a conversation)," never "all messages of all of Akshay's friends"
• No search needed (if needed later, provision another DB in a separate microservice)
• Paginated reads (by timestamp)
• No transactions needed
• Disk persistence

10 The cache & routing

We decided to absorb reads in a cache. Let's design it with the 5-step cache design process:

Type — why a local cache wins

• A single global cache? No — we need lots of cache, so it must be distributed (local or global).
• Network overhead per read? getMessages only reads a few messages → low network overhead, so a global cache could work.
• One cache server can't hold all the data → we'd need a distributed global cache.
• But: with a global cache, the app server for getMessages does nothing but forward the request → we'd have 2× the servers (app + cache) with half doing nothing. And read-your-write consistency becomes hard (it's trivial with a local cache, since local writes succeed).

Invalidation — write-through, cheaply

We want immediate consistency, but not the latency it usually costs. With a separate app server and cache server, immediate consistency needs 2PC (keep cache & DB in sync — two places). With a local cache, writing to the cache always succeeds, so no 2PC is needed. Hence we get write-through invalidation (immediate consistency) without high latency.

Eviction

LRU eviction.

Routing — consistent hashing for stateful servers

Our app servers are now stateful: if a server holds Harini's messages in cache, Harini's requests must always go to that server. So we route with consistent hashing.

• getMessages(user_id, conversation_id) → goes to my shard and my app-server+cache, because all my messages are there.
• sendMessage(sender_id, recipient_id) → routed to the recipient shard & app-server+cache first (data written there), then (async, via a message queue) replicated to the sender shard & app-server+cache. (This is exactly the "recipient-first" trick from Topic 8.)

3 Video streaming, Netflix-style

Netflix has to play video smoothly on a phone in a tunnel, a TV on gigabit fiber, and everything in between — for hundreds of millions of people worldwide. The magic comes from four ideas working together: many encodings, adaptive bitrate streaming, chunking, and CDNs.

Storing one video as many resolutions & bitrates

When a studio uploads a movie, Netflix doesn't keep one file. It runs the video through a transcoding pipeline that produces many versions of the same content at different combinations of resolution (240p, 480p, 720p, 1080p, 4K…) and bitrate (how much data per second — higher bitrate = better quality, bigger file). A single title might have a dozen or more such renditions.

Chunking — cut every rendition into tiny segments

Each rendition is sliced into short segments (a.k.a. chunks), typically 2–10 seconds long. So for one movie you end up with: (many renditions) × (many little segments), plus a small text file called a manifest (the index that lists every segment of every rendition and where to fetch it).

#EXTM3U
# 480p stream, ~800 kbps
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=854x480
480p/index.m3u8
# 720p stream, ~2.8 Mbps
#EXT-X-STREAM-INF:BANDWIDTH=2800000,RESOLUTION=1280x720
720p/index.m3u8
# 1080p stream, ~5 Mbps
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080
1080p/index.m3u8

# 720p/index.m3u8
#EXT-X-TARGETDURATION:6
#EXTINF:6.0,
seg_00001.ts
#EXTINF:6.0,
seg_00002.ts
#EXTINF:6.0,
seg_00003.ts

Adaptive Bitrate Streaming (HLS / DASH)

Adaptive Bitrate Streaming (ABR) is the trick that makes video "just work." The player constantly measures your current network speed and, for each upcoming segment, chooses the highest-quality rendition your connection can handle. Signal drops? Next segment comes from a lower rendition — no spinning wheel. Signal recovers? It steps back up. Because all renditions are chopped at the same segment boundaries, switching is seamless.

The two dominant protocols are HLS (HTTP Live Streaming, from Apple) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP). Both do the same job: a manifest plus HTTP-fetched segments, with the client deciding which quality to grab.

Thumbnails & preview images

Those little frames that appear when you scrub the timeline are thumbnail sprites — a grid of tiny still images generated during transcoding (e.g. one frame every 10 seconds), packed into a single image plus a tiny map of "time → which tile." Scrubbing just shows the right tile; no video decoding needed. Box-art and hero images are likewise pre-generated stills served from the CDN.

The role of CDNs for global delivery

A CDN (Content Delivery Network) is a fleet of caching servers spread across the globe, close to users. Instead of every viewer pulling video from one faraway data center, the segments are copied to an edge server near them. Netflix runs its own CDN called Open Connect, placing storage boxes inside internet providers' networks — sometimes literally in your ISP's building. Result: your movie travels meters, not oceans.

# Client picks quality for the NEXT segment based on measured speed + buffer
def pick_rendition(measured_kbps, buffer_seconds):
    if buffer_seconds < 5:             # running low — play it safe
        return lower_rendition()
    if measured_kbps > 5000:           # plenty of room
        return rendition("1080p")
    elif measured_kbps > 2800:
        return rendition("720p")
    else:
        return rendition("480p")

4 Live broadcast to millions

A live event — a cricket final, a product launch, a streamer with millions watching — is the hardest scaling problem in this session. The content is the same for everyone and it's being created this second. The core challenge is fan-out: one source, millions of simultaneous viewers.

How live differs from on-demand

The live pipeline

It reuses the streaming machinery (chunks, manifests, ABR, CDN) but adds a real-time front end:

Edge caching is what makes the fan-out survivable

Here's the saving grace: even though there are millions of viewers, they all want the exact same segment right now. So an edge server fetches segment #4021 from the origin once, then serves that one copy to the 200,000 viewers attached to it. This is the cache hit ratio doing heavy lifting — the origin sees thousands of requests, the edges absorb millions. Without edge caching, the origin would melt instantly.

The latency trade-off

Chunking creates a tension. Bigger segments (say 6s) are efficient and smooth but mean viewers are always ~10–30 seconds behind real life (each player buffers a few segments for safety). For a movie that's fine. For a live sports match where your neighbor cheers before your screen shows the goal, it's painful.

★ Putting it all together

Two very different products, but look how many ideas they share — sharding for scale, persistent connections and fan-out for delivery, and pushing data physically close to the user.

Quick self-check

📚 References & Further Reading

Class material

• 📄 Original class notes / handout (Google Doc) — open the shared class material for this session.
• Class handout: "[SST-2028] Case Study: Messaging Apps (FB Messenger, Whatsapp, Slack)".

Papers, docs & deep dives

• 📄 MDN — The WebSocket API — persistent two-way connections for real-time message push.
• 📄 MDN — Live streaming web audio and video (HLS / DASH) — adaptive bitrate streaming fundamentals.
• 📄 The Netflix Tech Blog — deep dives on Open Connect, transcoding, and global video delivery.
• 📄 Apache Cassandra — Data Modeling — partition/clustering keys behind wide-column message storage.