How big websites stay fast for millions of people

1 What is High-Level Design (HLD) & why it matters

High-Level Design (HLD) is the practice of planning how a whole software system fits together before you write the detailed code. It describes the big building blocks — servers, databases, caches, load balancers, queues — and how they talk to each other to serve many users at once, reliably and quickly. It deliberately stays "high level": it cares about boxes and arrows, not about which exact line of code runs inside each box.

HLD vs LLD — two different "design" jobs

In our class the very first distinction was between two kinds of design, and people mix them up constantly. Low-Level Design (LLD) is the architecture of the code itself. HLD is the architecture of the servers / machines that run the code.

Writing a program vs designing a system

This is the single most important mindset shift in this whole subject, so let's nail it. Writing one program means solving a problem once, for one run, usually on one machine. Designing a system means making that solution work for millions of users, continuously, across many machines, even when parts of it fail.

Functional vs non-functional requirements

Before designing anything, you must know what the system has to do. Requirements come in two flavours, and beginners almost always forget the second one.

A functional requirement describes what the system does — its features. "Users can post a photo." "Users can search for a product." "A payment can be refunded." If you can phrase it as "the system shall let the user ___", it's functional.

A non-functional requirement (NFR) describes how well the system must do it — its qualities under load. These are the heart of HLD. The four you'll meet constantly:

2 How the internet works (end to end)

To design systems that live on the internet, you need a mental model of how a request actually travels. Let's build it from the ground up with four core ideas: the client/server model, DNS, IP, and TCP — then watch them work together when you press Enter on a URL.

The client/server model

Almost everything online follows the client/server model. A client is the program that asks for something — your browser, a phone app. A server is a program (running on some machine in a data centre) that listens for requests and responds. The client says "please give me this page"; the server says "here it is." Your laptop is a client; YouTube's machines are servers.

DNS — the internet's phone book

Humans remember names like google.com; computers route traffic using numbers called IP addresses (e.g. 142.250.72.78). The DNS (Domain Name System) is the giant, distributed lookup service that translates a human-friendly name into the machine's IP address. This translation is called DNS resolution.

Who gives a website its IP? Registrars & ICANN

There's a step before DNS can even answer. Say Joshua wants to launch a website. First he must purchase a domain. He does that through a domain registrar — an intermediary that facilitates the purchase and management of domain names. Examples: GoDaddy, Namecheap, Cloudflare. The registrar then relays to ICANN (the central naming authority) that Joshua has bought this domain, and ICANN stores the IP address of Joshua's machine against the domain name he purchased.

You also need an Internet Service Provider (ISP) to be on the internet in the first place — the company that provides you the connection. In India that's Airtel / Jio / Vi; elsewhere ACT / Hayai / Starlink / Comcast. And an IP address is the (kind-of) unique address assigned to each machine connected to a computer network.

DNS — who actually maintains it?

So DNS is the distributed answer to that bottleneck. DNS maintains a list of IP addresses for each domain name (a name can map to many IPs). ICANN remains the central authority, but the actual DNS servers are maintained by lots of different entities — and in fact anyone can run a DNS server:

• Software companies — Google, Amazon, Meta
• Governments, militaries, research organizations, educational institutes
• ISPs — Airtel, BSNL

You can connect to any DNS server you like (e.g. Google Public DNS). And if you never configure one yourself? Your ISP also runs a DNS server, and that one is already configured in your router by default — so lookups still work.

IP — the addressing & delivery layer

The IP (Internet Protocol) is the set of rules for addressing and routing. Every machine on the internet has an IP address, like a postal address. IP's job is to get a packet of data from one address to another, hopping through many routers along the way. IP by itself is "best effort" — it doesn't guarantee packets arrive, or arrive in order. That guarantee is TCP's job.

TCP — the reliable courier

TCP (Transmission Control Protocol) sits on top of IP and adds reliability. It chops your data into numbered packets, makes sure each one arrives (re-sending any that get lost), and puts them back in the correct order at the other end. Before sending real data, the two sides do a quick three-way handshake (SYN → SYN-ACK → ACK) to agree they're both ready — like saying "Hello? / Yes, I hear you / Great, let's talk."

What happens when you type a URL and hit Enter

This is the classic interview question, and it ties everything above together. Here's the whole request → response lifecycle:

Let's narrate it once in words. You type google.com and hit Enter (1). Your computer asks DNS for the IP address behind that name (2). With the IP in hand, your machine opens a TCP connection to that server using the three-way handshake (3). Over that connection it sends an HTTP request — essentially the text "GET / please" (4). The server runs its code, perhaps querying a database, to build the answer (5). It sends back an HTTP response: a status code plus the page's HTML or data (6). Finally your browser reads that HTML, fetches any extra bits (images, CSS, scripts) the same way, and paints the page on screen (7).

GET /search?q=cats HTTP/1.1
Host: www.google.com
User-Agent: Mozilla/5.0
Accept: text/html

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Content-Length: 1256

<!DOCTYPE html><html>…the page…</html>

3 Case study: del.icio.us & the idea of an MVP

We anchor everything in a real product. del.icio.us was a simple bookmarking service, launched in 2003 by a creator named Joshua. For context on how early that is: YouTube came in 2004, Amazon AWS in 2006, Google Chrome in 2008.

The motivation

Back then, most people did not have the internet at home, and browser bookmarks were stored locally — the bookmarks you saved on one computer were not available on any other computer. So the pitch was: "Let's create a bookmarking service where people store their personal bookmarks on my website. My users can then access their bookmarks from any system, as long as they have an internet connection."

What's the smallest version worth building? The MVP

You don't build every feature on day one. You build a Minimum Viable Product (MVP) — also called a proof of concept or prototype. The three words each carry weight:

Personal computer vs server

Joshua initially ran the website from his personal laptop. Two problems showed up immediately:

• When Joshua went to sleep and turned his laptop off — the website went down.
• When Joshua downloaded a movie or played a game — the website became extremely slow.

The lesson: a personal computer is for personal use. To run a service you need a server — a dedicated computer whose sole purpose is to provide a (web) service over the internet. Joshua had to stop using that machine for personal use.

Designing the database

A first cut of the schema for the bookmarking service:

users ( id, name, password )

user_bookmarks ( user_id, URL )
  user_id : bigint        (8 bytes)
  URL     : varchar(1000) (1000 bytes)

4-byte integer
  unsigned : 0 … 2^32        (0 … 4 billion)
  signed   : -2^31 … 2^31-1  (-2 billion … 2 billion)

4 Challenges at scale — what breaks at 100 million users

When your app is new, one server handling everyone is perfectly fine. The trouble starts as you grow from 100 users to 100 million. Suddenly that single machine — let's call it the monolith on one box — runs into hard physical limits. Let's see exactly what breaks, with realistic numbers.

The interview question that started it all

This was posed as a real Staff Engineer @ Google interview question, and it's the perfect illustration of "easy turns hard at scale."

input:  cat, dog, apple, laptop, class, high, level, design
output: apple, cat, class, design, dog, high, laptop, level

with open('data.txt', 'r') as f:
    data = f.readlines()

print(sorted(data))

A quick units refresher (so the numbers mean something)

To reason about scale you must be fluent in data sizes. Note the difference between the decimal units (Kilo = 1,000) and the binary ones (Kibi = 2¹⁰ = 1,024).

The shape of the solution — and everything that can go wrong

At a high level the job is simple to state: Collect data ⇒ Sort ⇒ Store the data back. The hard part is that, spread across millions of machines, an enormous number of things can go wrong:

Despite all of these challenges, you must still complete the task — both efficiently and correctly. That tension is the heart of HLD.

The four resources that run out

A single server is just a computer. It has four finite resources, and at scale you slam into each one:

Back to del.icio.us: when does the disk fill up?

Let's make this concrete with our case-study app. del.icio.us went viral — Joshua was suddenly getting millions of new bookmarks each day, on a 2003-era machine with only a 40 GB hard disk. Let's do the napkin math.

1 KB / bookmark  ×  1,000,000 bookmarks / day
= 10^3 bytes  ×  10^6 / day
= 10^9 bytes / day
= 1 GB / day

40 GB  /  (1 GB / day)  =  40 days

Latency creeps in too

Scale isn't only about volume — it's about distance and delay. A user in Australia hitting a server in Virginia pays a physical penalty: data can't travel faster than light. A round trip across the world is easily 150–300 ms just in network travel, before any work is done. Pile on a busy CPU and a slow database query, and your "fast" app feels sluggish to half the planet. As load rises, latency doesn't grow gently — it tends to spike sharply once a resource nears 100%.

5 Vertical vs Horizontal scaling

There are exactly two ways to give a system more power, and knowing the difference is core HLD vocabulary.

Vertical scaling (scaling up)

Vertical scaling means making a single machine more powerful — adding more CPU cores, more RAM, faster disks. You keep one server; you just beef it up. It's wonderfully simple because nothing about your software has to change.

Horizontal scaling (scaling out)

Horizontal scaling means adding more machines and spreading the work across all of them. Instead of one super-server, you run many ordinary servers side by side. This is how every massive system on earth is built — but it introduces a new question: how do you split the work, and how do the machines stay in sync?

1,00,00,000 ₹  /  35,000 ₹  =  285 laptops

Which one should you use?

• Vertical scaling is easy — all you have to do is throw money at it. But it has limits: you cannot scale infinitely. It's bounded by current technology, and it's more costly.
• Horizontal scaling is not limited by current technology — you can scale (pretty much) infinitely. Eventually you have to scale horizontally, because vertical scaling will run out.
• But horizontal scaling is a huge pain — it is extremely difficult. (The entirety of the HLD curriculum is about the challenges of scaling horizontally and how to solve them.)

How far can vertical scaling actually go? (Jan 2025)

To make the "hard ceiling" concrete, compare a typical server to the biggest one money could buy as of January 2025:

Impressive — but still one machine, and therefore still a single point of failure with a real upper bound. That's why planet-scale systems must eventually go horizontal.

The secret ingredient: statelessness

Horizontal scaling only works smoothly if any server can handle any request. That requires the servers to be stateless: an individual server stores no user-specific memory between requests. Each request carries (or can look up) everything needed to be served by any machine.

6 Introduction to load balancing

In the last topic we said "just add more servers." But that immediately raises a question: when a user's request arrives, which of your many servers should handle it? The user only typed one address — they don't know or care that ten machines exist behind it. Something has to stand in front and decide. That something is a load balancer.

What a load balancer is

A load balancer (LB) is a component that sits in front of your pool of servers and distributes incoming requests across them. To the outside world, the load balancer is the website — users connect to it, and it quietly forwards each request to one of the servers behind it (often called the backend servers, or the server pool).

Why horizontal scaling needs a load balancer

Without a load balancer, having many servers is almost useless — you'd have no automatic way to route users across them. The load balancer is the piece that makes the pool act like one big, reliable service. It plays three high-level roles:

How does the LB know which servers are alive?

Servers can crash at any time, and the LB must never forward a request to a dead one. There are two complementary mechanisms, and the difference is a classic interview point — who initiates the ping:

But how does the LB even know which servers exist in the first place? Two common approaches:

• Self-registration: when a server first comes online in the network, it pings the LB to register itself.
• Configured IP range: you configure a range of IPs in the LB, e.g. 10.11.0.1 … 10.11.2.20, and it watches all of them.

Isn't the load balancer itself a bottleneck?

To answer this, compare what an app server does versus what a load balancer does. An app server (e.g. a Django server handling bookmarking requests) does a lot of work for every request — the request travels up through the OSI layers (the web server lives at the Application Layer):

1. Deserialize the request to get the payload
2. Authorization — check the user's permissions
3. Fetch the resource from the DB
4. Process the data
5. Generate a response
6. Serialize the response
7. Send the response

All that work means a typical app server handles only 100 – 1,000 requests/sec. The load balancer, by contrast, does not handle the request — it just (1) looks at the IP address in the request, (2) decides which server to forward to, and (3) re-routes it. Because it does so little, a typical LB easily handles 100,000+ requests/sec.

What if the load balancer crashes? (The SPOF question)

If everything funnels through one LB, then yes — absolutely — the LB is a single point of failure. If it goes down, the entire website goes down. How do we fix it?

• Tempting but wrong: "put another LB in front of the LBs." That doesn't help — you've just added a layer, and the SPOF problem now lives at the first LB instead.
• The real solution: run multiple LBs, and let the DNS act as the load balancer in front of the load balancers (DNS can hand out different LB IPs to different users).

Which server does the LB pick? Routing algorithms

When a request arrives, a routing algorithm decides which backend server gets it. Two you'll meet right away:

• Round Robin — hand requests to servers in turn: 1, 2, 3, 1, 2, 3, …
• Consistent Hashing — hash a key (e.g. the user) to consistently pick a server, which keeps things stable as servers are added or removed.

The deep dive on these is exactly the topic of Session 2.

7 How to succeed in HLD

HLD isn't memorised, it's developed as a way of thinking. Here's how to get the most out of this module:

• Be curious. Ask questions during the doubt session, devour all the reference material we provide, and explore interesting topics on your own.
• Think about the internals. For any app or software you use, think about its internal design: how might it handle scale, what unique challenges does it face, and how does that affect the features it supports (and doesn't support)?
• Think of the business use-case of every feature you consider. For example, how does showing a "blue checkmark for read" actually help WhatsApp?
• Interact with your peers. HLD is best learned through discussion — bouncing ideas in a group, talking about what you've seen at work and why, and sharing the tech challenges you and your company face.
• Solve the assignments before the next class. They're MCQs, so they're quick, but they require critical thinking — which is exactly what this module is built to develop.

★ Putting it all together

You just built the entire mental foundation of system design. Here's the one-paragraph story that connects all five topics:

Quick self-check

📚 References & Further Reading

Class material

• 📄 Original class notes / handout (Google Doc) — open the shared class material for this session.
• Class handout: "[SST-2028] HLD 1 - System Design 101".

Papers, docs & deep dives

• How DNS Works (illustrated) — a friendly comic walkthrough of DNS resolution.
• MDN: An overview of HTTP — the client/server request-response model and HTTP basics.
• The System Design Primer — a broad, beginner-friendly catalogue of scaling concepts.
• Cassandra: A Decentralized Structured Storage System — a classic look at horizontally scaling data across many machines.
• del.icio.us (Wikipedia) — background on the bookmarking service used as this session's case study.

DNS deep dives (from class)

• Cloudflare: What is DNS? — friendly explanation.
• Google Public DNS — a DNS server anyone can use.
• DNS Structure (network-insight) and the IETF 108 DNS deep-dive slides.
• Namecheap: setting up an A record and CNAME vs ALIAS vs URL.
• Domain extensions (.com .us .in .org): list of top-level domains and most common extensions.

Recommended by the instructor

• 📘 Reference book: Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann.
• YouTube channels: Alex Xu / ByteByteGo, Arpit Bhayani, Hussein Nasser, Martin Kleppmann.
• microservices.io — microservices reference.
• Engineering blogs by companies like Amazon, Meta, and Netflix.