Networking Fundamentals for System Design

TL;DR: A first HTTPS connection pays for DNS, TCP and TLS 1.3 before any application data moves. That is 3 round trips, or roughly 300 ms across the Atlantic ¹. Use TCP unless you can tolerate loss, serve HTTP/2 by default and HTTP/3 at the edge for mobile users, keep DNS TTLs short on anything that can fail over, never allow 0-RTT on writes, and default to SSE for server push. WebSockets are only worth their cost when the client pushes too.

Most system design problems look like questions about databases and caches until a user in another country opens your app. Then you discover that your 20 ms API takes half a second to feel responsive, and none of that time shows up in your server logs.

The delay lives in the network, and it is predictable. Once you can count the round trips in a connection, you can explain why a feature feels slow, why a failover took five minutes, and why “just use WebSockets” is often the wrong answer.

By the end of this lesson, you will be able to:

• Explain what each OSI layer does and where it shows up in real infrastructure
• Choose between TCP and UDP for a given workload
• Describe head-of-line blocking and how HTTP/2 and HTTP/3 address it
• Walk through a DNS lookup from your laptop to the authoritative server
• Sketch a TLS 1.3 handshake and explain when 0-RTT is safe
• Pick the right real-time transport: WebSocket, SSE, or long polling

Where the First 300 ms Go

Think of mailing a package overseas. You hand it to the post office and it arrives, without you thinking about the cargo ship or customs. The internet hides its layers the same way, but every hidden step still takes time.

Here is what a browser pays before it can send its first request to a server it has never contacted:

At a 100 ms round trip, that is 300 ms of silence. The request and its response then take at least one more round trip. HTTP/3 removes one of these steps by combining the transport and encryption handshakes, which is most of its appeal on a first connection.

Physics sets the floor. Light in fiber travels at roughly 204,000 km/s, two-thirds of its speed in a vacuum, so a New York to London round trip can’t go below about 56 to 62 ms ². Real transatlantic round trips sit around 70 to 90 ms ³. You can’t optimize that away, which is why the fix is usually fewer round trips or a server closer to the user.

You can measure all of this yourself with curl:

curl -so /dev/null https://example.com -w '
dns:        %{time_namelookup}s
tcp done:   %{time_connect}s
tls done:   %{time_appconnect}s
first byte: %{time_starttransfer}s
'

Each value is cumulative from the start of the request, so subtract neighbouring lines to get the cost of one step. Run it twice against a distant server, and the DNS line usually drops close to zero the second time because the answer is now cached.

The Four Network Layers You Actually Touch

The OSI model has 7 layers, but system designers work with four. The TCP/IP model collapses them into Application, Transport, Internet and Link, and most engineers use the two models interchangeably.

These numbers matter most when you choose a load balancer:

• A layer 7 load balancer parses HTTP, so it can route on path, headers or cookies. AWS ALB works this way, with a default idle timeout of 60 seconds ⁴.
• A layer 4 load balancer only sees the TCP 5-tuple (source IP and port, destination IP and port, protocol) and forwards raw bytes. AWS NLB works this way, with a default TCP idle timeout of 350 seconds ⁵.

TCP vs UDP: Pay for Reliability or Build Your Own

TCP gives you a reliable, ordered byte stream. It handshakes, retransmits lost packets, and paces itself to avoid congesting the network. UDP gives you a datagram: fire it and forget.

sequenceDiagram
    participant C as Client
    participant S as Server
    Note over C,S: TCP 3-way handshake (1 RTT)
    C->>S: SYN (seq=x)
    S-->>C: SYN-ACK (seq=y, ack=x+1)
    C->>S: ACK (ack=y+1)
    Note over C,S: Connection established, data can flow

Use TCP when correctness matters more than speed: HTTP, database connections, file transfer, SSH.

Use UDP when loss is tolerable or you want to build your own reliability: DNS queries (one packet, retry if it drops), video calls (a dropped frame beats a stalled stream), game networking, and QUIC.

For most services this is not really a decision. TCP is the default, and you reach for UDP when a stalled stream would be worse than a missing packet.

HTTP/1.1, HTTP/2, HTTP/3: Each Fixed the Last One’s Bottleneck

Each generation of HTTP exists because of a specific bottleneck in the one before it.

HTTP/1.1 (RFC 9112) is text-based and serial: one request at a time per TCP connection. Browsers work around this by opening up to 6 parallel connections per origin ⁶. You still see it everywhere in internal services, scripts and health checks.

HTTP/2 (RFC 9113) multiplexes many requests over one TCP connection, using binary framing and HPACK header compression ⁷. The catch is TCP head-of-line (HOL) blocking. TCP delivers bytes in order, so one lost segment stalls every HTTP/2 stream until the retransmit arrives, including streams that had nothing to do with it ¹.

HTTP/3 (RFC 9114) runs on QUIC (RFC 9000), which runs on UDP. QUIC gives each stream independent loss recovery, so a lost packet only stalls the stream it belonged to ⁸. It also combines the transport and TLS handshakes into a single round trip, and supports 0-RTT resumption for returning clients.

flowchart LR
    subgraph HTTP11["HTTP/1.1"]
        A1[1 request per connection]
        A2[Up to 6 parallel TCP conns]
    end
    subgraph HTTP2["HTTP/2"]
        B1[Binary framing, multiplexed streams]
        B2[Single TCP conn: TCP HOL blocking]
    end
    subgraph HTTP3["HTTP/3"]
        C1[QUIC over UDP]
        C2[Per-stream loss recovery, 0-RTT]
    end
    HTTP11 --> HTTP2 --> HTTP3

All three are in real use. In May 2023, Cloudflare reported HTTP/3 at about 28% of its global traffic, HTTP/2 at about 64% and HTTP/1.1 at about 9% ⁹. Its 2025 Radar data shows HTTP/3 at 21% of requests, HTTP/2 at 50% and HTTP/1.x at 29% ¹⁰. The two reports measure different things (traffic served versus requests received), so don’t read the difference as HTTP/3 shrinking.

What to pick: serve HTTP/2 by default. Add HTTP/3 at the edge when you have mobile or global users, and keep HTTP/1.1 where simple tooling matters more than throughput.

DNS: A Chain of Caches That Decides Your Failover Time

DNS translates names like api.stripe.com into IP addresses. The lookup walks a hierarchy, with caches along the way.

sequenceDiagram
    participant App as Your App
    participant Stub as Stub Resolver
    participant Rec as Recursive Resolver (1.1.1.1)
    participant Root as Root (.)
    participant TLD as TLD (.com)
    participant Auth as Authoritative (stripe.com)
    App->>Stub: gethostbyname("api.stripe.com")
    Stub->>Rec: Query api.stripe.com?
    Rec->>Root: Who serves .com?
    Root-->>Rec: Ask the .com TLD servers
    Rec->>TLD: Who serves stripe.com?
    TLD-->>Rec: Ask ns1.stripe.com
    Rec->>Auth: api.stripe.com?
    Auth-->>Rec: 34.98.120.1
    Rec-->>Stub: 34.98.120.1 (TTL 300)
    Stub-->>App: 34.98.120.1

Your OS ships a small stub resolver, which forwards the query to a recursive resolver such as Cloudflare’s 1.1.1.1 or your ISP’s server. The recursive resolver walks from the root to the TLD to the authoritative nameserver, then caches the answer for as long as its TTL allows. Cloudflare handles approximately 67 million DNS queries per second across its authoritative and resolver services ¹⁰.

DNS normally runs over UDP on port 53 and falls back to TCP when a response is too large. DNS-over-HTTPS (DoH) and DNS-over-TLS (DoT) add privacy by encrypting queries.

You can watch both the walk and the TTL:

dig +trace api.stripe.com   # follow the walk from the root down
dig api.stripe.com          # the second column of the answer is the remaining TTL

The TTL is the setting that bites in production, because it is the longest a cached client may keep using an old address:

TLS 1.3 and the 0-RTT Trap

TLS 1.3 (RFC 8446) cut the handshake from 2 round trips to 1 ¹¹. On a mobile network where each round trip costs 50 to 200 ms, you feel that.

sequenceDiagram
    participant C as Client
    participant S as Server
    Note over C,S: TLS 1.3 full handshake (1 RTT)
    C->>S: ClientHello + key_share + supported ciphers
    S-->>C: ServerHello + key_share + {Certificate} + {Finished}
    Note over C,S: Keys derived, application data flows
    C->>S: {Finished} + [Application Data]

The saving comes from a good guess. The client assumes which key exchange group the server prefers and sends its public key in the first message, so the server can answer with everything else in one reply.

For repeat visits, 0-RTT resumption goes further: the client sends application data in its very first packet, using a key from the previous session. That speed has a price. 0-RTT data is not forward secret, and an attacker who captures it can replay it to your servers ¹¹.

A replayed page view is harmless. A replayed payment is not:

POST /transfer-money HTTP/1.1
Host: bank.example.com
Content-Type: application/json

{"to": "alice", "amount": 500}

If that request arrives over 0-RTT, a captured copy can be sent again and the transfer may run twice. So the rule is to accept early data only for safe, idempotent requests, and answer anything else with HTTP 425 Too Early, which makes the client retry after a full handshake. Cloudflare leaves 0-RTT off by default and rejects non-idempotent methods over it ¹².

If you terminate TLS yourself, the proxy has to tell your app which requests arrived early. In nginx:

ssl_early_data on;
proxy_set_header Early-Data $ssl_early_data;

Your app then returns 425 for any non-GET request that carries Early-Data: 1.

Real-Time Updates: Default to SSE, Not WebSockets

HTTP is request-response. For server-pushed data like chat messages, stock ticks and notifications, you need a long-lived channel, and there are three options.

Long polling opens an HTTP request that the server holds until data is ready or a timeout fires. It works through any proxy, but every message is a fresh HTTP request with full headers.

Server-Sent Events (SSE) is a long-lived HTTP response with Content-Type: text/event-stream. It is one-way, from server to client, and it reconnects on its own. The format is simple enough to read on the wire:

id: 42
event: score
data: {"home": 2, "away": 1}

The browser side is just as small. If the connection drops, EventSource reconnects and sends Last-Event-ID: 42, so the server knows where to resume:

const stream = new EventSource('/scores');
stream.addEventListener('score', (e) => render(JSON.parse(e.data)));

WebSockets (RFC 6455) upgrade an HTTP connection into a full-duplex binary channel with only 2 to 14 bytes of framing per message ¹³. You get true two-way traffic, and in exchange you own sticky sessions, reconnect logic and proxy quirks.

graph LR
    Start{Does the client push data?} -->|No| OneWay{Latency tolerance?}
    Start -->|Yes| WS[WebSocket: full-duplex, binary frames]
    OneWay -->|Sub-second, modern infra| SSE[SSE: auto-reconnect, Last-Event-ID]
    OneWay -->|Must work through ancient proxies| LP[Long polling: HTTP held open]

What to pick: start with SSE. It covers dashboards, notifications and LLM token streams without sticky sessions. Move to WebSockets when the client sends frequent data of its own, like chat input, game state or cursor positions. Keep long polling for environments that block everything else.

SSE has one limit to plan for. Over HTTP/1.1, each stream uses one of the browser’s 6 connections per domain ⁶. Over HTTP/2 the limit becomes the negotiated SETTINGS_MAX_CONCURRENT_STREAMS. RFC 9113 sets no initial limit but recommends a value of at least 100, and 100 is a common server setting.

What Cloudflare’s HTTP/3 Rollout Showed

Cloudflare rolled out HTTP/3 across its network starting in September 2019, while the QUIC spec was still a draft ¹. Its edge runs on its own open-source QUIC library, quiche, behind Anycast BGP.

Its April 2020 production benchmark is a useful reality check ¹:

That is a real improvement but not a dramatic one. The gains come mainly from removing the extra TLS round trip and from per-stream loss recovery on lossy mobile links, while on fast wired links the difference is marginal. By May 2023, HTTP/3 carried about 28% of Cloudflare’s traffic, and about 40% of requests from Chrome Mobile ⁹.

The takeaway: treat HTTP/3 as a win for mobile and global users, not a blanket speed-up, and keep a fallback. Advertise it with Alt-Svc alongside HTTP/2, because some enterprise firewalls still block outbound UDP on port 443.

Five Mistakes That Cause Real Incidents

A long DNS TTL on a failover record

After an RDS CNAME failover, clients keep connecting to the dead IP until their cached answer expires ¹⁴. With a 300-second TTL, that can mean 5 minutes of errors after the database has already recovered.

Fix: use 30 to 60 second TTLs on failover-targeted records.

A proxy that quietly undoes connection reuse

You deploy HTTP/2 end to end on paper, but your proxy terminates it and opens a fresh upstream connection for every request, so each one pays a new handshake.

# Bad: no connection reuse to the upstream
location / {
    proxy_pass http://10.0.0.1:8080;
}

# Better: reuse a pool of upstream connections
upstream api {
    server 10.0.0.1:8080;
    keepalive 32;
}
location / {
    proxy_pass http://api;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

This restores connection reuse. It doesn’t give you multiplexing to the upstream, so use a proxy like Envoy if you need HTTP/2 on that hop.

WebSockets for one-way data

If data only flows from server to client, WebSockets add sticky sessions, custom reconnect logic and proxy problems, and you get nothing back for it.

Fix: use SSE with EventSource, which reconnects automatically ⁶.

0-RTT on non-idempotent requests

An attacker captures a POST sent over 0-RTT and replays it, and the server processes it more than once.

Fix: reject 0-RTT on POST, PUT and DELETE at your TLS terminator, and answer with HTTP 425 Too Early ¹².

SSE stalling at the 6-connection limit

An SSE-heavy app works fine until a user opens a seventh tab, which can’t open a stream because the browser has hit its per-domain limit over HTTP/1.1. Chrome and Firefox have marked this “Won’t fix” ⁶.

Fix: serve SSE over HTTP/2, or shard across subdomains.

Which Transport to Pick

Exercise

Design Challenge: You are building a live sports score app for 10 million concurrent users. Scores update every few seconds during a match. Latency should be under 1 second. Design the client-to-server transport. Which protocol do you pick, and why? How do you handle reconnects, scaling, and cold starts?

Quick Reference

Further Reading

• RFC 9114: HTTP/3 - The definitive HTTP-over-QUIC spec; read sections 1 to 3 for the motivation and stream mapping.
• RFC 8446: TLS 1.3 - Sections 2 and 4 for the handshake; Appendix E for the security analysis of 0-RTT.
• Comparing HTTP/3 vs HTTP/2 Performance - Cloudflare’s 2020 production benchmark with TTFB numbers, page load sweeps, and CUBIC vs BBR analysis.
• Even faster connection establishment with QUIC 0-RTT - Why Cloudflare disables 0-RTT by default and how they handle replay protection.
• How Discord Scaled Elixir to 5,000,000 Concurrent Users - Real WebSocket fan-out at scale: 500K sessions per Elixir VM, custom batching, and cascading failure patterns.
• MDN: EventSource - Authoritative reference on SSE browser behavior, the 6-connection limit, and Last-Event-ID semantics.
• The QUIC Transport Protocol: Design and Internet-Scale Deployment - Google’s SIGCOMM 2017 paper on deploying QUIC at 7% of internet traffic.
• DDIA Chapter 8 (Designing Data-Intensive Applications, Martin Kleppmann), pp. 274 to 287 - “The Trouble with Distributed Systems” on unreliable networks and timing assumptions.

Flashcards

References