Vertical vs Horizontal Scaling

TL;DR. Vertical scaling (bigger box) preserves single-writer simplicity, ACID guarantees, and operational sanity. Horizontal scaling (more boxes) removes the ceiling but introduces partitioning, replication lag, and months of engineering work. The production-canonical answer is “scale vertically first, then horizontally where you must.” A single AWS instance today offers up to 1,920 vCPUs and 32 TiB of RAM¹. Most systems never exhaust that. The dimension that decides is operational complexity per unit of throughput gained.

Learning Objectives

• Compare vertical and horizontal scaling across cost, complexity, availability, and ceiling dimensions.
• Identify the workload characteristics that justify staying on one machine versus sharding across many.
• Justify the hybrid path (vertical primary, read replicas, shard last) as the default progression.
• Evaluate real systems (Stack Overflow, Figma, Shopify) and explain why each chose its scaling strategy.

The Core Trade-off

The tension is not performance. Both approaches deliver throughput. The tension is coupling. Vertical scaling keeps one failure domain, one write-ahead log, one query plan cache, and one set of runbooks. Horizontal scaling introduces partitioning, replication, consensus, cross-partition queries, rebalancing, and schema coordination. Each of those is a multi-quarter engineering project².

The interview-canonical answer is “scale horizontally” because infinite boxes sound impressive. The production-canonical answer, for most systems, is the opposite. Figma’s databases team spent nine months to shard their first Postgres table and described horizontal sharding as “an order of magnitude more complex than our previous scaling efforts”². Shopify’s 2015 pivot to sharding was not a design choice but a forced move when they could no longer buy a larger database server³.

flowchart LR
    subgraph Vertical["Vertical (scale up)"]
        V1["4 vCPU, 16 GB"] -->|"replace"| V2["96 vCPU, 768 GB"]
    end
    subgraph Horizontal["Horizontal (scale out)"]
        LB["Shard router"] --> H1["Node 1"]
        LB --> H2["Node 2"]
        LB --> H3["Node 3"]
        LB --> H4["Node N"]
    end

Side-by-Side Comparison

The table misleads on availability. “Multiple failure domains” only holds if each shard’s dependencies are also isolated. Shopify identified this risk when all database shards shared a single Redis instance, leading them to restructure for per-pod isolation⁸. True blast-radius isolation requires per-pod duplication of caches, queues, and config³.

The cost row hides a discontinuity. Within the r5 family, pricing is perfectly linear: r5.12xlarge at $3.024/hr is exactly half of r5.24xlarge at $6.048/hr⁹⁴. Step to memory-optimized silicon and the curve breaks: u7i-12tb.224xlarge (896 vCPU, 12 TiB) runs $125.58/hr⁵, a 20.8x price jump for only 9.3x the vCPUs over r5.24xlarge.

When to Pick Vertical

The workload fits on one machine. Most do. Stack Overflow served 209 million HTTP requests per day, 505 million SQL queries per day, and 160 billion Redis ops per month on four SQL Servers (two clusters of primary plus replica) and two Redis nodes for 16 years¹⁰. They could run the entire Q&A network on a single web server during testing¹⁰.

Single-writer OLTP with cross-row transactions. If your queries join arbitrary rows or need atomic multi-table commits, a single primary gives you ACID for free. Sharding removes it².

Operational simplicity is the priority. WhatsApp served 465 million monthly users with only about 10 engineers running the Erlang platform end-to-end¹¹. Their philosophy: “Keep down management overhead by getting really big boxes and running efficiently on SMP machines”¹¹.

You have not exhausted cheaper levers. Query optimization, indexing, connection pooling, and caching routinely claw back 10x headroom. Stack Overflow dropped ASP.Net processing time from 757 hours/day to 447 hours/day through code optimization alone, despite adding 61 million requests/day¹⁰.

When to Pick Horizontal

You have genuinely exhausted vertical capacity. Measured, not assumed. The workload saturates the largest available instance after query optimization, indexing, and caching are applied. Shopify hit this wall in 2015: “it was no longer possible to continue buying a larger database server”³.

Availability requires independent failure domains. Even if one machine handles the load, you need replicas across AZs or regions for survival. Shopify’s pod architecture exists so that “a single pod’s failure wouldn’t spiral to a platform outage”³.

Write throughput exceeds single-node IO. Shopify’s 2024 Black Friday peaked at 7.6 million database writes per second across 100+ MySQL shards⁸. No single node handles that.

The data model has a natural partition key. Shopify shards by shop_id¹². Figma shards by UserID, FileID, or OrgID². Discord shards by (channel_id, time_bucket)¹³. If your data has no clean partition boundary, sharding will be painful.

The Hybrid Path

Most production systems follow a canonical progression: vertical primary, then read replicas, then caches, then vertical partitioning, then horizontal sharding. Each step is cheaper than the next. Most companies stop before the last step.

Figma’s progression from 2020 to 2024 is textbook¹⁴²: single r5.12xlarge primary, upgrade to r5.24xlarge (the largest RDS Postgres instance available at the time), add read replicas and PgBouncer, vertically partition tables into separate Postgres clusters (50 tables moved in one operation, 30 seconds of partial availability), then finally horizontally shard the largest remaining tables. Their database fleet grew 100x in four years², but horizontal sharding was the last lever pulled, not the first.

flowchart TD
    A["Single primary DB"] --> B{"Read-bound?"}
    B -->|yes| C["Add read replicas"]
    B -->|no| D{"Primary CPU saturated?"}
    C --> E["Add caching layer"]
    E --> D
    D -->|no| I["Stop here. Most companies do."]
    D -->|yes, multiple hot tables| F["Vertical partition: group tables into separate DBs"]
    D -->|yes, single hot table| G["Horizontal shard that table"]
    F --> H{"Still saturated?"}
    H -->|no| I
    H -->|yes| G
    G --> J["Pod architecture for blast-radius isolation"]

Real-World Examples

Stack Overflow: vertical for 16 years. Dell R720xd SQL Servers (24 cores, 384 GB RAM, 4 TB PCIe SSD each) in a two-cluster AlwaysOn setup served the entire Q&A network from 2010 to 2025¹⁰¹⁵. DB CPU utilization was deliberately kept very low as headroom policy¹⁰. The 2025 move to GCP was driven by datacenter vendor closure, not scaling limits¹⁵.

Figma: vertical first, horizontal last. Started on a single RDS Postgres at 65% CPU during peak traffic in 2020¹⁴. Exhausted every vertical lever over three years. First horizontally sharded table shipped in September 2023 after nine months of engineering, with 10 seconds of partial availability². They deliberately stayed on RDS Postgres rather than migrating to CockroachDB or Vitess because rebuilding operational expertise was higher risk².

Shopify: forced off vertical. Hit the MySQL single-node ceiling in 2015³. Built a pod architecture where each pod is a fully isolated slice: one MySQL shard, its own Redis, its own Memcached, with no cross-pod runtime communication³. By Black Friday 2024: 173 billion requests in 24 hours, 284 million RPM peak, 100+ active MySQL shards⁸.

Common Mistakes

Decision Checklist

• Have you measured the actual bottleneck (CPU, RAM, IO, lock contention), or are you assuming you need to scale?
• Have you applied query optimization, indexing, and caching before concluding the machine is too small?
• What is the largest instance your cloud provider offers, and have you benchmarked your workload on it?
• Does your workload have a natural partition key (tenant_id, user_id, shop_id)?
• Can you split reads (easy to scale out) from writes (hard to scale out)?
• What is the engineering lead time for sharding, and does the business timeline justify it?

flowchart TD
    Start["Need more capacity"]
    Q1{"Measured the<br/>bottleneck?"}
    Q2{"Biggest instance<br/>tried?"}
    Q3{"Read-heavy or<br/>write-heavy?"}
    Q4{"Natural<br/>partition key?"}
    Fix["Fix the query/index/lock first"]
    ScaleUp["Scale up: bigger instance"]
    Replicas["Add read replicas + cache"]
    Shard["Horizontal shard writes"]
    Start --> Q1
    Q1 -->|no| Fix
    Q1 -->|yes| Q2
    Q2 -->|no| ScaleUp
    Q2 -->|yes| Q3
    Q3 -->|reads| Replicas
    Q3 -->|writes| Q4
    Q4 -->|yes| Shard
    Q4 -->|no| ScaleUp

Key Takeaways

• Scale vertically first. A single machine in 2026 offers 1,920 vCPUs and 32 TiB of RAM¹. Most workloads never exhaust it.
• Horizontal scaling is not a design choice; it is a forced move when vertical is exhausted. Treat it as such.
• The hybrid path (vertical primary, read replicas, cache, vertical partition, then shard) is the canonical progression. Each step buys years of runway.
• Sharding costs months of engineering and permanently removes cross-shard ACID. Do the math before committing.
• Horizontal scaling without per-shard dependency isolation is not high availability; it is correlated failure with extra steps.

Further Reading

• Stack Overflow: The Architecture 2016 Edition: the canonical “vertical scaling done right” post; read before arguing for horizontal.
• How Figma’s databases team lived to tell the scale: nine-month horizontal sharding project; the best public account of the complexity cost.
• A Pods Architecture To Allow Shopify To Scale: why Shopify was forced off vertical in 2015 and how pods give failure-domain isolation.
• How Discord Stores Trillions of Messages: pure horizontal at chat scale; Cassandra to ScyllaDB with per-node and latency numbers.
• The growing pains of database architecture (Figma): textbook vertical partitioning story, 2020 to 2023.
• Amazon EC2 High Memory U7i Instances: current single-machine ceiling reference.

Flashcards

References