Trade-off Thinking

TL;DR: “It depends” is technically correct and completely useless unless you finish the sentence. Every design choice buys one property and pays with another. Senior engineers name both sides, commit with a reason tied to a specific constraint, and record what they gave up so their future selves can revisit. Jeff Bezos’s 70% rule says most decisions should be made with about 70% of the information you wish you had¹. Waiting for 90% is almost always too slow.

Learning Objectives

After this module, you will be able to:

• Finish every “it depends” with a named dependency, a threshold, and a recommendation
• Use the six-part articulation template (Context, Options, Chosen, Why, Given up, Reversible?) to structure any design decision
• Classify decisions as one-way or two-way doors and match process weight accordingly
• Identify cognitive biases (sunk cost, resume-driven, survivorship, NIH) that corrupt architecture decisions
• Apply Kent Beck’s 3X model to recognize when a correct past decision becomes wrong today
• Write a lightweight Architecture Decision Record that survives team turnover

Intuition

You are buying a car. You want safety, fuel economy, cargo space, acceleration, and a low monthly payment. You cannot have all five at their maximum. A minivan maximizes cargo and safety but sacrifices acceleration. A sports car maximizes acceleration but sacrifices cargo and cost. A compact sedan optimizes fuel economy and cost but gives up cargo and acceleration.

Nobody calls this “hard.” You name what matters most for your life right now, you pick the car that fits, and you accept what you gave up. You do not pretend the minivan is also fast.

System design works the same way. You have consistency, availability, latency, throughput, cost, simplicity, and developer velocity. You cannot max all of them. The engineer’s job is to name which ones matter most for this system at this scale, pick the architecture that fits, and say out loud what was sacrificed.

The rest of this chapter teaches you how to do that naming, picking, and saying out loud in a way that builds trust, survives team turnover, and sounds senior in an interview.

Theory

What a trade-off actually is

A trade-off is a design decision where improving one measurable property of the system necessarily degrades another. The word “necessarily” matters. If a choice improves property X at no cost to anything else, it is not a trade-off but a free lunch, and in a mature design review the first response to a free-lunch claim is to look for the hidden cost.

The mechanism is physical, economic, or algorithmic:

• Physical: The speed of light caps intercontinental round-trip time to roughly 80 to 150 ms (e.g., US-to-Europe ~80 ms, US-to-Asia ~150 ms), so strong consistency across regions buys latency.
• Algorithmic: Every index that speeds reads slows writes, because each write updates both the row and the index.
• Economic: Hiring PostgreSQL operators is cheaper than hiring Spanner-equivalent operators, so self-hosted strong consistency buys operational cost.

Fred Brooks’s “No Silver Bullet” (1986) gives the meta-category: essential complexity is what the problem demands; accidental complexity is what the tools force on you². You can trade accidental complexity for accidental complexity all day (pick Kafka or RabbitMQ; pick Go or Rust) and the system gets no better. You only move forward by trading accidental cost for essential progress.

The primary axes

Most system design decisions move along a small set of recurring dimensions:

The right way to say “it depends”

“It depends” is correct. It is also the verbal equivalent of a blank stare. The fix is a three-part sentence:

1. Name the dependency. “It depends on whether we need cross-region consistency.”
2. State the branches. “If yes, use synchronous replication and accept 150 ms write latency. If no, use async replication and accept stale reads up to 2 seconds.”
3. Recommend for this context. “Given our SLO of 200 ms writes and single-region deployment, I would start with synchronous replication.”

This is the six-part articulation template compressed into speech. The full written version, distilled from Michael Nygard’s ADR format³ and Bezos’s reversibility doctrine¹:

flowchart LR
    C[Context:<br/>the constraint] --> O[Options:<br/>2-4 candidates]
    O --> CH[Chosen:<br/>active voice]
    CH --> W[Why:<br/>link to context]
    W --> G[Given up:<br/>name the loss]
    G --> R[Reversible?<br/>one-way or two-way door]

Use this template in ADRs, design docs, interview answers, and Slack threads. It is the single most senior-sounding verbal shape in engineering.

Cognitive biases that ruin decisions

Humans are bad at trade-off analysis under pressure. Kahneman’s “Thinking, Fast and Slow” catalogs the System 1 heuristics that hijack design reviews⁴:

• Sunk cost. “We have already spent 6 months on this migration, we cannot stop now.” Yes you can. The 6 months are gone regardless.
• Availability bias. The most recent incident dominates risk estimates. A team that just had a Kafka outage overweights “Kafka is unreliable” even if the root cause was misconfiguration.
• Confirmation bias. You seek evidence for the option you already like. The engineer who wants to use Rust finds benchmarks; the one who wants Go finds hiring data.
• Resume-driven development. Engineers pick technology that advances their careers, not their systems. Kelsey Hightower argued publicly that Kubernetes is “a platform for building platforms,” not an endgame, a point often cited in discussions of teams adopting it for resume value on workloads that do not need it⁵.

The antidote is institutional: Architecture Decision Records force written, delayed commitment³. Pre-mortems force you to imagine failure before you ship. Weighted decision matrices force you to make the argument about weights (a discussable topic) rather than about options (an opinion).

One-way vs two-way doors

Bezos introduced the one-way/two-way door taxonomy in his 2015 letter to shareholders, where he called them “Type 1” (consequential, irreversible) and “Type 2” (changeable, reversible) decisions⁶. In his 2016 letter he revisits the concept more memorably: “Many decisions are reversible, two-way doors. Those decisions can use a light-weight process. For those, so what if you’re wrong?”¹ He pairs this with the 70% rule and “disagree and commit” as a tool for unblocking two-way-door decisions under disagreement.

The failure mode he describes: large organizations defaulting to the heavyweight Type 1 process for decisions that are actually reversible Type 2 doors, resulting in “slowness, unthoughtful risk aversion, failure to experiment sufficiently, and consequently diminished invention.”⁶

flowchart TD
    D[Proposed decision] --> Q1{Reversible within<br/>a few weeks?}
    Q1 -->|Yes: two-way door| F[Fast path:<br/>decide in standup,<br/>document inline]
    Q1 -->|No: one-way door| Q2{High confidence<br/>about outcomes?}
    Q2 -->|Yes| S[Write ADR,<br/>1-day review,<br/>commit]
    Q2 -->|No| R[Write RFD,<br/>3-5 day discussion,<br/>prototype first]
    F --> V[Revisit at next<br/>architecture review]
    S --> V
    R --> V

For engineers: choosing a programming language for a new service is usually a one-way door (rewrite cost is huge). Choosing a cache eviction policy is usually a two-way door (change the config, restart). Choosing a database engine is somewhere in between, and the skill is recognizing which side it falls on for your specific system.

The honesty test

Dan McKinley’s “Choose Boring Technology” (2015) gives every team a budget of roughly three “innovation tokens”⁷. Each new piece of technology spends one. The benefit of a new tool is usually advertised loudly; the operational cost (maintenance, 3 AM on-call, unknown-unknowns) is invisible at adoption time and crushing in year two.

McKinley’s line: “The grim paradox of this law of software is that you should probably be using the tool that you hate the most. You hate it because you know the most about it.”⁷

The honesty test: for every pro you list, you must also state its corresponding con. If your design doc has 7 pros and 0 cons for your preferred option, you are writing a pitch deck, not an architecture decision.

The dimension-counting anti-pattern is what happens when teams decide by counting bullets rather than weighing them. Seven pros vs five cons does not mean the seven-pro option wins. One con (“we lose all data during a region failure”) can outweigh seven pros. Use a weighted decision matrix to expose the real disagreement: about weights, not about options⁸.

When trade-offs change over time

Kent Beck’s 3X model names three stages of product evolution, each with different optimal trade-offs⁹:

flowchart LR
    E1[Explore<br/>learn fast,<br/>accept messy code] --> E2[Expand<br/>remove scaling<br/>bottlenecks]
    E2 --> E3[Extract<br/>optimize for<br/>efficiency and ops]
    E1 -.-> T1[speed over quality]
    E2 -.-> T2[throughput over features]
    E3 -.-> T3[cost over novelty]

• Explore: Low scale, high uncertainty. Optimize for learning speed. Accept messy code, ship experiments, use the tool you know.
• Expand: The product works and growth exposes bottlenecks. Engineering effort shifts to removing rate-limiting resources.
• Extract: Mature phase. Optimize for efficiency, profitability, and operational stability.

A trade-off that maximized learning speed in Explore (hand-rolled Rails app, one database) becomes a liability in Expand when the growth curve is non-linear. The 3X model gives vocabulary for “the right call last year is the wrong call now” without anyone being wrong⁹.

Real-World Example

Discord: Cassandra to ScyllaDB (2017 to 2023)

Discord’s message storage evolved through all three 3X phases, and each transition was a trade-off revisited¹⁰.

Explore (2015 to 2017): MongoDB replica set. Fast to build, familiar to the team. Gave up: horizontal write scaling.

Expand (2017 to 2022): Migrated to Cassandra with replication factor 3. Grew from 12 nodes to 177 nodes storing trillions of messages. The trade-off bought: write throughput and horizontal scaling. The trade-off paid: JVM garbage collection pauses caused latency spikes (p99 of 40 to 125 ms for historical reads), required manual node reboots, and created “gossip dance” operational toil during compaction¹⁰.

Extract (2022 to 2023): Migrated to 72 ScyllaDB nodes, each with 9 TB disk. Results: p99 historical reads dropped to 15 ms; p99 inserts held steady at 5 ms; cluster size dropped from 177 to 72 nodes¹⁰.

The key engineering decisions illustrate trade-off thinking:

1. Kept the data model, changed the engine. Cassandra Query Language compatibility made this a two-way door at the schema layer (application code barely changed) even though it was a one-way door at the ops layer (new C++ database, new failure modes).
2. Rewrote the Spark migrator in Rust. The initial Spark-based migration estimated 3 months. The team spent engineering time building a Rust migrator that finished in 9 days at 3.2 million messages per second¹⁰. Classic trade-off: dev time vs wall-clock time-to-value.
3. Accepted the one-way door of abandoning a JVM-GC-prone database for a shard-per-core C++ one, motivated by years of on-call toil.

What Discord gave up: JVM ecosystem compatibility, Cassandra’s larger community, and the team’s existing operational muscle memory. What they gained: 3x latency improvement, 60% fewer nodes, and elimination of GC-induced incidents.

flowchart TD
    P[Proposed change:<br/>Cassandra to ScyllaDB] --> Q1{Is Cassandra<br/>the bottleneck?}
    Q1 -->|Yes: GC pauses,<br/>compaction cascades| Q2{Payback in<br/>under 12 months?}
    Q2 -->|Yes: 60% fewer nodes,<br/>3x latency| Q3{Team has ScyllaDB<br/>experience?}
    Q3 -->|No| Q4{Invest in training<br/>and gradual rollout?}
    Q4 -->|Yes: Rust data services<br/>as abstraction layer| GO[Proceed]

Trade-offs

These rows describe decision-making approaches, not substitutable alternatives. Readers pick by door type and context, the same axis the one-way/two-way flowchart earlier in this chapter encodes. Drop the “Our Pick” framing; each row is already a when-to-use recommendation.

The approach is a function of reversibility and stakes, not preference. Match process weight to the door type and to how much blast-radius a wrong answer carries.

Common Pitfalls

Exercise

Design Challenge: Your team operates a Python monolith on EC2. It serves 10k RPS at p99 of 600 ms (SLO: 300 ms). A junior engineer proposes rewriting it in Go microservices on Kubernetes. Senior leadership asks for your recommendation. Present the trade-offs using the six-part articulation template and give a defensible recommendation.

Key Takeaways

• Every “it depends” needs a named dependency, a threshold, and a recommendation. Finish the sentence.
• Use the six-part template (Context, Options, Chosen, Why, Given up, Reversible?) for every significant decision. It fits in an ADR, a Slack message, or a 30-second verbal answer.
• Classify decisions as one-way or two-way doors. Two-way doors get fast, lightweight process. One-way doors get deliberate, written process¹.
• Name what you gave up. If your design doc has only pros, you are writing marketing, not engineering.
• Cognitive biases (sunk cost, resume-driven, survivorship, confirmation) are organizational facts, not moral failings. Counter them with written artifacts and explicit weights.
• Trade-offs change over time. Beck’s 3X model (Explore, Expand, Extract) gives vocabulary for “the right call last year is the wrong call now” without blame⁹.
• Sometimes the best decision is the null decision: keep what you have. “Technology we operate at 3 AM” is usually worth more than “technology that is new to us.”

Further Reading

• Choose Boring Technology (Dan McKinley, 2015) - The essay that named innovation tokens; read it once a year as a calibration exercise against novelty bias.
• Documenting Architecture Decisions (Michael Nygard, 2011) - The original ADR post; 1,200 words that changed how teams record decisions.
• 2016 Letter to Shareholders (Jeff Bezos) - Source of the one-way/two-way door taxonomy and the 70% rule.
• How Discord Stores Trillions of Messages (Bo Ingram, 2023) - The clearest single post on a multi-year infrastructure trade-off revisited with specific numbers.
• RFD 1: Requests for Discussion (Oxide Computer, 2020) - A working RFD process embedded in a public company’s engineering culture; extends ADRs with a lifecycle.
• An Elegant Puzzle (Will Larson, Stripe Press, 2019) - Engineering-manager framing of trade-offs, weighted matrices, and when to formalize decisions.
• Under Deconstruction: Shopify’s Monolith (Philip Muller, 2020) - The modular monolith as a third option between “microservices” and “ball of mud”; 2.8M lines of Ruby, still one deploy¹¹.
• Spanner, TrueTime and the CAP Theorem (Eric Brewer, 2017) - The author of CAP revisiting his own theorem in light of a system that trades dollars (atomic clocks) for stronger consistency.
• Fast/Slow in 3X: Explore/Expand/Extract (Kent Beck, 2018) - The three-phase model for recognizing when a correct past decision becomes wrong today.

Flashcards

References