Graceful Degradation: When Partial Service Beats No Service

TL;DR: Production systems are never “fully up” or “fully down.” The real question is which features you keep and which you shed. Graceful degradation is the discipline of deciding, before an incident, that checkout stays up while recommendations disappear. The toolkit is small: load shedding (reject early, reject cheap), feature flags (kill switches on non-essential paths), cached fallbacks (stale data beats no data), and static fallbacks (pre-rendered HTML when the app tier is gone). AWS distinguishes “throughput” from “goodput,” where goodput is requests completed successfully and quickly enough to be useful¹. When a server passes its goodput knee, the engineering objective is to keep goodput high by rejecting or degrading the excess, not by accepting work you cannot finish.

Learning Objectives

After this module, you will be able to:

• Design a priority tiering of features for load shedding
• Implement load shedding at the load balancer, gateway, or app layer
• Use feature flags to turn off expensive features under load
• Serve stale cached content as a fallback
• Communicate degradation to users without making things worse

Intuition

You run a restaurant on a Friday night. The kitchen is at capacity. Orders are backing up. You have two choices: keep accepting every table’s full order and watch every dish come out cold and late, or trim the menu. You tell the host: “No specials tonight. Appetizers are limited to three options. Desserts are off.” The entrees, the thing people came for, still arrive hot and on time.

Nobody leaves a one-star review because the creme brulee was unavailable. They leave one-star reviews because their steak arrived 90 minutes late and cold. The restaurant that trims its menu under pressure delivers a better experience than the one that promises everything and delivers nothing well.

Software systems work the same way. When a dependency fails or traffic spikes beyond capacity, you can keep accepting every request and watch latency climb until clients time out, or you can shed the non-essential work and protect the critical path. Recommendations can disappear. Personalization can fall back to “popular items.” Analytics can stop collecting. But login, checkout, and core reads must keep working.

Resilience Patterns gave you the defensive mechanisms: timeouts, circuit breakers, bulkheads. This chapter teaches you what to do with those mechanisms. Which features get the circuit breaker’s fallback? Which requests does the load shedder reject first? Who decides, and when?

Theory

Critical-path tiering

The first step in graceful degradation is not technical. It is a product conversation: which features matter most?

Google’s internal RPC framework uses four fixed criticality levels: CRITICAL_PLUS, CRITICAL, SHEDDABLE_PLUS, and SHEDDABLE². A server rejects requests of lower criticalities first when overloaded. Criticality propagates across RPC hops so downstream services apply the same priority. AWS pushes tier decisions as close to the client edge as possible rather than making global prioritization decisions inside every service¹.

A practical three-tier model for most systems:

flowchart TD
    subgraph T1["Tier 1: Always on"]
        Login["Login / session"]
        Checkout["Checkout / payment"]
        CoreRead["Core reads: product page, repo, feed"]
    end
    subgraph T2["Tier 2: Shed under moderate stress"]
        Recs["Recommendations"]
        Search["Advanced search filters"]
        Analytics["Real-time analytics"]
    end
    subgraph T3["Tier 3: Shed first"]
        Personal["Per-user personalization"]
        Experiments["A/B experiments"]
        SocialProof["Live widgets, review counts"]
    end
    T3 --> T2 --> T1

The organizational challenge is tier inflation. If every team labels their feature CRITICAL_PLUS, the signal is destroyed². Tie tiers to SLOs: Tier 1 features get a three-nines target, Tier 2 gets two-nines, Tier 3 gets best-effort. SLOs, SLIs, and Error Budgets provides the framework for setting these targets.

Load shedding

Load shedding is the act of rejecting requests at the earliest layer that has enough information, with a fast, cheap response, instead of accepting work the server cannot complete.

When a server passes its goodput knee, latency rises, clients time out, and retries amplify the load. The fix is to keep accepted work bounded and reject the rest with a cheap 503 + Retry-After. AWS recommends shedding at the earliest point where you have the information to decide: edge proxy, API gateway, or app middleware¹.

Where to shed:

• L4 load balancer: connection limits, SYN flood protection. Blunt but fast.
• API gateway: per-client rate limits, per-endpoint quotas. Knows the route but not the request body.
• App middleware: priority-based shedding using request metadata (user tier, endpoint criticality). Most precise but most expensive to reject.

Priority-based shedding combines tiering with load shedding. Tag each request with its criticality. Under stress, reject Tier 3 first, then Tier 2. Tier 1 is the last to shed. Netflix’s concurrency-limits library partitions capacity: 90% reserved for live traffic, 10% for batch. Under stress, batch traffic is shed while live traffic continues³.

sequenceDiagram
    participant Client
    participant Edge as Edge proxy
    participant Svc as Service (at capacity)
    participant DB
    Client->>Edge: POST /checkout (Tier 1)
    Edge->>Svc: forward
    Svc->>DB: write
    DB-->>Svc: ok
    Svc-->>Edge: 200
    Edge-->>Client: 200 OK

    Client->>Edge: GET /recommendations (Tier 3)
    Edge->>Svc: forward
    Svc-->>Edge: 503 + Retry-After
    Edge-->>Client: 503
    Note over Svc: Concurrency limit hit, Tier 3 shed first

Adaptive concurrency limits solve the tuning problem. Netflix’s library applies TCP congestion-control algorithms (Vegas, Gradient2) to HTTP/gRPC. It samples latencies, detects queueing via Little’s Law (Limit = Average RPS * Average Latency), and shrinks the allowed in-flight count dynamically³. No static RPS ceiling to go stale as services autoscale.

Feature flags as kill switches

A feature flag used for degradation is an ops toggle: a manually-managed circuit breaker that turns off a specific feature without a redeploy⁴. Pete Hodgson’s taxonomy names four categories (Release, Experiment, Ops, Permissioning). Ops toggles are the kill-switch category.

The lifecycle is simple: on when healthy, flip off during incident, flip back after recovery. Platforms like LaunchDarkly, Unleash, and OpenFeature provide the distributed config store, targeting rules, and audit log. Many can auto-flip a flag when an observability signal (error rate, latency) crosses a threshold, connecting directly to Observability pipelines.

Tier-based auto-disable extends this: when p99 latency exceeds the SLO, automatically disable Tier 3 features. If error rate exceeds 5%, disable Tier 2. Operators retain manual override for Tier 1.

Flag debt is the carrying cost. Flags are cheap to add, expensive to remove. Every deployment has exponential toggle-state combinations to test. Hodgson: “Savvy teams view their Feature Toggles as inventory which comes with a carrying cost”⁴. Add a removal task when the flag is created. Put expiration dates in the config. Knight Capital lost approximately 440 million USD in 45 minutes when a code deployment left deprecated logic active on one of eight servers, a cautionary tale about stale code paths that resembles the risks of unmanaged flag debt⁴.

Cached and static fallbacks

When a dependency is unreachable, you have two options: error or stale data. For read-heavy paths, stale data almost always wins.

RFC 5861 defines two HTTP directives for this⁵:

• stale-while-revalidate=N: serve stale for N seconds while revalidating in the background.
• stale-if-error=M: serve stale for M seconds when the origin returns 500, 502, 503, or 504.

Cache-Control: max-age=600, stale-if-error=1200

This tells CDNs: content is fresh for 10 minutes; if revalidation fails, keep serving the cached copy for up to 20 more minutes. Cloudflare’s Always Online feature takes this further: when the origin is completely unreachable, Cloudflare serves static snapshots of popular pages from the Internet Archive. It cannot serve dynamic content, logins, or POST requests, but for marketing pages and documentation, it turns a full outage into a stale-but-available experience⁶.

In-app last-known-good is the application-level equivalent. When a recommendations service times out, return the last successful response from a local cache. The user sees slightly stale recommendations rather than an error or an empty widget.

Static fallbacks are the last resort before a maintenance page:

• Pre-rendered HTML for the homepage served from S3/CDN
• Skeleton UIs that show structure without data
• Read-only mode (writes disabled, reads served from replicas)

The Amazon counter-argument. AWS’s explicit position is that fallback “almost never” helps in distributed systems because the fallback path is rarely exercised, hard to test, and shares fate with the primary in non-obvious ways⁷. Their preferred alternative: make the primary path reliable, push data proactively (e.g., IAM credentials pushed hours in advance), or convert fallback into failover by exercising both paths continuously. This is not a contradiction. It is a distinction: a “degraded response” (intentional, tested, continuously exercised) is safe. A “fallback” (rarely-run code path full of latent bugs) is dangerous.

Communicating degradation to users

Silent degradation is an anti-pattern. If a feature is off, tell the user.

Status pages (Statuspage, Instatus) communicate system-wide state. But the AWS S3 outage of February 2017 showed the failure mode: the status page itself depended on S3, so it could not report S3 being down⁸. Host your status page on independent infrastructure.

In-app banners keyed to the same flag as the backend kill switch. When recommendations are disabled, show “Recommendations temporarily unavailable” in the widget space. Do not show an empty div that looks like a bug.

Structured error responses using RFC 9457 application/problem+json (which obsoletes RFC 7807) let clients and SDKs reason about the degradation mode programmatically:

{
  "type": "https://example.com/probs/degraded",
  "title": "Recommendations temporarily unavailable",
  "detail": "Upstream recommender is disabled; showing popular items.",
  "instance": "/products/SKU-123"
}

The principle: be honest, not silent. Users tolerate “we are experiencing issues, some features are limited” far better than mysterious empty pages or unexplained slowness.

Real-World Example

GitHub’s read-only mode (October 2018)

On October 21, 2018, a 43-second network connectivity loss between GitHub’s US East Coast hub and its primary data center triggered a cascade. Orchestrator (GitHub’s MySQL topology manager) promoted West Coast databases to primary. When the network healed, the application tier wrote to West Coast primaries across a cross-country link, introducing latency the app could not absorb. One busy cluster had 954 writes during the brief partition that were not replicated to the West Coast, while the West Coast ingested new writes from the application tier for nearly 40 minutes before engineers could intervene⁹.

GitHub’s response was deliberate degradation: read-only mode. They paused webhook delivery and GitHub Pages builds rather than risk overwriting user data. For 24 hours and 11 minutes, users could browse repositories and issues but could not push, open pull requests, or receive webhooks⁹.

The numbers tell the story of the backlog: over 5 million webhook events queued, 80,000 Pages builds queued, and roughly 200,000 webhook payloads exceeded their internal TTL and were dropped⁹.

stateDiagram-v2
    [*] --> Healthy
    Healthy --> PartialShed: error rate > 1% OR p99 > SLO
    PartialShed --> ReadOnly: write-path errors > 5%
    ReadOnly --> StaticFallback: app tier unhealthy
    StaticFallback --> Maintenance: full outage declared
    Maintenance --> StaticFallback: partial restore
    StaticFallback --> ReadOnly: app tier back
    ReadOnly --> PartialShed: writes < 1% errors
    PartialShed --> Healthy: error rate < 0.1% for 5 min

The key engineering decision: prioritize data integrity over site usability. GitHub explicitly chose “frustrated but not defrauded” users. The majority of GitHub traffic is reads, so most users saw a slow but functional site. The post-incident actions included accelerating active/active multi-DC design and investing in chaos engineering to validate failure scenarios before they happen in production⁹.

This is the model for graceful degradation done right: a pre-planned operating mode, a clear trigger, a defined scope of impact, and a recovery path.

Defense in depth

Graceful degradation is not a single technique you pick from a menu. Mature services stack these layers on top of one another so the failure of one does not bring the whole system down. The chapter’s architecture diagram describes these as “three reinforcing layers”; the table below reflects that. Every row is additive, and the “Our Pick” column says when to add each layer, not which one to choose instead of the others.

Common Pitfalls

Exercise

Design the degradation plan for an e-commerce site: list features by priority, specify what gets shed at 70% capacity vs 90% vs database-down, and design the UX so customers understand what is happening without losing trust.

flowchart LR
    Client --> CDN["CDN edge<br/>stale-if-error"]
    CDN --> Shed["Edge shedder<br/>adaptive concurrency"]
    Shed --> Flags["Feature flag router"]
    Flags -->|Tier 1| Core["Core services"]
    Flags -->|Tier 2/3 disabled| Stub["Cached stub / empty"]
    Core --> DB[("Primary DB")]
    Core --> Cache[("Local cache<br/>last-known-good")]

Key Takeaways

• Degradation is a product decision dressed as a technical one. Classify features into tiers before the incident, not during.
• Shed early and cheaply. The worst case is accepting work you cannot complete, wasting resources, and still failing the client.
• Distinguish “degraded response” (intentional, tested, continuously exercised) from “fallback” (rarely-run code path full of latent bugs). AWS warns that fallback almost never helps⁷.
• Cached fallbacks buy time. Stale content is almost always better than errors for read-heavy paths.
• Read-only mode is underused. Most systems can degrade writes before reads, preserving data integrity while keeping the majority of users functional.
• A kill switch no one has flipped does not exist. Rehearse degradation quarterly. See Chaos Engineering and Game Days for the methodology.
• Communicate honestly. Silent degradation erodes trust faster than a banner saying “some features are limited.”

Further Reading

• Using load shedding to avoid overload - David Yanacek’s canonical primer on goodput, the knee, and rejecting early; the single best starting point for this topic.
• Avoiding fallback in distributed systems - Jacob Gabrielson’s counter-intuitive argument for why fallback usually makes outages worse, with the 2001 Amazon cache-to-DB disaster.
• Handling Overload (Google SRE Book, Ch. 21) - Criticality tiers, client-side throttling, and retry budgets from Google’s production experience.
• Feature Toggles - Pete Hodgson’s taxonomy of flag types and the carrying-cost argument; essential reading before building a flag platform.
• Netflix concurrency-limits - Adaptive shedding via TCP congestion-control algorithms; the modern replacement for static RPS ceilings.
• RFC 5861: HTTP Cache-Control Extensions for Stale Content - The formal spec for stale-while-revalidate and stale-if-error directives.
• GitHub October 21 post-incident analysis - 24 hours of deliberate read-only degradation; the best public example of “partial service beats no service.”
• Performance Under Load - Netflix on adaptive concurrency and the end of static timeouts; explains Vegas and Gradient2 algorithms.

Flashcards

References