Design a Multi-Tenant SaaS Platform

TL;DR. A multi-tenant SaaS platform hosts 50,000 customer organizations on shared infrastructure while enforcing per-tenant isolation, tiered SLAs (99.5% free to 99.99% enterprise), and exactly-once metered billing. The pivotal decision is where each tenant sits on the pool-to-silo spectrum: pool (shared DB with Postgres RLS) for cost efficiency, bridge (schema-per-tenant) for moderate isolation, and silo (dedicated cluster) for enterprise compliance¹. Salesforce runs hundreds of thousands of orgs on a shared multitenant database with governor limits²; Shopify isolates millions of stores in fully independent pods³. The cardinal sin is cross-tenant data leakage, a missing WHERE tenant_id = ? that returns another customer’s data.

Learning Objectives

• Design a tenant-isolation architecture that maps silo, bridge, and pool models to pricing tiers and unit economics
• Implement tenant routing with tenant_id propagation that survives async hops, cache layers, and background jobs
• Build a metering pipeline achieving >99.99% billing accuracy using idempotent, exactly-once event semantics
• Contain noisy neighbors with per-tenant quotas across CPU, connections, and API rate limits
• Justify when to promote a “whale” tenant from pool to silo with zero-downtime migration
• Prove GDPR-compliant deletion via crypto-shredding for append-only stores

Intuition

A multi-tenant SaaS looks like a trivial CRUD app with a tenant_id column. It handles 10 tenants fine. At 50,000 tenants spanning a 200,000x range in data volume (5 MB free trial to 1 TB enterprise), the architecture collapses in three ways.

First, isolation: a single missing tenant filter in a query, cache key, or background job leaks one customer’s data to another. Unlike a single-tenant bug that shows too much of your own data, a multi-tenant leak shows someone else’s data. That is an immediate SOC 2 finding, contract termination, and potential regulatory action⁴.

Second, economics: the top 10% of tenants drive 80% of load. If you silo every tenant, you pay for 50,000 idle database clusters. If you pool everything, one enterprise’s batch import saturates the shared connection pool and degrades service for 49,999 others.

Third, billing: every API call, storage byte, and compute second becomes a line item on an invoice. A 0.1% metering drift on $100M ARR is $100K/year of invisible revenue leak or customer overcharge⁵.

The insight that unlocks the design: tenancy is a spectrum, not a switch. Run all three isolation models simultaneously. Free tenants land in the pool (cheap, shared, RLS-enforced). Pro tenants get bridge isolation (schema-per-tenant). Enterprise whales get silo (dedicated infrastructure). Price the spectrum. Let tenants promote themselves by paying more, or auto-promote them when their load threatens the pool.

Requirements

Clarifying Questions

• Q: Greenfield or retrofit? Assume: Greenfield. Retrofitting multi-tenancy onto a single-tenant system is harder but uses the same primitives.

• Q: Pricing model? Assume: Hybrid. Seat-based base fee plus usage-based overage (API calls, storage). Stripe Billing integration.

• Q: Compliance targets? Assume: SOC 2 baseline for all tiers. HIPAA and data residency available on enterprise tier only.

• Q: Data residency? Assume: EU tenants pinned to EU region. US default. Enterprise can choose region at signup.

• Q: Noisy-neighbor guarantee? Assume: Enterprise tier gets guaranteed isolation (silo). Pool tiers get best-effort with hard rate limits.

• Q: Tenant customization? Assume: Custom fields via metadata (no runtime DDL). White-label branding on enterprise tier.

Functional Requirements

• Tenant signup with self-service provisioning (free/pro) and sales-assisted onboarding (enterprise)
• Tenant-scoped resource management: users, data, API keys, audit logs
• Per-tenant usage metering feeding automated invoicing
• Tenant admin controls: invite users, assign roles, view usage dashboard
• Tenant deletion with cascading cleanup and GDPR-grade erasure within 30 days⁶

Non-Functional Requirements

• Tenants: 50,000 active, ranging from 1-user free trials to 100K-user enterprises
• Load: 50K req/sec peak (business hours); 14K sustained
• Latency: p50 < 50 ms, p99 < 200 ms for tenant-scoped reads
• Availability: 99.5% (free), 99.9% (pro), 99.99% (enterprise)
• Billing accuracy: >99.99% (drift < 0.01% per billing period)
• Isolation: zero cross-tenant data leakage; enforced at database layer, not application trust

Capacity Estimation

Key ratios:

• Pareto distribution: 10% of tenants generate 80% of load. Design quotas around the 90th percentile, not the mean.
• Read:write ratio: ~10:1 for typical SaaS workloads; metering is append-only at 7K events/sec.
• Tenant router cache hit rate: must exceed 99.5% to avoid becoming a bottleneck⁷.

API and Data Model

API Design

POST /v1/tenants
  Body: { "name": "Acme Corp", "tier": "pro", "region": "eu-west-1" }
  Returns: 201 { "tenant_id": "t_abc123", "status": "provisioning" }

GET /v1/tenants/{tenant_id}
  Returns: 200 { "tenant_id": "...", "tier": "pro", "usage": {...}, "quota": {...} }

POST /v1/tenants/{tenant_id}/users
  Body: { "email": "alice@acme.com", "role": "admin" }
  Returns: 201 { "user_id": "u_xyz", "invite_sent": true }

GET /v1/tenants/{tenant_id}/usage?month=2026-05
  Returns: 200 { "api_calls": 1420000, "storage_gb": 42.3, "billable_total": "$847.20" }

DELETE /v1/tenants/{tenant_id}
  Returns: 202 { "deletion_job_id": "del_789", "estimated_completion": "2026-06-03" }
  Side-effect: async erasure across all stores within 30 days

Context propagation: every authenticated request carries tenant_id in JWT claims. The API gateway extracts it, validates against the route, and sets X-Tenant-Id header for downstream services. Middleware rejects any request reaching the data layer without a tenant context.

Data Model

-- Tenant metadata (PostgreSQL, control plane)
CREATE TABLE tenants (
  tenant_id   UUID PRIMARY KEY,
  name        TEXT NOT NULL,
  tier        TEXT NOT NULL CHECK (tier IN ('free','pro','enterprise')),
  region      TEXT NOT NULL,
  status      TEXT NOT NULL DEFAULT 'provisioning',
  shard_id    INT,
  created_at  TIMESTAMPTZ DEFAULT NOW()
);

-- Pool model: shared tables with RLS
ALTER TABLE orders ENABLE ROW LEVEL SECURITY;
ALTER TABLE orders FORCE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON orders
  USING (tenant_id = current_setting('app.tenant_id')::uuid)
  WITH CHECK (tenant_id = current_setting('app.tenant_id')::uuid);

erDiagram
    TENANT ||--o{ USER : has
    TENANT ||--o{ API_KEY : owns
    TENANT ||--o{ USAGE_EVENT : generates
    TENANT {
        uuid tenant_id PK
        text name
        text tier
        text region
        int shard_id
    }
    USER {
        uuid user_id PK
        uuid tenant_id FK
        text email
        text role
    }
    USAGE_EVENT {
        uuid event_id PK
        uuid tenant_id FK
        text event_type
        int quantity
        timestamp ts
    }

High-Level Architecture

flowchart LR
    Client --> Edge[Edge LB + TLS]
    Edge --> Router[Tenant Router<br/>JWT + shard map cache]
    Router -->|tier=free/std| Pool[(Pool DB<br/>Postgres + RLS)]
    Router -->|tier=pro| Bridge[(Bridge DB<br/>schema per tenant)]
    Router -->|tier=enterprise| Silo[(Silo DB<br/>dedicated cluster)]
    Router --> Quota[Quota Service<br/>Redis token buckets]
    Router -.usage events.-> Kafka[Kafka]
    Kafka --> CH[(ClickHouse)]
    CH --> Recon[Reconciler]
    Recon --> Stripe[Stripe Billing]
    subgraph Control["Control Plane"]
        Onboard[Onboarding<br/>State Machine]
        Admin[Admin API]
        Audit[(Audit Log S3)]
    end
    Admin --> Router
    Onboard --> Router

Write path: A tenant’s API request hits the edge load balancer, which terminates TLS and forwards to the tenant router. The router extracts tenant_id from the JWT, looks up the tenant’s tier and shard assignment in a hot cache (Redis, >99.5% hit rate), enforces the per-tenant rate limit, then routes to the appropriate data plane (pool, bridge, or silo). The data plane sets SET LOCAL app.tenant_id = ? before executing any query, activating RLS policies⁸⁹.

Read path: Identical routing. For pool tenants, Postgres RLS silently filters rows. For silo tenants, the connection targets a dedicated cluster. Cache keys always include tenant_id as a prefix to prevent cross-tenant cache collisions.

Async path: Usage events flow from the application layer into Kafka, partitioned by tenant_id. ClickHouse consumers deduplicate by event_id and append to tenant-partitioned tables. A nightly reconciler aggregates usage and pushes meter events to Stripe with idempotency keys⁵.

Deep Dives

Tenant isolation: pool, bridge, and silo in one platform

The isolation model is the single decision that determines cost structure, blast radius, compliance posture, and operational complexity. AWS SaaS Lens formalizes three models¹¹⁰:

Pool (shared DB + RLS): All tenants share tables. A tenant_id column on every row plus Postgres RLS policies enforce isolation at the database layer⁸⁹. The application sets SET LOCAL app.tenant_id = '<uuid>' at connection checkout. RLS evaluates the USING clause per row before any user predicate; rows that fail are silently filtered. FORCE ROW LEVEL SECURITY closes the table-owner bypass loophole⁸. Admin scripts, bulk jobs, and reporting queries all inherit the filter automatically⁸.

Bridge (schema-per-tenant): Each tenant gets a dedicated Postgres schema inside a shared cluster. Stronger isolation than RLS (no accidental cross-schema queries), easier per-tenant migrations, but more schema objects to manage¹¹¹².

Silo (dedicated cluster): Enterprise tenants get their own database instance, Redis, and optionally their own Kubernetes namespace. Hardest isolation, cleanest cost attribution, required for HIPAA or data-residency compliance⁴.

A mature platform runs all three simultaneously. Tier assignment:

flowchart LR
    subgraph Pool
      Free[Free trial]
      Std[Standard]
    end
    subgraph Bridge
      Pro[Pro]
      Team[Team]
    end
    subgraph Silo
      Ent[Enterprise]
      Ded[Dedicated]
    end
    Pool -->|"promote: CPU>30% or data>1TB"| Bridge
    Bridge -->|"promote: contract or compliance"| Silo

Promotion mechanics: When a pool tenant’s CPU exceeds 30% of the shared cluster or data exceeds 1 TB, the system triggers a migration. The pattern is dual-write to both old and new locations, backfill historical data, verify checksums, cut reads over, then clean up. Shopify’s Pod Mover migrates a pod between data centers in under a minute without dropping requests³.

Noisy-neighbor containment

In the pool model, any tenant can theoretically consume 100% of shared resources. Defenses are layered:

API rate limits: Per-tenant token buckets in Redis. Free tier: 100 req/sec. Pro: 1,000 req/sec. Enterprise: unlimited (silo absorbs it). Overage returns structured 429s with Retry-After headers.

Connection pool quotas: pgbouncer configured with per-tier pool sizes. Free tenants share a 50-connection pool. Pro tenants get 200 connections. Enterprise tenants connect to their dedicated instance with 500 connections.

Statement timeouts: SET LOCAL statement_timeout = '10s' per tenant context. A runaway query from one tenant cannot hold connections indefinitely.

Governor limits (Salesforce model): Hard caps enforced at the runtime layer before execution. Salesforce limits each transaction to 100 SOQL queries, 50,000 total rows across all SOQL queries per transaction, and 10,000 ms CPU¹³. Queries estimated to exceed limits are refused before they run, not killed mid-execution.

Cellular architecture (Shopify model): Shopify’s pods are fully isolated datastores (MySQL shard, Redis, Memcached) serving a subset of shops. No cross-pod runtime communication. A noisy merchant’s load cannot reach across pods³.

Metering pipeline and exactly-once billing

Metering is billing. Every usage event becomes a dollar on an invoice. The pipeline must be exactly-once: double-counting overcharges customers; under-counting leaks revenue.

Architecture:

sequenceDiagram
    participant App
    participant Kafka
    participant CH as ClickHouse
    participant Recon as Reconciler
    participant Stripe
    App->>Kafka: event{tenant_id, qty, ts, event_id}
    Kafka->>CH: consume, dedupe by event_id
    Note over CH: append-only, tenant-partitioned
    Recon->>CH: SUM qty GROUP BY tenant, period
    Recon->>Stripe: push meter events with identifier
    Stripe-->>Recon: recorded usage
    Recon->>Recon: diff internal vs Stripe, alert if >0.01%

Idempotency: Stripe’s Meter Event API accepts an identifier field per event. If a caller retries on timeout, the duplicate identifier is silently dropped⁵. Without it, Stripe auto-generates one, but the caller cannot dedupe across retries.

Rate limits: Stripe’s v1 meter endpoint handles 1,000 events/sec; the v2 stream path handles 10,000 events/sec⁵. For 7K events/sec aggregate, batch and fan out across multiple Stripe customer meter streams.

Reconciliation: A nightly job compares ClickHouse aggregates against Stripe-recorded usage per tenant. Drift beyond 0.01% triggers an alert. At $100M ARR, 0.01% is $10K/month, the threshold where customers notice and dispute.

Timestamp tolerance: Stripe accepts events with timestamps within the past 35 days and no more than 5 minutes in the future⁵. This accommodates clock drift and late-arriving events from mobile clients.

Tenant onboarding as a state machine

Provisioning is modeled as an explicit, idempotent, checkpointable state machine. Each step has an idempotency key; failures retry from the last successful checkpoint.

stateDiagram-v2
    [*] --> Signup
    Signup --> EmailVerify: submit
    EmailVerify --> PaymentMethod: verified
    PaymentMethod --> ProvisionCompute: card on file
    ProvisionCompute --> SeedData: ready
    SeedData --> Welcome: seeded
    Welcome --> TrialStart: sent
    TrialStart --> Converted: paid
    TrialStart --> Suspended: expired
    Converted --> [*]
    Suspended --> [*]
    ProvisionCompute --> ProvisionCompute: retry on failure
    SeedData --> SeedData: retry on failure

Pool tenants provision in seconds (a metadata insert plus config rows). Silo tenants provision in minutes to hours (spin up a database, configure backups, set up DNS). This asymmetry is why enterprise tenants sign contracts before provisioning and free tenants self-serve into the pool.

Real-World Example

Shopify Pods: cellular multi-tenancy for millions of stores.

Shopify’s journey illustrates the full isolation spectrum. Until 2015, Shopify ran on a single MySQL database. In 2015, they sharded by shop ID. But sharding alone left the application brittle: any Sharding.with_each_shard code fanned out to all shards and failed if any one was down³.

In 2016, Shopify reorganized into pods. A pod is a fully isolated set of datastores (MySQL shard, Redis, Memcached) serving a subset of shops. Stateless workers can talk to any pod, but each unit of work (request or job) is pinned to exactly one pod via the Sorting Hat load balancer, which matches each request to a pod using rules and injects a header³.

The key insight: no cross-pod runtime communication. A failure in Pod 7 cannot cascade to Pod 12. Each pod has a paired data center for DR. Pod Mover migrates a pod between data centers in under a minute without dropping requests or jobs³.

For their largest merchants (Shopify Plus “whales”), Shopify provides premium isolation: effectively their own pod or a lightly populated one. This is the silo model dressed in cellular clothing.

The contrast with Salesforce is instructive. Salesforce runs hundreds of thousands of customer organizations on a single shared multitenant database with a single schema, using metadata-driven virtual schemas and governor limits². Salesforce chose radical pooling with runtime enforcement. Shopify chose radical isolation with cellular boundaries. Both work at massive scale. The difference is the failure mode: Salesforce’s governor limits prevent noisy neighbors but create a complex runtime; Shopify’s pods prevent cross-tenant interference structurally but require pod-aware tooling for every operation.

Trade-offs

The biggest meta-decision: pool vs. silo is not a one-time choice. Real SaaS runs all three simultaneously, assigning tenants by tier. The architecture must support promotion (pool to silo) without downtime. Design the migration path before the first whale forces it at 2 a.m.

Scaling and Failure Modes

At 10x (500K tenants, 500K req/sec):

• Tenant router cache becomes the bottleneck. Mitigation: Atlassian’s approach with in-process sidecar caches achieving >99.5% hit ratio and 11 microsecond p50 latency⁷. Background refresh prevents cold-cache storms.
• Pool DB connection limits saturate. Mitigation: promote heavy tenants to bridge; add read replicas for pool.

At 100x (5M tenants, 5M req/sec):

• Single-region architecture fails availability targets. Mitigation: cell-per-AZ architecture (Slack model). Each AZ runs a full stack; edge Envoys drain a failing AZ by reweighting in seconds¹⁴.
• Metering pipeline Kafka partitions become hot. Mitigation: re-partition by (tenant_id, event_type) for better distribution; increase partition count.

At 1000x:

• Architectural rewrite: move to Shopify-style pods where each pod is a self-contained unit. Global control plane for tenant CRUD; data plane fully isolated per pod.

Failure modes:

• Cross-tenant data leak: Immediate P0 incident. Response: revoke affected tenant’s sessions, audit all queries in the window, notify the supervisory authority within 72 hours per GDPR Article 33, and notify affected data subjects without undue delay per Article 34¹⁵. Prevention: RLS at DB layer, not application trust⁹.
• Tenant router cache poisoned: Atlassian’s 2019 TCS incident: a data migration brought the service up with empty tables; sidecars cached the empty state and products went offline across regions⁷. Mitigation: anomaly detection on 200/404 response ratios; dummy “content-check” keys fetched periodically.
• Billing pipeline drift: Stripe-recorded usage diverges from internal totals. Response: halt invoicing, run reconciliation, issue credits. Prevention: idempotency keys on every meter event⁵.

Common Pitfalls

Follow-up Questions

1. How do you migrate a tenant from pool to silo with zero downtime?

Dual-write to both pool and silo. Backfill historical data from pool to silo. Verify row counts and checksums. Cut reads to silo (shadow-read pool for validation). Stop writes to pool. Clean up pool rows. Shopify’s Pod Mover achieves this in under a minute³.

2. A tenant reports “my users can see another tenant’s data.” What is the incident response?

Immediately revoke all sessions for both affected tenants. Identify the leaking query via audit logs. Determine blast radius (how many tenants, how many rows, what time window). Notify the supervisory authority within 72 hours per GDPR Article 33; notify affected data subjects without undue delay per Article 34¹⁵. Root-cause: was RLS disabled? Was a cache key missing tenant_id? Was a background job running without tenant context?

3. What is the unit cost per tenant in pool vs. silo?

Pool tenant costs ~$0.50/month in shared infrastructure. Silo tenant costs ~$200/month for a dedicated RDS instance, Redis, and monitoring. The 400x cost difference justifies the pricing page: enterprise tier at $5K/month can absorb silo cost; free tier at $0 cannot.

4. How do you implement per-tenant SAML/OIDC SSO without complicating the pool tier?

Enterprise tenants configure their IdP (Okta, Azure AD) in the admin console. At login, the system resolves the tenant from the email domain or subdomain, redirects to the tenant’s IdP, and receives a SAML assertion. Pool tenants use the platform’s built-in auth. The tenant router handles both paths transparently.

5. How do you handle nested tenancy (holding company with subsidiaries)?

Model as a tree: parent org owns child orgs. Billing rolls up to the parent. Data isolation remains per-child (each child is a separate tenant_id). Cross-child visibility requires explicit grants. Salesforce’s “org” model supports this via managed packages and connected apps.

6. How do you implement crypto-shredding for GDPR erasure?

Encrypt each tenant’s data with a tenant-scoped data key derived from a per-service CMK in AWS KMS¹⁸. On deletion, destroy the tenant’s data key. Ciphertext remains in Kafka topics, S3 audit logs, and immutable backups but is computationally unrecoverable¹⁶. For enterprise tenants, offer BYOK where the CMK lives in the customer’s own KMS account¹⁹. Salesforce provides per-org encryption keys for Platform Encryption²⁰. GDPR Article 17 requires erasure “without undue delay”; the common interpretation is 30 days⁶.

Exercise

Exercise 1: Design the noisy-neighbor detection and response

Your platform runs 5,000 tenants in a shared Postgres pool. One tenant’s analytics dashboard query is scanning 50M rows and holding connections for 30+ seconds, causing p99 latency to spike from 200 ms to 8 seconds for all other tenants. Design the detection, immediate response, and long-term prevention.

Key Takeaways

• Tenancy is a spectrum, not a switch. Pool is cheap and risky; silo is safe and expensive. Run all three simultaneously and price the isolation level.
• tenant_id propagation is a safety property. Enforce it at the database layer via RLS, not in application code. Test cross-tenant reads fail closed.
• Metering is billing. Exactly-once semantics with idempotency keys and monthly reconciliation are non-negotiable at SaaS scale⁵.
• Noisy-neighbor containment requires layered defenses. Rate limits, connection quotas, statement timeouts, and governor limits together, not any one alone.
• Design promotion paths before the first whale. Pool-to-silo migration at 2 a.m. under pressure is how you lose your largest customer.
• Crypto-shredding solves GDPR for append-only stores. Destroy the key, orphan the ciphertext¹⁶.

Further Reading

• AWS SaaS Tenant Isolation Strategies. The canonical silo/bridge/pool taxonomy with concrete implementation patterns for each model.
• Shopify: A Pods Architecture To Allow Shopify To Scale. The original pods post explaining cellular isolation, Sorting Hat routing, and Pod Mover migration.
• Atlassian: Six-Nines Availability via Tenant Context Service. The deepest public writeup of a tenant-metadata service at scale, including poisoned-cache detection and recovery.
• PostgreSQL Row Security Policies. The reference for RLS: USING vs WITH CHECK, FORCE ROW LEVEL SECURITY, and interaction with the query planner.
• Stripe: Record Usage with the API. Primary source on idempotent meter events, rate limits, and timestamp tolerance windows.
• Slack: Migration to a Cellular Architecture. Why AZ became the cell boundary and how Envoy xDS enables seconds-order traffic draining.
• Conduktor: Crypto Shredding for Kafka. Practical implementation of GDPR-compliant deletion in append-only streaming systems.
• Salesforce Platform Multitenant Architecture. Metadata-driven design, governor limits, and the flex-column data model at 150K+ orgs.

Flashcards

References