Bulkheads & Fault Isolation
Partition resources and blast radius so one tenant, dependency, or workload cannot sink the whole service.
2–5 minute refresher
Timeout→Backoff→Jitter→Retry budget→Circuit breaker→Load shedding
Shared capacity shares failure; partitions turn a fleet-wide outage into a bounded local event.
30second
refresher
refresher
Bulkheads reserve independent capacity for tenants, priorities, dependencies, or request classes. When one compartment saturates, the rest retain enough resources to serve useful work and recover.
What problem does it solve?
A shared thread pool, connection pool, queue, or retry budget lets one slow dependency or noisy tenant consume every worker and create correlated failure.
How it works
- Identify the scarce resource and failure unit.
- Choose isolation dimensions: tenant, dependency, priority, or cell.
- Set per-compartment concurrency and queue bounds.
- Reserve recovery and control-plane capacity.
- Rebalance cautiously using measured demand and fairness policy.
Decision guide
Key trade-offs
| Choice | What you gain | What it costs |
|---|---|---|
| Fine isolation | Small blast radius | Stranded capacity |
| Shared overflow | Higher utilization | Failures can cross boundaries |
| Priority reserve | Protects critical work | Lower classes shed earlier |
What happens if?
The pool is separate, the database is not
Two APIs use independent executors but share one database connection pool. A slow query consumes every connection, so the supposed bulkhead does not contain the failure.
Where it appears
- Multi-tenant SaaS
- API gateways
- Dependency clients
- Background job fleets
Senior interview modeHow do bulkheads differ from circuit breakers?Show answer
A circuit breaker stops calls based on dependency health. A bulkhead limits how much shared capacity those calls can consume. They complement each other: one controls admission, the other contains resource damage.