Load Shedding
Reject lower-value work early so critical operations remain healthy during overload.
2–5 minute refresher
Timeout→Backoff→Jitter→Retry budget→Circuit breaker→Load shedding
Finite safe capacity → admit valuable work → reject before saturation
30second
refresher
refresher
Load shedding rejects, defers, samples, or degrades work when demand exceeds safe capacity. Admission decisions happen before expensive resources are consumed.
What problem does it solve?
Once a service is saturated, latency rises, callers time out and retry, queues grow, and useful throughput can collapse.
How it works
- Identify the saturation signal and safe concurrency envelope.
- Classify requests by priority, cost, deadline, and tenant.
- Reject early with a retryable or degraded response.
- Reserve capacity for critical operations and recovery probes.
- Ramp admission back gradually after pressure falls.
Decision guide
Key trade-offs
| Choice | What you gain | What it costs |
|---|---|---|
| Early rejection | Preserve latency and throughput | Visible errors |
| Priority queues | Protect critical journeys | Starvation risk |
| Graceful degradation | Retain partial value | More product states to operate |
What happens if?
Every caller retries rejected work immediately
Shedding becomes a synchronized retry storm. Return backoff guidance, apply jitter and retry budgets, and make overload responses explicit enough that clients do not treat them as ordinary transient errors.
Where it appears
- API admission control
- Database connection pools
- Search result degradation
- Brownout patterns
Senior interview modeWhere should load shedding happen?Show answer
As early as possible where request cost, priority, and system saturation are known—before queues, database connections, or downstream calls are consumed.