Distributed Caching
Reduce latency and origin load while making staleness, invalidation, and failure behavior explicit.
2–5 minute refresher
Leaderaccept writes
replicate ↓
Follower A
Follower B
Follower C
A cache is a bounded, disposable replica with a deliberately weaker freshness contract.
30second
refresher
refresher
Distributed caches keep derived or copied data near readers. Their value comes from a clear freshness contract, stable key design, bounded memory, and predictable behavior when cached data or the cache fleet disappears.
What problem does it solve?
Without caching, popular reads overload origins. With careless caching, stale data violates invariants, hot keys stampede the origin, and cache failure becomes a fleet-wide incident.
How it works
- Classify data by freshness and safety requirements.
- Choose cache-aside, read-through, or write-through deliberately.
- Set TTLs with jitter and explicit invalidation where needed.
- Coalesce misses and protect origins with concurrency limits.
- Measure latency and origin work by hit, miss, stale, and error path.
Decision guide
Key trade-offs
| Choice | What you gain | What it costs |
|---|---|---|
| Long TTL | Higher hit rate | More staleness |
| Explicit invalidation | Faster freshness | Lost or reordered invalidations |
| Serve stale | Availability during failure | Weaker consistency |
What happens if?
A popular key expires everywhere
Thousands of requests miss simultaneously and regenerate the same value, overwhelming the database. Request coalescing, soft TTLs, jitter, and stale-while-revalidate limit the stampede.
Where it appears
- API response caches
- Feature stores
- Session caches
- Model artifact caches
Senior interview modeWhen should a cache fail open versus fail closed?Show answer
Fail open with stale data only when the product invariant allows it. Authorization, balances, and revocation often fail closed; feeds and recommendations can usually serve bounded stale results.