Functional
- Return ranked suggestions for a normalized prefix.
- Respect locale, safety policy, and product availability.
- Continuously incorporate popularity and catalog changes.
End-to-end walkthrough
Search Autocomplete
Return useful ranked suggestions within a keystroke-scale latency budget.
Frame before solving
Requirements & boundaries
Quality attributes
- End-to-end p99 below 50 ms.
- A bad model or index must roll back in minutes.
- No query should expose private or disallowed terms.
Explicitly out of scope
- Full document search results.
- Long-form semantic question answering.
Size the important constraints
Back-of-the-envelope estimates
Serve from memory with regional replicas.
Compress shared prefixes and tier the long tail.
Client debounce, cancellation, and cache matter.
Use separate fast safety overlays and slower full builds.
These are reference assumptions, not universal facts. In an interview or architecture review, change them when the product context changes.
Define the contract
API & data model
Core operations
/v1/suggest?q=&locale=&limit=Return ranked suggestions and index version./v1/eventsIngest privacy-safe impression and selection signals./v1/terms/{id}Urgently suppress a term through the policy overlay.Authoritative records
locale+prefix, candidate_ids, base_scores, snapshot_versionCompact top-K list keeps serving bounded.id, display_text, features, policy_flags, valid_untilSeparate text/features from repeated prefix postings.Trace the critical path
Architecture & request flow
- 1Normalize prefix→
- 2Route by language or market→
- 3Retrieve candidates→
- 4Apply policy filters→
- 5Rank and return
Query API
Enforce latency and safety budgets
Prefix index
Map prefixes to candidates
Ranking service
Blend popularity and context
Build pipeline
Continuously publish new index snapshots
Reason about the hard parts
Critical design deep dives
Index structure
A compressed trie or finite-state transducer shares prefixes; each node stores a bounded top-K candidate list. Partition first by locale, then by prefix range or hash while keeping common prefixes replicated.
Ranking path
Retrieve a few dozen candidates, apply hard policy and availability filters, then use a small model to rerank. Enforce a strict deadline and return base popularity order if features or model are late.
Index publication
Build immutable snapshots, validate offline quality/safety, canary by traffic slice, then atomically switch a version pointer. Emergency suppressions live in a tiny overlay that updates independently.
Make trade-offs explicit
Architecture decisions
ChoiceWhyCost
Precomputed prefix indexMakes serving predictableFreshness follows the build cadence
Snapshot rolloutEnables atomic, reversible updatesRequires double capacity during swaps
Failure-first review
What happens if…?
Ranker exceeds budget
Return cached popularity results instead of timing out the request.
Bad index ships
Canary snapshots and atomically roll back the active version.
Avoid premature complexity
How the design evolves
Catalog scale
Database prefix query with cache
Move here when: Small corpus and modest QPS.
Dedicated index
In-memory prefix snapshots per locale
Move here when: Latency and query load rise.
Personalized global
Base index + bounded reranker + policy overlay
Move here when: Quality and market-specific relevance matter.
Test the reasoning
Interview follow-ups
How do you handle a one-character prefix?
Strong answer signal: Replicate/cache hot nodes, cap candidates, and apply stronger rate limits.
How does removal propagate faster than rebuild?
Strong answer signal: High-priority deny/suppression overlay checked at serving time.
What happens when the ranker fails?
Strong answer signal: Deadline, circuit breaker, and deterministic base-ranking fallback.
Build from primitives