How Retrieval Works in Modern Ranking Systems (www.shaped.ai)

This piece outlines the retrieval stage of modern ranking systems: the engineering and ML stack that, in tens of milliseconds, reduces a corpus of millions or billions of items to a high-recall slate (typically ~500–1,000 candidates) that’s likely to contain the best possible matches. The system buys online speed with heavy offline compute — training retrieval models, precomputing d-dimensional item embeddings, and building indices (ANN libraries like FAISS/ScaNN for vectors, inverted indexes like Lucene/Elasticsearch for attributes), plus simple heuristic lists stored in KV stores (e.g., Redis). The key design choice is to favor recall over precision so downstream scoring can do the fine-grained ranking. Online, retrieval runs a three-step flow: (1) pre-filtering applies cheap binary constraints (e.g., open, delivery zone, max time) to cut the search space; (2) candidate generation uses an ensemble of retrievers in parallel — heuristics (popular, trending, newest), memorization-based CF (precomputed co-occurrence/I2I matrices), generalization-based vector search (Two-Tower embeddings for semantic matches), and attribute/sparse search (keyword filters) — and (3) post-filtering removes seen items, enforces business/safety rules, then deduplicates and truncates to the candidate budget. The result is a robust, low-latency, high-recall foundation for downstream scoring (DLRMs/Transformers) and highlights trade-offs and engineering decisions critical for scalable personalization, fairness, and safety.