LLMs as Retrieval and Recommendation Engines (medium.com)

Recent research is advancing the use of large language models (LLMs) as powerful retrieval and recommendation engines, moving beyond traditional keyword matching and embedding-based methods. Instead of relying on sparse (e.g., BM25) or dense vector similarity approaches (e.g., DPR), generative information retrieval (GenIR) uses LLMs to directly generate relevant item identifiers or titles from large catalogs based on user queries or profiles. This shift enables more nuanced understanding of complex, semantic queries and personalized recommendations by leveraging the LLM’s generative capabilities. Technically, GenIR combines retrieval and ranking either in a two-step process—LLM retrieves candidates, and a ranker orders them—or via an end-to-end unified model that simultaneously retrieves and ranks items. A key innovation is the use of semantic IDs, structured codes representing item attributes, allowing LLMs to generate compact, consistent outputs constrained to real catalog entries through techniques like prefix trees (tries). This constrained decoding mitigates hallucinations and ensures valid results. Fine-tuning LLMs on query–item pairs further improves performance, making these models competitive with or superior to classical approaches, especially for complex or nuanced user requests. The implications for AI/ML are significant: GenIR offers flexible, interpretable, and scalable retrieval and recommendation that blend language understanding with catalog structure. Leading tech firms are actively exploring this paradigm, already demonstrating substantial gains on benchmark datasets. As this field matures, especially with accessible open-weight models and code tutorials forthcoming, GenIR stands poised to redefine how search and recommendation systems deliver personalized, context-aware results.