Coral: An LLM-Native Harness for Production Recommender Systems (arxiv.org)

A new framework, CORAL (Constraint-Optimized Recommender via an Agentic Loop), has been introduced to enhance production recommender systems by integrating large language models (LLMs) into a continuous optimization framework. This innovative approach addresses the challenge of adapting recommender systems to shifting content and user behaviors, which traditionally relies on slow, manual engineering efforts. CORAL enables a self-improving loop where an agent learns from live operational data, analyzes past decisions, and optimizes recommendations without the need for parameter updates, ultimately streamlining the recommendation process. The significance of CORAL lies in its ability to automate the ongoing optimization that has typically required human oversight. The framework demonstrated improved engagement rates and reduced serving costs across two large-scale social platforms during A/B testing. By balancing user engagement and efficiency, CORAL not only enhances performance metrics but also minimizes operational costs, suggesting a transformative shift in the way recommender systems can be developed and maintained. This advancement positions the AI/ML community to leverage LLM capabilities more effectively in real-time applications, leading to more responsive and intelligent systems.