The Shift from Models to Compound AI Systems (bair.berkeley.edu)

In a notable shift within the AI community, the focus is moving from developing individual Large Language Models (LLMs) to creating compound AI systems that combine multiple interacting components. This trend, highlighted by advancements from projects like Google’s AlphaCode 2 and AlphaGeometry, reveals that complex systems can generate state-of-the-art performance by employing techniques such as generating numerous potential solutions and filtering them or integrating traditional problem-solving methods with machine learning. Data indicates that about 60% of current LLM applications utilize retrieval-augmented generation (RAG) methods, showcasing a growing preference for intricate system designs that enhance the reliability and flexibility of AI outputs. This evolution from standalone models to compound systems is significant for the AI/ML community as it opens new avenues for performance improvement, control, and real-time adaptability. Rather than waiting for larger models to improve in capability, developers are discovering that system engineering can yield more significant benefits. This paradigm shift not only raises important questions about design optimization and resource allocation but also emphasizes the need for new tools and methods to manage and secure these complex systems as they become increasingly prominent in AI applications. As the industry embraces this trend, compound AI systems are predicted to play a pivotal role in shaping the future of artificial intelligence in 2024 and beyond.