GEPA: System Optimization Through Reflective Text Evolution (github.com)

GEPA (Genetic-Pareto) introduces a novel framework for optimizing systems built from textual components—such as AI prompts, code snippets, or specifications—by leveraging large language models (LLMs) for reflective, feedback-driven evolution. Unlike conventional reinforcement learning approaches, GEPA iteratively mutates and selects candidates via a Pareto-aware evolutionary algorithm that uses LLM-generated reflections on system execution traces, including errors and performance signals, to guide improvements with minimal evaluations. This approach enables co-evolution of multiple modular components, making it highly effective for complex, domain-specific tasks. Technically, GEPA’s flexibility stems from its adapter-based architecture, allowing integration into various environments, from single-turn prompt tuning to evolving multi-step reasoning programs and agentic systems. For instance, GEPA boosted GPT-4.1 Mini’s accuracy on the AIME math benchmark from 46.6% to 56.6% in just two iterations by optimizing system prompts. In more complex settings, such as the DSPy Full Program Adapter, GEPA evolved multi-step reasoning chains achieving 93% accuracy on the MATH dataset—significantly surpassing baseline performance. The framework supports customized adapters for diverse applications, demonstrated by integrations with terminal-use agents and math problem solvers, showcasing its broad applicability. By combining evolutionary search with LLM-enabled reflective feedback, GEPA provides a powerful, sample-efficient alternative for optimizing textual components within AI/ML systems. Its open-source implementation and rich tutorial ecosystem encourage community-driven expansion and adaptation, positioning GEPA as a promising tool for pushing the boundaries of prompt and program optimization across various domains.