Stop tuning prompts by hand. Engineer the loop that tunes them (github.com)

A new approach to automatic prompt optimization has emerged, eliminating the need for manual tuning in AI models. By utilizing DSPy and GEPA, researchers can automate the iterative process of refining prompts which traditionally involved writing, testing, and modifying prompts manually. The system requires only inputs such as a starting prompt, labeled examples, a scoring metric, and feedback on previous attempts, allowing it to optimize prompts autonomously. This shift places humans in an orchestrator role, defining the objectives while the system fine-tunes the wording to enhance clarity and accuracy. This technique proved significant in tests, where a simple initial prompt dramatically evolved into a comprehensive unfairness rubric without any human intervention. Automation improved the unfair-recall metric from 65% to an impressive average of 86.5%, showcasing the ability of the system to uncover complex decision rules hidden in the task. The results indicate that prompt optimization should be leveraged particularly for tasks involving ambiguous or non-obvious criteria, positioning prompts more like software components that can be systematically improved rather than fixed templates. This innovation reflects a broader trend in treating AI prompts as dynamic entities capable of continuous refinement, which could greatly enhance model performance across various applications in the AI/ML community.