Following the Text Gradient at Scale (ai.stanford.edu)

A new paradigm in reinforcement learning (RL) is gaining traction with the introduction of Feedback Descent, which leverages rich, natural language feedback rather than relying solely on scalar rewards. Traditional RL methods tend to overlook valuable evaluative details by compressing feedback into a single numerical score, which can obscure the underlying reasons for successes and failures in tasks. With Feedback Descent, both evaluators and editors collaborate to provide structured feedback that highlights specific improvement areas, leading to more targeted adjustments. This approach not only shows significant promise in reducing the number of trials needed for better outcomes but also allows the integration of complex knowledge in a way that scalar methods cannot. Demonstrated across multiple domains—including molecular design, SVG image optimization, and prompt optimization—Feedback Descent outperformed established RL methods, achieving an average 3.8 times reduction in docking calls for molecular tasks and excelling in creative optimization challenges. By allowing systems to continuously improve based on detailed textual feedback, this method illustrates a shift toward a more sophisticated understanding of the learning process, indicating that text can serve as an effective medium for driving advancements in AI, particularly for complex and resource-intensive tasks like computational drug discovery.