Training AI to Paint with Code (surya.website)

A new AI initiative has emerged, allowing users to create images by editing code rather than relying solely on text prompts. This project utilizes a trained language model to produce JavaScript sketches that render images, employing reinforcement learning (RL) to refine its outputs. The system operates through a four-step loop where prompts lead to code generation, which is then evaluated against hand-rated reference paintings. This method allows for more granular customization and engagement in the image creation process, challenging the traditional interface of AI art generation. The significance of this project lies in its exploration of RL application to creative tasks, which pose unique challenges in establishing verifiable reward systems. The team's findings suggest that balancing the rigidity and flexibility of reward functions is critical; overly strict criteria limit creativity, while too vague a framework leads to aimless outputs. By adapting their training methodology—such as optimizing the system prompt and employing a carefully curated pool of reference images—the researchers hope to enhance the model's ability to generalize aesthetic preferences effectively. Although the process is slower than conventional image generation methods, it represents a pivotal shift in how creativity can be harnessed within AI systems. A comprehensive technical report is set for release on June 26.