MaLiang-Harness: Programmable Image and Video Generation (arxiv.org)

A recent announcement introduced MaLiang-Harness, a novel framework for programmable image and video generation that addresses the challenges associated with the Program-to-Visual (P2V) gap. This framework integrates a process of construction, inspection, and revision, allowing users to control how images and videos are created using executable programs. Central to MaLiang-Harness are mechanisms like the Persistent Executable Generation (PEG) state, which preserves program and task context, and Revision-aware Editing and Verification (REV), which ensures edits are connected to visual outputs. This organized approach not only enhances the construction process but also provides valuable visual feedback, making it easier to identify and correct discrepancies. The significance of MaLiang-Harness lies in its ability to systematically analyze how Machine Learning Language Models (MLLMs) transform executable code into visual content. Its evaluation of 11 leading closed-source MLLMs highlights stark contrasts in their performance, revealing that higher general capability scores do not always correlate with successful visual generation. Notably, the GPT-6-Astra model achieved an impressive 100% generation success across both evaluation benchmarks, with high-quality outputs in image and video tasks. This framework informs the AI/ML community about the limitations of current benchmarks in predicting visual generation abilities, paving the way for enhanced methodologies in programmable visual content creation.