Self-Improving VLM Judges Without Human Annotations (arxiv.org)

A new framework has been introduced for developing self-improving Vision-Language Model (VLM) judges without relying on costly and often outdated human preference annotations. This innovative method utilizes self-synthesized data through an iterative process consisting of three key stages: generating diverse multimodal instruction-response pairs of varying quality, assessing and filtering these pairs based on reasoning accuracy, and training the judge model using only the highest quality judgments and reasoning traces. The approach shows promising results, enhancing the accuracy of the Llama-3.2-11B multimodal judge from 0.38 to 0.51 on the VL-RewardBench, surpassing even larger models like Llama-3.2-90B and GPT-4o. This advancement is significant for the AI/ML community as it alleviates the dependency on human annotations, which can slow down model iterations and become irrelevant as VLMs evolve. The success of this self-judging framework not only suggests the feasibility of creating models that continuously assess themselves but also highlights a path toward more efficient and scalable VLM development. This self-sufficient evaluation method could lead to faster iterations and improvements in various domains such as correctness, reasoning, and safety in AI applications.