BabyVision: Visual Reasoning Beyond Language (unipat.ai)

A new benchmark called BabyVision has been introduced to evaluate the visual reasoning capabilities of multimodal large language models (MLLMs), particularly against young children's abilities. While today’s MLLMs excel in language tasks, they struggle in visual reasoning tasks that require understanding without relying on language. BabyVision focuses on tasks that minimize linguistic influence and assess fundamental visual skills across four core categories: Fine-grained Discrimination, Visual Tracking, Spatial Perception, and Visual Pattern Recognition, through a set of 388 detailed questions. Initial findings show that even the most advanced MLLMs, such as Gemini 3-Pro-Preview, perform at the level of a three-year-old, underscoring a significant gap in foundational visual competencies compared to human performance. This benchmark is significant for the AI/ML community as it highlights essential weaknesses in models that excel in language but fall short in basic visual tasks, drawing attention to the systemic limitations of current training and data paradigms. The research reveals that MLLMs struggle with visual reasoning due to their reliance on text, which often results in the loss of critical visual details and an inability to maintain consistent perceptual identity or 3D spatial awareness. By quantifying these gaps, BabyVision serves as a crucial tool to guide future advancements toward improving the visual reasoning capabilities of AI systems, ultimately pushing toward a more integrated form of multimodal understanding.