ClockBench – Visual Reasoning AI Benchmark (github.com)

ClockBench is a newly introduced visual reasoning benchmark designed to evaluate AI models on their ability to interpret analog clocks. The public dataset includes 10 clocks sampled from a larger private set of 180, which remains restricted to prevent data leakage into training sets. This careful curation ensures the benchmark remains a reliable test for model generalization in time-reading tasks, addressing a nuanced aspect of visual comprehension often overlooked in AI evaluation. The benchmark provides an easy-to-use framework with two main scripts: one to run evaluations via OpenRouter API, where users specify their API key and model choice, and another to grade the resulting output. Both scripts generate detailed JSON reports to facilitate performance analysis. This streamlined setup supports rapid experimentation and benchmarking for developers focused on visual reasoning capabilities, particularly in interpreting analog time displays—a challenging task for current AI systems. By introducing ClockBench, the community gains a targeted, well-structured tool for probing models’ spatial and temporal reasoning in visual contexts, fostering advances in AI interpretability and multimodal understanding. With its open-source nature and invitation for contributions, ClockBench encourages collaborative development and continuous improvement, making it a significant addition to AI/ML benchmarking resources.