DocumentAI Visual Benchmark - GPT 5.5, Gemini 3.5, Qwen... (www.maltebuettner.eu)

Recent advancements in DocumentAI have led to the development of a visual benchmark, analyzing the performance of popular AI models like GPT-5.5, Gemini 3.5, and Qwen. The benchmark focused on extraction and the accuracy of bounding boxes generated by these models, which are critical in determining how effectively they can process and understand visual information from documents. Utilizing the ExtractBench dataset and additional extractions via OpenRouter, various models were put to the test to gauge their capabilities in identifying and accurately delineating content on specific PDF pages. This benchmarking is significant for the AI/ML community as it highlights the ongoing evolution of document processing technology, particularly in extracting relevant information and determining its placement within the document layout. Key metrics used in the evaluation included coverage of the fields generating bounding boxes, intersection-over-union (IoU) for assessing box accuracy, and centroid distance for assessing location fidelity. By utilizing a refined JSON schema that incorporates bounding box data points, the evaluation not only provides insights into the models' extraction capabilities but also informs future improvements in DocumentAI technologies, emphasizing the importance of precise visual comprehension in AI applications.