Show HN: A/B test your own VLMs for document parsing (Self-hosted Arena) (github.com)

A developer has introduced DocParse Arena, a self-hosted platform that allows users to A/B test various document parsing models, including global leaders like Claude, GPT, and Gemini, directly against their own documents. This platform enables side-by-side comparisons through blind evaluations, delivering results in real-time via streaming technology. Built predominantly with Claude Code, it supports diverse models and offers features like ELO ranking for matchups, custom prompt management, and multi-provider integration. The significance of DocParse Arena lies in its democratization of model evaluation, allowing researchers and developers to assess self-hosted VLMs (Vision Language Models) in a controlled, competitive environment without the limitations of existing platforms. Key technical features include real-time token streaming of OCR results, automatic handling of PDF documents, and a Docker one-click deployment process. This innovation not only enhances the evaluation process but also encourages collaboration and transparency in the AI/ML community, potentially accelerating advancements in document parsing technologies.