OlmOCR 2: Unit test rewards for document OCR (allenai.org)

Allen AI released olmOCR 2 (olmOCR-2-7B-1025), an end-to-end vision-language OCR model that pushes state-of-the-art accuracy on real-world English document digitization by training directly on verifiable correctness. Built for complex layouts—multi-column pages, dense tables, math, degraded/historical scans—olmOCR 2 scores 82.4 on olmOCR-Bench (≈+4 points vs prior release) and posts large gains on hard cases (tables 84.9%, math scans 82.3%, multi-column 83.7%). The release includes model weights (FP8 and full precision), training code, datasets, and APIs (Hugging Face, DeepInfra, Parasail), plus a production-friendly FP8 quantized build that runs ~3,400 output tokens/sec on a single H100 (≈10,000 pages for <$2). The core innovation is training with unit-test-style, verifiable rewards: deterministic checks (e.g., table structure, equation fidelity, reading order) are used both to supervise optimization and to evaluate performance. They synthesized a labeled training mix (olmOCR-synthmix-1025) by re-rendering sampled PDFs to semantic HTML, deriving exact Markdown/HTML/LaTeX targets and 30k+ automatic test cases, then applied Group Relative Policy Optimization (GRPO) using pass/fail rewards. During training the model generates 28 completions per page and is rewarded for completions that pass more tests; auxiliary rewards encourage clean termination and metadata extraction. The result is a single-pass decoder that emits structured outputs (Markdown/HTML/LaTeX) directly, reducing brittle post-processing and making fine-tuning on your own pages straightforward.