OCR Showdown: Cardinal vs. the Rest (trycardinal.ai)

Cardinal ran a head-to-head OCR benchmark against legacy engines (Tesseract, Textract), cloud OCRs (Azure, Mistral OCR), and modern LLM-based extractors (Gemini 2.5 Pro, GPT-5, Claude 4 Sonnet) on three intentionally hard images: handwriting with annotations and tables, complex spanning tables, and filled-in checkboxes. Results show Cardinal handling the messiest cases—filled checkboxes, multi-span table structure, and handwritten notes—more reliably than the alternatives. LLMs sometimes read handwriting better (Gemini) but routinely failed to emit reliable bounding boxes, hallucinated fields, struggled with long documents, and were costly to run; GPT-5 and Claude also had latency, hallucination, or cost issues. Azure and Textract were less prone to hallucination but performed poorly on handwriting and scanned/irregular docs; Tesseract and Mistral largely failed on the test set. The takeaway for the AI/ML community: OCR remains unsolved for real-world, messy documents where structure (bounding boxes, table spans), checkbox semantics, and handwriting robustness matter as much as plain text recognition. Pure LLM approaches face practical limits—lack of structured outputs, hallucination risk, latency and cost at scale—while traditional engines miss modern document complexity. Cardinal’s results highlight the importance of hybrid systems engineered for structured extraction and production constraints, not just raw text accuracy; the authors also publish scripts to reproduce the tests.