The Writing Is on the Wall for Handwriting Recognition (newsletter.dancohen.org)

Researchers and archivists have long wrestled with handwritten text recognition (HTR): printed OCR reached ~99% accuracy years ago, but HTR systems typically stalled around ~80% because handwriting varies wildly by writer, era, and page layout. That appears to be changing. The article reports that multimodal large models like Google’s Gemini 3 Pro can now transcribe challenging historical manuscripts—George Boole’s letters, War Department correspondence (1784–1800), and many lines of Charles Carroll—often near-perfectly, and with a built-in “show thinking” explanation that walks through paleographic reasoning (assessing letter shapes, layout, context). Gemini still flags genuinely unreadable passages (e.g., heavy cross-writing in a Jane Austen letter), but its ability to infer page order, cursive conventions, and likely alternatives marks a big leap over earlier approaches (crowdsourcing or Transkribus-style neural models that required large per-script training corpora). For the AI/ML community this is important: it demonstrates strong generalization of multimodal models to noisy, variable handwriting without extensive writer-specific training, enabling scalable digitization and searchability of archives. Key technical implications include improved image-text alignment and contextual inference, more transparent model rationales via verbalized reasoning, and reduced annotation burden. Caveats remain—bleed-through, cross-writing, and name disambiguation can still defeat models, and the “show thinking” can mask uncertainty or produce plausible but incorrect rationales—so human oversight and provenance-tracking will be essential as these tools are adopted in scholarship.