Image transcription humbled me, plus thoughts on the bitter lesson (digitalseams.com)

A developer tried to auto‑transcribe social‑media screenshots from a Bluesky dataset using Tesseract.js (client‑side OCR) and found the reality messier than early demos suggested. Using Tesseract’s bounding boxes and a conservative filter (confidence ≥0.80 and text boxes covering ≥20% of the image), an initial tweet example produced nearly perfect output, but applying it across ~10,000 images pared down to only ~100 “high‑confidence” candidates. Real‑world issues—UI chrome (timestamps, reaction counts, follow buttons), multi‑tweet threads and quote tweets, emojis, and varied layouts—frequently broke recognition. The author concludes many common formats would need bespoke image‑processing tweaks to be reliably transcribed on the client. This failure prompted reflection on the “bitter lesson”: handcrafted heuristics are often outpaced by scale. Today, multimodal LLMs (e.g., ChatGPT, Claude) give near‑perfect transcriptions but are compute‑heavy and impractical for client‑side use; in a few years lighter open‑source models may close that gap. The practical takeaway for the AI/ML community is a tradeoff: invest time building brittle, specialized pipelines now to improve accessibility, or accept the wait for scaled models that generalize better. The post underscores resource prioritization decisions and suggests hybrid approaches (specialized components plus scaling) may be the most pragmatic path forward.