Scanning for Pangram Errors (veryfineprint.substack.com)

Pangram, a new AI detection tool, has gained attention for its ability to identify AI-generated writing with an impressive false positive rate of just 1 in 10,000. Recently, the tool was tested against over 2.8 million words from old books that had never been digitized, providing a unique benchmark for its accuracy. The results reinforced Pangram's claims, as most segments were correctly labeled as human-written, and any AI flags resulted from inaccuracies introduced during the optical character recognition (OCR) process rather than misidentification by Pangram itself. The significance of Pangram lies in its potential to address the rising prevalence of unlabeled AI-generated works in creative communities, such as the romance book sector. Despite skepticism about AI detection tools—due to past overconfidence from vendors—this latest testing approach alleviates concerns that Pangram might overly rely on memorized data from its training set. While the company advocates for human oversight in verifying Pangram's results, the low false positive rate suggests it could be a valuable resource for writers and publishers in navigating the increasingly complex literary landscape influenced by AI.