AI-Detectors Biased Against Non-Native English Writers (2023) (hai.stanford.edu)

A Stanford study finds that popular "AI detectors"—seven tools rushed to market after ChatGPT—are both unreliable and systematically biased against non-native English writers. While detectors performed near-perfectly on essays by U.S.-born eighth-graders, they labeled 61.22% of TOEFL essays from non-native speakers as AI-generated; 19% of those essays were unanimously flagged by all seven detectors and 97% were flagged by at least one. The authors warn these tools could unfairly accuse or penalize foreign-born students and workers if used in high‑stakes educational or professional settings. Technically, the root cause is that many detectors rely on perplexity-based signals (metrics tied to lexical richness, diversity, syntactic and grammatical complexity) that correlate with “writing sophistication.” Non-native writers typically score lower on those measures, producing false positives. Detectors are also trivially circumvented by prompt engineering—asking a generator to “elevate” text increases perplexity and evades detection. The paper urges immediate caution in deploying detectors in education, calls for moving beyond perplexity to robust detection methods (or embedding watermarks in generative models), and recommends hardening models against adversarial rewriting and rigorous evaluation to avoid discriminatory outcomes.