Training a model to identify AI-generated web content from structure alone (arxiv.org)

Researchers have developed a new model, named SlopShape, capable of identifying AI-generated web content by analyzing its structural features rather than relying solely on word-level characteristics. This method builds upon previous findings that showed AI-generated texts exhibit specific patterns in how information is structured and presented. In a study involving over 13,000 blog posts—2,250 human-written and 11,250 AI-generated—the model achieved an impressive 98.0 macro-F1 score in detecting AI content purely from 187 derived structural features. Remarkably, its accuracy remains high even when AI text is reworded. This advancement is significant for the AI/ML community as it moves beyond traditional text-detection approaches, which often struggle with rephrased content. By focusing on structural signatures, SlopShape not only enhances the reliability of detection methods but also provides insights into how AI models generate content differently from humans. The team has made their tools, including the detection pipeline and associated code, publicly available, fostering an open approach to addressing the implications of AI in content creation and combatting misinformation.