Detecting LLM-Generated Web Novels Using "Classical" Machine Learning (AIGC Tex (blog.lyc8503.net)

An experimental project has demonstrated that traditional machine learning models can effectively detect texts generated by large language models (LLMs), distinguishing them from human-written content with an impressive accuracy of around 85%. This development is significant for the AI/ML community, particularly as concerns grow about the proliferation of AI-generated content on platforms like online fanfiction sites and academic papers. By leveraging techniques such as TF-IDF and classifiers like Linear SVC and Naive Bayes, the model was trained on both human-written texts and LLM-generated samples, achieving reliable detection rates even on unseen models. The implications of this tool are substantial, offering a potential solution for educators and content platforms facing challenges in identifying AI-generated work. The method's effectiveness significantly outpaces many existing online detection tools, which often rely on LLMs to evaluate the originality of texts. While the project showcased promising results, especially as it employed a serverless approach for online use, it also highlighted the need for continuous advancement in detection methods as AI-generated outputs become increasingly sophisticated. Overall, this initiative reinforces the ongoing battle between AI content creation and detection, sparking ideas for further applications in diverse domains.