Antislop: Identifying and Eliminating Repetitive Patterns in LLMs (iclr.cc)

Researchers have introduced Antislop, a novel framework designed to identify and eliminate repetitive lexical patterns, known as "slop," in outputs generated by large language models (LLMs). These repetitive patterns negatively impact the readability and quality of AI-generated text, making it easily distinguishable from human writing. Antislop incorporates three key innovations: the Antislop Sampler, which efficiently suppresses undesirable patterns during inference; an automated pipeline that evaluates specific model slop against human writing to create targeted training data; and Final Token Preference Optimization (FTPO), a unique fine-tuning technique that adjusts token logits directly to minimize slop without sacrificing overall performance. The significance of Antislop lies in its ability to enhance the quality of LLM outputs while effectively managing the suppression of repetitive patterns. Demonstrating its efficacy, the Antislop Sampler can suppress over 8,000 repetitive patterns while ensuring high-quality text generation, a feat where traditional token banning struggles at just 2,000 patterns. Furthermore, FTPO achieves a remarkable 90% reduction in slop while enhancing performance across a range of evaluation benchmarks, such as GSM8K and MMLU, thereby confirming its potential for broader application in the AI/ML community. By providing accessible code and datasets under an MIT license, the team encourages further research and development in improving the quality of AI-generated content.