Genre glitches and unexpected promotional phrases as a sign of AI writing (jilltxt.net)

Recent observations in AI-assisted writing have highlighted a phenomenon termed "genre glitches," where text generated by large language models (LLMs) abruptly shifts genres. This occurs when a phrase—often promotional and overly descriptive—intrudes on an otherwise informational context, suggesting that LLMs are activating irrelevant associative pathways in their training data. For instance, an article on tomato price inflation included an odd description of tomatoes that felt more suited for a food blog than a financial report. Similarly, a legal document mentioned a train trip in a way that clashed with its formal tone, veering into a travel advertisement. These genre glitches are particularly significant for the AI/ML community as they reveal the limitations and quirks of LLMs in understanding and adhering to genre conventions. Such examples underscore the challenges of deploying AI in professional writing contexts, where the text must maintain clarity and relevance. The ease with which AI can generate bizarrely inappropriate phrases points to the necessity for continued refinement of LLMs to improve their contextual awareness and ensure their outputs align with the intended purpose of the documents. This phenomenon emphasizes the importance of careful oversight when using LLMs for tasks that require precision and coherence, shedding light on a critical area for further research in AI writing systems.