OpenAI's New ChatGPT Watermark Breaks If You Edit One Word in Four (rews.cc)

OpenAI has announced the deployment of its new watermarking technology, textGrain, for ChatGPT and Codex within the European Union. This system is designed to mark AI-generated text so it is detectable, in compliance with the EU's AI Act, which mandates that generative outputs be labeled as machine-generated. The technical workings of textGrain involve using a secret key to bias the model's word choices through an optimized transport problem, while still allowing the overall output quality to remain comparable to unwatermarked text. However, the report highlights that the watermark's reliability significantly decreases if a user edits even a single word; detection drops from 92% to 66% with one word substitution and to 17% when one in four words is altered. This development is significant for the AI/ML community as it illustrates the tension between regulatory compliance and technical robustness in watermarking systems. OpenAI's approach is shaped by the urgency of regulatory deadlines and potential fines for non-compliance, which has prompted them to release this system, despite previous concerns about the circumvention potential and accuracy of earlier watermarking systems. As OpenAI aims to eventually open-source textGrain, the AI community will be watching closely to assess its effectiveness and integrity once exposed to independent evaluations. This case not only highlights the challenges in implementing detectable watermarking for text but also underscores the broader implications of regulatory frameworks shaping technological advancements in AI.