Strengthening our defenses against distillation attacks (www.anthropic.com)

In a strategic move to combat distillation attacks on its AI model, Claude, Anthropic has implemented new defenses that have reportedly reduced data extraction by 90 to 99 percent. Distillation attacks, which covertly replicate the capabilities of AI models, pose significant risks, allowing unauthorized parties to create powerful but inadequately safeguarded models that could be misused in sensitive areas like cybersecurity and bioengineering. Notably, this method has been exploited by companies abroad, such as Zhipu, to emulate US-model capabilities, raising national security concerns. Anthropic's enhanced defenses focus on controlling the flow of reasoning data, blocking extraction attempts, and making repeated unauthorized access more difficult. By introducing measures like "preserved thinking," which restricts how new accounts can interact with the model's reasoning, and a two-stage classification system to identify and thwart distillation efforts, the company has effectively neutralized many previous methods used by distillers. Although these techniques require adjustments for legitimate users and have trade-offs, Anthropic believes that the imperative to protect AI models from misuse justifies the efforts and costs involved.