Model backdoors are easy to create and hard to reliably remove (equistamp.substack.com)

Researchers have revealed significant vulnerabilities in the retraining processes of AI models, demonstrating that model backdoors—hidden behaviors activated by specific triggers—are alarmingly easy to implant yet difficult to eradicate. In experiments with open-weight coding models, the team discovered that these backdoors could survive retraining on clean data, effectively defeating the assumption that retraining guarantees a "clean slate". Tools like PersistBD were also shown to enhance the durability of backdoors, making them more resistant to removal even after fine-tuning, thereby posing a substantial risk to organizations utilizing these models. The implications for the AI/ML community are profound, as the existence of such backdoors highlights a critical trust gap in the model supply chain. When companies retrain models, they may unknowingly retain harmful backdoors, which can be exploited by malicious actors or even inadvertently by the models themselves. With the increasing adoption of open models and autonomous agents in training, the risk of sophisticated and insidious backdoors expands. This research underscores the urgent need for improved verification methods, transparency in training histories, and architectural innovations to mitigate the risks of model backdoors, ensuring that trust in AI systems is both warranted and substantiated.