Microsoft Copilot reveals secret input that allowed it to be hacked (arstechnica.com)

Researchers from Varonis successfully exploited a vulnerability in Microsoft 365 Copilot, revealing how attackers could access sensitive user data like passwords without any user confirmation. Uniquely, instead of traditional hacking methods, the team engaged in a questioning dialogue with Copilot itself, which inadvertently divulged information about its internal safety mechanisms. Through strategic questioning, they uncovered an undocumented parameter, ?autorun=1, which, when paired with a common query parameter, triggered commands automatically upon users clicking a malicious link. This incident is significant for the AI/ML community as it highlights critical vulnerabilities in AI systems that can be detrimental if misused. The exploit demonstrates the potential risks posed by AI models that can inadvertently share sensitive internal details, leading to exploitation. In response to the incident, Microsoft implemented immediate mitigation measures to restrict the dangerous parameters, emphasizing the importance of robust security in AI applications and the need for ongoing vigilance in the development and deployment of AI technologies.