Remote Code Execution on a $1B Legal AI Tool (www.promptarmor.com)

vLex, the developer of the Vincent AI legal research tool, recently addressed critical vulnerabilities that exposed users to phishing and remote code execution (RCE) threats. These issues stemmed from indirect prompt injection attacks, where malicious code was embedded in documents uploaded by users. When Vincent AI processed these documents, it inadvertently output HTML that generated a convincing fake login pop-up, enabling attackers to capture sensitive user credentials. Notably, the tool was acquired for $1 billion just last month by Clio, underscoring its significance in the legal tech landscape. The implications of these vulnerabilities are profound for the AI/ML community, particularly regarding the security of AI-driven applications in sensitive domains. The incident highlights the need for stringent security protocols, including robust input validation to prevent such prompt injection attacks. During testing, it was uncovered that not only could the model's outputs execute malicious JavaScript, but they could persist in stored chats, increasing the attack surface. vLex has since implemented rapid updates based on responsible disclosure, reinforcing the importance of security in rapidly evolving AI environments while providing critical insights into the risks facing legal AI tools.