We benchmarked our prompt-injection detector against OWASP's LLM Top (www.cerbereag.site)

A team has released their first public benchmark for AgentGuard's prompt-injection detection system, measuring its performance against the OWASP Top 10 for LLM Applications. The results reveal that a deterministic regex approach successfully identifies 91.5% of prompt injection attacks with zero false positives, demonstrating remarkable speed at under a millisecond. When an added machine learning (ML) layer enhances recall to 98.1%, it introduces a latency increase to approximately 450 milliseconds and a notable false positive rate of 33.3% for certain benign prompts that resemble malicious queries. This advancement is significant for the AI/ML community as it addresses the growing concern of prompt injection attacks in language model applications, which could compromise security and integrity. The findings highlight the trade-offs between detection accuracy and processing speed, particularly in sensitive scenarios. The research team emphasizes the ongoing work needed to reduce false positives and improve the performance of the ML layer while revealing sophisticated techniques like obfuscation as a continuing challenge. The project is open for public collaboration, encouraging feedback to refine the detection capabilities further.