Death by a Thousand Prompts: Open Model Vulnerability Analysis (arxiv.org)

Researchers performed an automated adversarial audit of eight open-weight large language models to measure their resistance to prompt-injection and jailbreak attacks in both single-turn and multi-turn settings. Using an automated testing pipeline, they found pervasive vulnerabilities: multi-turn attacks succeeded between 25.86% and 92.78% of the time, a 2×–10× increase in success over single-turn baselines. Models optimized for capability (examples cited: Llama 3.3, Qwen 3) showed higher multi-turn susceptibility, while safety-focused designs (e.g., Google Gemma 3) delivered more balanced—but not foolproof—performance. The analysis highlights that extended conversations dramatically weaken current guardrails. This matters because open-weight models are widely used as foundations for fine-tuning and deployment; multi-turn failure modes can cascade into real-world operational and ethical harms if left unmitigated. Key technical implications: safety evaluations must include multi-turn, stateful adversarial testing; alignment choices and development priorities materially affect robustness; and layered defenses (rate-limiting, system-level checks, runtime monitoring, professional AI security tools) are necessary complements to model-level alignment. The authors recommend a security-first design philosophy and deployment architectures that assume persistent adversarial probing to reduce exposure in enterprise and public applications.