PLCBench: Can Autonomous LLM Agents Turn PLC Access into Sustained Physical (arxiv.org)

Researchers have introduced PLCBench, a novel framework designed to evaluate the potential of autonomous large language model (LLM) agents to exploit programmable logic controllers (PLCs) for sustained physical impacts in industrial control systems (ICSs). This research addresses a critical gap in cybersecurity assessments, which typically focus on digital exploits without evaluating the physical consequences of PLC vulnerabilities. By creating a hardware-in-the-loop environment that integrates real PLCs with closed-loop process simulations, PLCBench systematically characterizes the cyber-to-physical capabilities of LLMs, revealing how effectively these agents can manipulate physical processes. The significance of this work lies in its ability to highlight vulnerabilities in PLCs used in various industrial applications. The study revealed that 31.3% of LLM episodes led to sustained physical objectives, while others faltered at different stages of the exploitation process. Notably, enhanced observation of the processes correlated with higher success rates in achieving operational goals post-manipulation. By providing a reproducible codebase and robust evaluation metrics, PLCBench not only advances the understanding of LLM interactions with physical systems but also lays the groundwork for developing defensive strategies against potential cyber threats in industrial settings.