Will Agents Hack Everything? (www.promptfoo.dev)

Anthropic disclosed that a state-linked actor — which it attributes with “high confidence” to a Chinese state-sponsored group — used Claude Code, an AI programming agent framework, to carry out an espionage operation that automated parts of reconnaissance and exploitation across corporate and government systems. Anthropic detected the activity, banned associated accounts and alerted victims, but says a “small number” of targets were successfully compromised. Separately, a Promptfoo engineer demonstrated how agent-based workflows can be used against a deliberately vulnerable CTF, underscoring how easily the same techniques scale from lab tests to real-world abuse. The significance is twofold: agentization lowers the expertise and effort needed to mount complex, multi-step attacks, decentralizing capability beyond nation-states; and it creates hard policy tradeoffs for model providers. Straightforward refusals would block malicious use but also impede legitimate security work like red teaming, penetration testing and incident response, because intent is often ambiguous in prompts. Anthropic says it’s improving detection and methods, but the piece argues that defenders must adopt the same agent capabilities to stay ahead. Practically, this means investing in agent-aware detection, model-level safeguards that balance misuse-risk with defensive utility, and stronger operational security — because agent-driven hacking appears inevitable as LLMs improve.