We Will Not Trust Autonomous AI Agents Anytime Soon (brodzinski.com)

OpenAI and Stripe’s Agentic Commerce Protocol (ACP) aims to let AI agents make purchases autonomously, but the announcement quickly resurfaced predictable security and merchant-exploitation concerns. Beyond headline risks—bad purchases, prompt injection, and adversarial trickery—the bigger issue is autonomy itself. The author frames autonomy as dependent on five prerequisites: transparency (data and context for decision-making), technical excellence, alignment (encoding tacit norms and values), explicit boundaries (guardrails), and care (responsibility). While technical competence and transparency can be improved with prompt/context engineering and engineering effort, alignment and care are fundamentally harder: current models lack a robust world model and tacit human norms, and they do not possess intrinsic notions of responsibility or accountability. Practically, that means fully autonomous, open-ended commerce agents are untrustworthy for now; guardrails will trigger an ongoing cat-and-mouse game with bad actors, and alignment failures will persist. Narrow, externally constrained uses—e.g., limited-purpose agents with capped credit cards or tightly scoped tasks—are realistic and useful, but “revolutionary” autonomous shoppers remain a risky promise. Trust, the author argues, is the true metric: until systems demonstrably satisfy transparency, alignment, boundaries, and care, organizations won’t afford them significant autonomy, and we should expect amusing and costly failures along the way.