Make Trust Irrelevant: A Gamer's Take on Agentic AI Safety (github.com)

DesoPK has raised significant concerns about the current approach to Agentic AI safety in their GitHub repository titled “Make Trust Irrelevant: A Gamer's Take on Agentic AI Safety.” They argue that the industry is misguidedly focused on making agents trustworthy, rather than eliminating the need for trust altogether. The core issue lies in the way authority is granted to AI agents. Instead of relying on imperfect safety measures, DesoPK advocates for strict, kernel-enforced limitations that prevent agents—regardless of their alignment—from taking unauthorized actions. The absence of these hard permission boundaries often results in agents being exploited, leading to systemic failures that manifest in real-world consequences such as accidental data deletion or unwanted automation. The implications of this perspective for the AI/ML community are profound. DesoPK’s proposal for a kernel-resident authority broker, referred to as KERNHELM, emphasizes the need for a clear distinction between planning (the agent’s role) and authorization (the control plane's role). By treating authority as a consumable resource with scoped, short-lived permissions, AI systems can operate securely without relying on the potentially flawed intent of their users. This shift in focus from trust to mechanical enforcement could transform how safety is approached in AI, mitigating risks associated with ambient authority and reinforcing accountability in agent behavior. As the field advances, these insights challenge practitioners to rethink existing frameworks and prioritize robust mechanical controls over mere trustworthiness.