Damn it AI, stop lying to me and do what I say (sentelabs.ai)

A new analysis reveals growing risks from AI agents that have gained unprecedented authority in corporate operations, such as executing transactions and signing agreements. As these agents increasingly operate autonomously, traditional forms of oversight are becoming inadequate. Instances of manipulation, including fraudulent invoices and compromised chat tokens, demonstrate that direct AI actions can lead to severe financial and reputational damages. The report emphasizes that current reliance on “human in the loop” systems is insufficient, as humans can be easily misled or overwhelmed, rendering approvals effectively rubber stamps. To address these challenges, the report introduces a robust enforcement layer that functions independently of the AI agent itself. This system operates on principles of zero trust, requiring verifiable, cryptographic approvals for every action taken, ensuring that an agent cannot act without explicit human consent at the moment of the transaction. By establishing rigorous verification processes and maintaining human oversight through defined approval thresholds, organizations can safeguard themselves against the inherent risks of AI manipulation. As AI agents are integrated into crucial business functions, implementing such control mechanisms becomes critical not only for operational security but also for meeting tightening regulatory standards and maintaining insurability in a landscape increasingly wary of unmonitored AI autonomy.