Limitations on Safe, Trusted, Artificial General Intelligence (arxiv.org)

A recent study has presented rigorous mathematical definitions for safety, trust, and Artificial General Intelligence (AGI), revealing a fundamental incompatibility between these concepts. The authors argue that while safety ensures an AI system never makes false claims and trust relies on this safety, AGI necessitates capabilities that can surpass human performance. Their findings suggest that a system that is both safe and trusted cannot function as AGI, as there will always be tasks that are easily solvable by humans but beyond the reach of such a system. This research carries significant implications for the AI/ML community, particularly in the ongoing discourse about the development of safe and trustworthy AI systems. By drawing parallels to Gödel’s incompleteness theorems and Turing’s undecidability of the halting problem, the study underscores the inherent limitations of achieving AGI in a manner that aligns with strict safety and trust. The conclusions invite researchers to reconsider how they define and pursue the goals of AI, suggesting that practical interpretations of safety and trust may need to deviate from the strict mathematical framework proposed to enable advancements in AGI.