International Agreement to Prevent Premature Creation of ASI (techgov.intelligence.org)

The MIRI Technical Governance Team has unveiled a detailed proposal for an international agreement aimed at halting the premature advancement toward artificial superintelligence (ASI), a move many experts deem critical due to the catastrophic risks associated with misaligned AI. The agreement primarily focuses on limiting the scale of AI training and restricting specific AI research to mitigate the potential dangers of ASI, which include misuse by malicious actors and existential threats to humanity. By suggesting a collaborative effort led by the United States and China, the framework aims to ensure rigorous monitoring of AI capabilities and restrict the development of technologies that could lead to unregulated superintelligence. Under this proposed agreement, training runs for new AI systems would be capped at certain computational thresholds, and the number of AI chips permitted in unmonitored facilities would be strictly limited. The coalition members would consolidate their AI chip resources into monitored data centers, enhancing accountability and reducing risks. Acknowledging the urgency of the issue, the team argues that proactive measures must be taken now, before potential feedback loops result in irreversible misalignments or lead to a point of no return in AI advancements. By fostering international cooperation and utilizing established verification mechanisms, this agreement seeks to balance ongoing AI development with the imperative of safeguarding humanity’s future.