The 9s of AI Reliability (www.obviouslywrong.org)

The AI industry is facing a critical challenge: while models often achieve high mean accuracy, their reliability can be significantly lower, leading to potential failures in real-world applications. For example, many AI systems presently operate at 30-70% reliability, which can cripple businesses—such as Zillow's 2021 downfall due to miscalculating home prices based on unreliable AI predictions. This discrepancy underscores the importance of not just measuring average performance, but also focusing on tail reliability, which is essential for building trust in AI systems. Enhancing AI reliability is vital for unlocking vast economic potential; moving from 70% to 90% reliability could shift values from billions to trillions in economic output. The disparity between what AI can do in theory versus actual deployment calls for a reevaluation of AI training and evaluation methodologies. As the article suggests, optimizing for reliability—through system design, inference adaptability, and comprehensive research agendas—will be key to transforming AI capabilities and establishing a reliable foundation for future applications in various sectors.