What if AI worked at 1.000.000 tokens per seconds? (www.echohive.ai)

A thought experiment explores the implications of a theoretical AI model capable of processing one million tokens per second—a concept that significantly challenges our understanding of current AI/ML capabilities. The discussion outlines four key aspects of processing speeds: context capacity, input processing, aggregate throughput, and single-agent output. For example, generating one million tokens in one second could enable the rapid creation of numerous outputs, such as a thousand story endings or multiple drafts for applications, underscoring both the potential speed of output generation and the inherent limitations in evaluation and testing. This hypothetical speed raises critical questions for the AI/ML community, particularly regarding the importance of human judgment in the AI workflow. While faster generation allows for more options, it does not equate to quality or useful insights. Essential elements such as taste, specifications, and physical evidence remain scarce, emphasizing that as generation becomes cheaper, discerning valuable results becomes increasingly crucial. Ultimately, this thought experiment illustrates that in the race for efficiency, careful consideration of what "done" means and the methodology behind selecting the best outputs are paramount for effective AI deployment.