The Brain vs. Deep Learning Part I: Computational Complexity (timdettmers.com)

In a recent insightful blog post, the author explores the intricacies of the brain's information processing capabilities by comparing them to the architecture of deep learning systems, particularly convolutional networks. This examination not only highlights striking similarities between biological neurons and artificial architectures but also delves into the computational complexity inherent in these processes. The author argues that advancements in neuroscience over the past decade have rendered outdated predictions about artificial super-intelligence—previously estimated to emerge as early as 2030—less plausible, particularly due to significant discoveries about the brain's dynamic information processing abilities. The discussion emphasizes that recent breakthroughs challenge the foundational estimates used by futurists like Ray Kurzweil, who based his computations on a limited understanding of brain complexity from 2005. For instance, new insights reveal that brain connections actively process information and that even non-firing neurons contribute to learning in meaningful ways. By integrating these findings with comparisons to deep learning, the author presents a framework that underscores the limitations of brain simulations in forecasting AI progress, ultimately urging skepticism towards predictions that rely on these models. This exploration is pivotal for AI/ML practitioners as it shapes the discourse on the potential and timelines of achieving human-like intelligence in artificial systems.