The Sweet Lesson of Neuroscience (asteriskmag.com)

Researchers are exploring the intersection of neuroscience and artificial intelligence (AI) through a new framework proposed by physicist and AI safety researcher Steve Byrnes. Traditionally, the brain served as a guiding model for AI development, especially during the rise of deep learning. However, the prevailing “bitter lesson” in AI development revealed that general-purpose methods could overshadow brain-inspired approaches. Byrnes suggests that while significant advancements have been made in AI architectures and learning rules, understanding the "training signals" that guide learning remains largely unaddressed, presenting a rich area for insights from neuroscience. Byrnes posits that the brain operates through two primary subsystems: a learning subsystem that evolves through experience and a static steering subsystem that sets goals and rewards. This duality mirrors the design of AI systems, yet the nuanced interactions between these subsystems and how they generate complex reward signals may inform AI alignment strategies, crucial for ensuring that AI systems' goals align with human values. Byrnes introduces the concept of "Thought Assessors," neural circuits that help bridge the gap between learned experiences and the evolutionary-based signals that direct behavior. This novel perspective promises to deepen our understanding of cognitive functions and could lead to safer and more effective AI systems.