We Don't Understand Neural Networks at the Algorithmic Level (kripken.github.io)

A new preprint by Olivia Guest, Nancy Abigail Nuñez Hernández, and Mark Blokpoel challenges the prevailing debate on the understanding of Large Language Models (LLMs) and Artificial Neural Networks (ANNs). While some prominent scientists argue that the complex mechanisms behind these models are beyond our comprehension, the authors assert that we do, in fact, understand key aspects of ANNs, including their structure and functional roles. However, they emphasize that despite this knowledge, the algorithmic details—the processes through which these systems operate—remain largely elusive. This gap in understanding the "algorithmic level" of ANNs is critical, as it relates to how these systems function and can be manipulated successfully. This discussion is significant for the AI/ML community as it touches on fundamental questions about the interpretability and predictability of machine learning models. Recognizing that while we can define what ANNs aim to do and how they are implemented, understanding the intermediate algorithms is essential for developing trustworthy AI systems. This insight highlights the ongoing need for further exploration into the workings of ANNs to enhance both their reliability and our grasp of their inner mechanisms, which may have profound implications for their application in high-stakes fields like healthcare and finance.