LLMs Are Turing-Complete (yager.io)

In a recent development, it has been established that transformer models, which underpin large language models (LLMs), are Turing-complete. This means they can execute any computable algorithm, fundamentally challenging the skepticism surrounding their capabilities in tasks like coding and complex computations. The core innovation involves the combination of multilayer perceptrons (MLPs) for arbitrary calculations and attention mechanisms that provide writable memory, essential components for any functional computer. To illustrate this, a compiler was created using an assembly language and a Llama transformer model, demonstrating how LLMs can produce direct outputs without the need for prior training. This model utilizes its context as a writable memory store, enabling it to implement a Turing-complete evaluation and suggesting that LLMs can learn to execute any computable function. Consequently, this finding implies that the AI/ML community need not transition away from transformers, as they are not just efficient but also fundamentally capable of completing any computational task. This redefines existing boundaries and emphasizes the transformative potential of LLMs in various applications.