LLM from pre-1930 derives quantum mechanics and relativity (michaelhla.com)

An innovative experiment has demonstrated that a language model (LLM) trained on texts predating 1900 can exhibit surprising intuition regarding fundamental physics concepts, including elements of quantum mechanics and relativity. While the model struggles with complex physics-related tasks, it managed to declare insights such as "light is made up of definite quantities of energy" and recognized the equivalence of gravity and acceleration. This experiment is significant for the AI/ML community as it explores the potential of LLMs to perform out-of-distribution reasoning, a key challenge in achieving true artificial general intelligence (AGI). The project's methodology involved carefully curating a dataset of ~22 billion tokens from various historical texts and applying rigorous filtering to eliminate modern influences. The selected model, built on principles of optimal scaling, was trained with the aim of generating coherent explanations around experimental observations made before 1900. Ultimately, this endeavor not only highlights the innovative use of AI to probe historical scientific concepts but also opens new pathways for research into the limits and capabilities of machine learning models. If successful, it could provide valuable insights into the nature of intelligence and its manifestation in both humans and machines.