Before AI's Kepler Moment (ashvardanian.com)

A thoughtful essay argues that today’s large language models are the AI equivalent of Ptolemy’s epicycles: powerful, parameter-rich approximations that fit data but may lack a compact, explanatory foundation. Just as epicycles became an ever-more baroque stack of circles (mathematically equivalent to a Fourier series) to match planetary motion until Kepler and Newton revealed simpler laws, modern neural networks—feedforward layers h = σ(Wx + b) and transformer attention Attention(Q,K,V)=softmax(QK^T/√d_k)V stacked into models with 10^11–10^12 parameters—may be chasing predictive accuracy without capturing the underlying principles of intelligence. The significance for AI/ML is both cautionary and constructive: we might be in a “pre-Copernican” era where scaling and clever engineering produce startling utility but not conceptual understanding. The author bets that certain primitives—memory (retrieval, attention, episodic recall) and optimization (gradient descent, evolutionary search)—are likely to survive any paradigm shift, while other architectural details could be swept away by a more compact theory. The piece urges the community to pursue deeper principles (causality, simulation, self-play, compression) rather than only stacking more parameters, because science advances when predictive fit yields to simpler, general laws that explain why systems behave as they do.