Pen and Paper LM 160: a language model small enough to run by hand (maciej.bearblog.dev)

The Pen & Paper LM 160 is an innovative microscopic language model designed for manual operation, featuring a compact architecture that allows users to run it entirely by hand. Unlike typical language models, it doesn't generate conventional prose but instead predicts the next token in a small workflow language with predefined tokens such as QUESTION, TASK, and DONE. The model's simplistic workflow enables users to create straightforward task loops, illustrating various processes that demonstrate its utility in structured problem-solving. This development is significant for the AI/ML community as it emphasizes transparency and interpretability in machine learning models. With only 160 parameters, the Pen & Paper LM 160 allows users to explore and manipulate its inner workings easily, making it an excellent tool for education and experimentation. Each component—from weights to biases—can be examined and adjusted, offering insights into model behavior. By bringing the complexity of a language model to a tangible and accessible format, this initiative champions a hands-on approach to understanding AI, potentially sparking new interest in computational thinking and model design.