Thinking in Higher Dimensions – Beautiful Visualizations by Jos Leys (2011) (www.youtube.com)

Stanford has launched CS336 Language Modeling from Scratch I for 2025, a hands-on course that guides students through building a language model end-to-end. Inspired by operating systems classes that build entire OSes from the ground up, this course covers crucial steps including data collection and preprocessing for pre-training, constructing transformer architectures, model training, evaluation, and deployment. By demystifying these processes, it aims to equip learners with a deep and practical understanding of language model development. This initiative is significant for the AI/ML community as language models are foundational to modern NLP applications, enabling versatile, general-purpose systems that tackle diverse downstream tasks. With the rapid evolution of AI, gaining expertise beyond using pre-built models—learning how to create them from scratch—empowers researchers and engineers to innovate more effectively and optimize models for specific challenges. The course offers a unique curriculum that blends theory with practical coding and experimentation, fostering a generation of practitioners capable of contributing to advancing language model design and deployment. Technical highlights include comprehensive coverage of transformer model architecture, dataset curation and cleaning strategies crucial for training large-scale models, as well as hands-on experience scaling training workflows and evaluating model performance. By unpacking these core components, the course provides critical insight into the inner workings that drive state-of-the-art NLP systems today. More details and enrollment options are available at the official course website.