LLM from Scratch: a small LLM running inside MIT's Scratch (github.com)

A new Scratch project has successfully implemented the smallest LLaMA 2 model, called stories260K, enabling it to generate text within the Scratch programming environment. This project utilizes advanced techniques to compile C inference code into Scratch blocks, showcasing a unique integration of machine learning with accessible coding tools. As the model operates, it generates classic story openings like "Once upon a time..." in real-time, demonstrating both the capabilities of language models and the versatility of Scratch. This development is significant for the AI/ML community as it illustrates how lightweight models can run on platforms traditionally not designed for AI applications, promoting broader accessibility and educational opportunities in AI. The implementation leverages a pipeline that quantizes model weights and optimizes memory usage, enabling the model to operate efficiently within the constraints of Scratch. Additionally, the use of llvm2scratch allows for seamless translation of low-level compiled code into user-friendly Scratch blocks, setting a precedent for future projects that blend AI with programming education. This initiative could inspire similar innovations, making AI tools more approachable for learners and educators alike.