Local Transformer Language Model Running on GameBoy Color (github.com)

A novel proof-of-concept demonstration has successfully run a quantized transformer language model, TinyStories-260K, locally on a Game Boy Color, showcasing an innovative application of AI technology in retro gaming. This project leverages the GBDK-2020 development environment, enabling the handheld console to accept prompts through its buttons and an on-screen keyboard, tokenize input, and perform autoregressive text generation. While the current implementation is slow and the output quality is basic, it marks a significant milestone in embedding AI capabilities into constrained hardware, expanding the boundaries of machine learning in unexpected formats. This effort is noteworthy for the AI/ML community as it highlights the possibilities of running functional models on limited resource devices without the need for modifications. The architecture employs fixed-point calculations and incorporates essential transformer components, including RoPE attention and RMSNorm, to achieve inference within the 8 KB RAM limit of the Game Boy Color. Although the output generation is rudimentary, the project opens avenues for further exploration of AI applications in low-power environments, suggesting the potential for future enhancements like improved decoding techniques or optimized processing capabilities.