Porting microgpt to Futhark, Part I (www.kmjn.org)

A new project has been initiated to port Andrej Karpathy's microgpt implementation, a lightweight version of a GPT-2-like neural network in Python, to Futhark, a functional programming language designed for high-performance computing. While microgpt is compact and accessible at around 200 lines of Python code, it struggles with scalability, hitting Python's recursion limits when attempting to expand the model. The port aims to maintain a close 1-to-1 translation of the original while leveraging Futhark's parallel processing capabilities, thus improving scalability and performance. This first part of the series focuses on translating key elements of the forward pass, such as the model's architecture, data structures, and several essential functions like linear transformation, softmax, and RMS normalization. While the translation introduces some complexity and increases the line count slightly, it also enhances performance potential by taking advantage of Futhark's features. The project promises to advance not only the usability of microgpt but also the understanding of functional programming in machine learning, setting the stage for further expansion, including training the model, in upcoming parts.