5M Param LM Running in Minecraft (github.com)

A 5-million-parameter language model called CraftGPT has been made to run entirely inside Minecraft using a custom redstone processor (MCHPRS) and was trained on the TinyChat dataset. The project is primarily a proof-of-concept art/engineering demo: the model has a tiny 64-token context window, is highly error-prone and often produces ungrammatical or irrelevant output, and token generation is extremely slow. Running it in unmodified vanilla redstone would take years per response; MCHPRS (a highly optimized redstone “CPU”) reduces this to hours but still requires a beefy machine (32–64+ GB RAM) and a nontrivial setup (compile the Rust crate, set plot scale to 7, load the provided world into ./target/release as world, run on Minecraft 1.20.4, use /rp c -io, /rtps unlimited and /wsr 1). Why it matters: the project demonstrates that neural models can be encoded and executed on unconventional computational substrates, highlighting constraints-driven engineering, education, and creativity rather than practical NLP utility. Technically notable details include the use of binary RNG seeds, a visual progress bar and binary token counter implemented in-game, and optimization flags that suppress non-IO block updates to speed compilation/execution. For AI/ML researchers, it’s an intriguing exploration of extreme model compression, token-efficiency limitations, and how hardware/software co-design shapes feasible model deployments — useful for teaching, benchmarking constrained inference, and inspiring unconventional accelerator designs.