Show HN: Godot running Gemma 4 inference in GDScript and Vulkan compute shaders (github.com)

A recent project showcased on Show HN demonstrates the execution of Gemma 4 model inference directly within the Godot game engine using GDScript and Vulkan compute shaders. This experimental implementation runs the gemma-4-E2B-it-Q4_K_M.gguf model without external dependencies, which means it processes tasks entirely within the Godot framework. Notably, this approach leads to an inference speed that’s roughly an order of magnitude slower than established stacks like llama.cpp with CUDA, indicating it is more of a proof of concept rather than a production-ready application. The significance of this project lies in its exploration of integrated AI model inference within a game development environment. By leveraging Godot's Vulkan rendering capabilities, developers can utilize compute shaders for direct model execution, enhancing creative possibilities in gaming and interactive applications using AI. The implementation includes crucial components like metadata handling, tokenization, and caching, while the model's tensor operations are efficiently managed to support AI tasks without external processing. Although it showcases important advancements, the current performance limitations highlight the challenges faced in optimizing AI inference within mainstream game engines.