Running modern LLMs in the browser using Three.js and WebGPU (ben3d.ca)

Three-LLM has made a significant advancement by enabling the execution of various language models, including GPT-2 and SmolLM2, directly in web browsers using Three.js and WebGPU. This innovative approach transforms the models' inference graphs into TSL compute shaders, allowing these sophisticated AI algorithms to run on users' GPUs without the need for server-side resources. The demo showcases the capability to load diverse Hugging Face model configurations directly in the browser, ranging from lightweight models like TinyStories to more complex architectures such as Qwen and Phi, with model weights being expanded to 32-bit floats during runtime. This development is noteworthy for the AI/ML community, as it pushes the boundaries of client-side computation, demonstrating how modern language models can operate efficiently in an accessible environment. By employing Three.js's rendering capabilities for general computation, Three-LLM effectively manages the intricate operations required for language model inference—such as matrix-vector multiplications and attention mechanisms—while eliminating the need for precompiled binaries. The integration of sophisticated techniques like grouped-query attention and rotary position embeddings further showcases the potential of executing advanced AI models on consumer hardware, thereby broadening access to powerful AI tools and fostering further innovation in the field.