William: A tiny poetry model in the browser (akshit.org)

A new local poetry generation model named William has been developed to run directly in web browsers, significantly enhancing accessibility to AI-driven creative tools. Unlike traditional models that rely on server infrastructure, William operates entirely on the client side, producing poetry one token at a time without sending data to a remote server. This innovation allows for a streamlined and private user experience, particularly appealing for those interested in experimenting with AI in writing and creativity. William is built as a small decoder-only transformer featuring 6 layers, a 384 hidden dimension, and a 256-token context window. Its two-stage training process harnessed targeted datasets, initially learning from a filtered corpus of poetry and subsequently fine-tuning with specific poem title/body pairs. The model is efficiently packaged using ONNX and is dynamically quantized, resulting in a lightweight 14 MB file that can be executed through ONNX Runtime Web. This development not only showcases the potential for localized AI applications but also emphasizes the growing trend of compact models that perform complex tasks directly in users’ browsers, broadening access to AI tools for creative expression.