Show HN: Jam Storyteller – Attention? Memory Is All You Need (github.com)

A developer has unveiled "Jam Storyteller," a novel language model featuring 100 million parameters. Unlike traditional models, its mixing matrix remains static, with values restricted to {-1, 0, 1}, resulting in zero multiplications during the mixing stage. This innovative approach simplifies computation, potentially enhancing performance and efficiency in natural language processing tasks. The significance of Jam Storyteller for the AI/ML community lies in its emphasis on memory over attention mechanisms, challenging the conventional norms of model architecture. By providing two downloadable versions of the model—an INT8 variant (tinyworld.jam) and an f32 version (tinyworld.jam32) for PyTorch—developers can easily experiment with and implement it. This initiative not only opens doors to new avenues in language modeling but also raises questions about the future role of attention mechanisms in AI, highlighting the potential for simplified models to achieve competitive performance without the computational overhead typically associated with more complex architectures.