Show HNs (github.com)

A new text classification architecture called GrafoPropagation has been announced, featuring about 990,000 parameters and leveraging geometric von Mises-Fisher (vMF) attention. This model distinguishes itself through unique components like vMF Dual-Scale Attention, where both local and global query/key projections enhance expressivity, and a Riemannian Temporal Embedding, which allows for better position-aware dynamics. The architecture also integrates a Global Workspace Memory for effective information management and includes innovative techniques such as quantum learning rate modulation. The significance of GrafoPropagation lies in its compact yet powerful design, making it accessible for various applications within the AI/ML community. Its WordNet pre-training, combined with fine-tuning on datasets like AG News, positions it as a versatile tool for multi-label classification tasks. Additionally, the model's ability to scale in complexity (up to ~30M parameters) and its focus on incorporating advanced attention mechanisms and memory processes underscore its potential for improving performance in complex NLP tasks while remaining resource-efficient. The built-in configuration flexibility further enables researchers and developers to tailor the architecture to their specific needs efficiently.