A startup claims it broke through a bottleneck that's holding back LLMs (www.technologyreview.com)

Miami-based startup Subquadratic emerged from stealth mode, claiming to have resolved a longstanding mathematical bottleneck affecting large language models (LLMs). Their new model, SubQ, reportedly processes data up to 12 times faster than typical models, while being both cheaper and more energy-efficient. Independent evaluations suggest that SubQ not only maintains competitive performance with leading models from Google DeepMind and OpenAI on tasks like coding but also excels in analyzing large data sets with a remarkable context window of 12 million tokens. The significance of Subquadratic's breakthrough lies in its innovation of sparse attention, which dramatically reduces the computational load compared to traditional dense attention used in transformers. By dynamically prioritizing which token relationships to analyze, SubQ aims to revolutionize LLM efficiency, offering the potential for more practical applications in data-intensive domains. However, skepticism remains due to the limited availability of SubQ for public testing and concerns about its foundational architecture’s dependency on existing models. As the AI community watches closely, the promise of SubQ could herald a new era of LLM development focused on efficiency and accessibility.