DeepSeek tests “sparse attention” to slash AI processing costs (arstechnica.com)

Chinese AI startup DeepSeek released an experimental build of its simulated-reasoning model, DeepSeek‑V3.2‑Exp, which introduces “DeepSeek Sparse Attention” (DSA). DSA is DeepSeek’s implementation of sparse attention — a class of techniques that reduce the quadratic compute and memory cost of transformer attention by attending only to a subset of tokens. DeepSeek claims DSA provides “fine‑grained sparse attention for the first time” and used the savings to cut API prices by 50% as a proof point. The company is positioning the work against limited access to high-end AI chips (due to export restrictions) and follows a lineage of prior work (OpenAI’s sparse transformers around GPT‑3 and Google’s Reformer). For the AI/ML community this matters because cheaper, more efficient attention directly eases scaling to long contexts and lowers running costs, potentially improving latency and throughput for chat apps and long-document tasks. Key implications include faster long-conversation performance and reduced infrastructure spend — especially for labs that can’t scale hardware. Open questions remain about accuracy/robustness trade-offs, how “fine‑grained” DSA differs technically from existing sparse schemes, and whether it generalizes across workloads and hardware. If validated, DSA could intensify competition around cost‑efficient model architectures and accelerate broader access to long‑context models.