Large Language Models Inference Engines Based on Spiking Neural Networks (arxiv.org)

Researchers introduced NeurTransformer, a methodology for turning transformer-based language models into spiking neural network (SNN) inference engines to cut the computational and energy costs of deploying large models. The paper targets the two main bottlenecks of SNN transformers — slow, inefficient surrogate training and conversion approaches that require many spike time-steps (high latency) — by (1) replacing conventional self-attention with a spike-based self-attention (SSA) block, (2) converting transformer feed-forward blocks to SNN equivalents, and (3) fine-tuning the SSA via supervised surrogate-learning methods. The approach is positioned as an inference-focused, scalable route to bring transformer workloads onto more energy-efficient neuromorphic-style hardware. On benchmarks with three GPT-2 variants, the converted models show a tradeoff between fidelity and efficiency: converted GPT-2 small models incurred a 5–12% loss in cosine-similarity of representations while exhibiting a 9.7% reduction in perplexity (a surprising mixed signal on quality). Crucially, the SSA attention block yields major energy savings versus the baseline attention implementation, with estimated reductions of 64.7–85.3% on digital hardware. The work highlights a pragmatic pathway for lowering inference energy and potentially latency for LLMs using SNN primitives, while underscoring remaining accuracy and conversion-time-step tradeoffs that the community must address.