Transformers for Software Engineers (blog.nelhage.com)

Anthropic interpretability engineer presents “Transformers for Software Engineers,” a hands-on, software-centric walkthrough of decoder-only (GPT-style) Transformers, illustrated with a mostly complete Rust implementation and concrete GPT-3–scale hyperparameters (96 layers, D_model=12,288, D_head=128, vocab≈50k). The post reframes the model as a functional program running on an unusual machine: a residual stream of per-token state vectors that are never overwritten but incrementally updated. It’s aimed at SWE readers moving into ML and interpretability, and the code is available on GitHub to follow along. Technically, the write-up emphasizes core interpretability-friendly abstractions: State/Query/Update types, embeddings as state lookups (mathematically a matrix multiply), and unembedding as an array of per-token logit functions (again equivalent to a matrix). Layers are residual blocks composed of attention then MLP, and the model operates autoregressively to produce logits over the vocabulary per position. The author highlights useful intuitions—query/update as the primary I/O on an opaque high-dimensional state, the power of exponentially many near-orthogonal directions to encode features, and thinking in software-engineering terms to reverse-engineer the algorithms implemented across weights. This perspective and runnable code make circuits-style interpretability more accessible to engineers who want to trace and name the computations inside large Transformers.