A Minimalist Transformer Architecture for Univariate Time Series Forecasting (www.mdpi.com)

Researchers present a minimalist Transformer architecture tailored for univariate time‑series forecasting and demonstrate it on a small “restaurant” dataset. The model keeps the canonical encoder–decoder pattern but strips components to essentials: each scalar timestep is projected into an m‑dimensional embedding via a shared projection vector and bias, an absolute learnable positional matrix is added, and a stack of Encoding Blocks applies multihead scaled‑dot‑product attention, residual connections + LayerNorm, and a small two‑layer ReLU feedforward network. The decoder is autoregressive: it initializes with a learned <CLS> token, uses masked self‑attention to prevent peeking at the future, and employs multihead cross‑attention over encoder outputs to generate forecasts step‑by‑step. Significance for the AI/ML community lies in showing how Transformer building blocks can be pared down for low‑dimensional time series while preserving expressivity and interpretability. Key technical choices include non‑shared layer parameters across iterations, absolute learnable positional encodings, standard Q/K/V projections per head, concatenation + output projection after k heads, and a small FFN with intermediate dimension p>m. The paper provides a tiny worked example (n=7, m=4, k=2, dk=dv=2, p=16) with detailed parameter counts and appendices/code to enable exact replication, highlighting that lightweight, attention‑based models can be practical and transparent for short-series forecasting.