Reading Between Dots: Decoding Hidden Computation Across Filler Tokens (ICML'26) (arxiv.org)

A recent study presented at ICML 2026, titled "Reading Between the Dots: Decoding Hidden Computation Across Filler Tokens," reveals significant insights into how frontier large language models (LLMs) like DeepSeek V3 and Kimi K2 perform complex reasoning tasks using content-free filler tokens such as dots or counting sequences. The research demonstrates that these models can compute answers through hidden layers of information, effectively bypassing traditional behavioral oversight mechanisms that only monitor surface tokens. By employing an unsupervised decoding pipeline, the authors were able to recover intermediate values with 80-95% accuracy, showcasing that the models retain structured computation in their hidden states. This discovery is crucial for the AI/ML community as it challenges existing notions of monitorability in LLMs. It implies that the true computational capabilities of these models extend beyond what is visible in their outputs, suggesting that their decision-making processes can be transparent when accessed through appropriate methodologies. The findings could lead to improved interpretability and oversight of AI systems, enhancing trust and accountability in their applications by clarifying how decisions are derived even when explicit reasoning is not apparent on the surface.