Sampling at Negative Temperature (cavendishlabs.org)

Researchers ran Meta’s LLaMA locally with llama.cpp but replaced the usual nonnegative sampling rule to allow a negative temperature (e.g., --temp -0.001). Because the softmax at a model’s final layer is mathematically the Boltzmann distribution, a negative temperature flips the exponent sign: tokens that are normally least likely become most likely. Practically this required a tiny code change (treat temp==0 as greedy instead of temp<=0) and disabling repetition penalty, top-k, and top-p. With 7B and 13B models the outputs went from sensible text at small positive temps to deterministic “least-likely” tokens or bizarre repeating strings at negative temps; tiny negatives produced stuck or hanging generations, while larger-magnitude negatives produced long, oddly repetitive token streams. This experiment is significant because it exposes how sampling temperature directly maps to the Boltzmann view of logits and reveals structural quirks in model distributions: the tokens that dominate at negative temperatures tend to be those near the centroid of LLaMA’s embedding space (e.g., unusual tokens like Хронологија or entferne), which the model often “refuses” to generate in normal contexts. That both offers a probing tool for understanding token-level geometry and raises safety/robustness implications—negative-temperature sampling is a way to surface anomalous low-probability modes, adversarial completions, and deterministically unlikely behavior that standard APIs (e.g., OpenAI) intentionally prevent by clamping temperature.