Reasoning with Sampling: Your Base Model Is Smarter Than You Think (arxiv.org)

Researchers show that many advanced reasoning behaviors attributed to reinforcement‑learned LLMs can instead be elicited at inference time from the base model by clever sampling. Drawing inspiration from Markov chain Monte Carlo and “sharpened” distributions, they introduce a simple iterative sampling algorithm that uses the base model’s own likelihoods to steer sampling toward higher‑quality solutions. Applied in single‑shot settings across tasks like MATH500 (math reasoning), HumanEval (coding), and GPQA (general QA), the sampler substantially boosts performance—often nearly matching or even outperforming RL‑posttraining—while preserving sample diversity instead of collapsing to a narrow set of outputs. The significance is twofold: technically, it demonstrates that stronger reasoning need not require expensive posttraining, curated datasets, or external verifiers; practically, it offers a low‑cost, broadly applicable tool to extract latent capabilities from existing models. By framing decoding as an iterative, likelihood‑guided sampling process, the work challenges the assumption that new behaviors must emerge only through RL fine‑tuning and suggests immediate improvements to reasoning pipelines in domains where training data, verifiers, or compute are limited.