Supplementary Information for the DeepSeek R1 paper [pdf] (static-content.springer.com)

DeepSeek’s supplementary material for the Nature paper details DeepSeek-R1, an RL-driven post-training pipeline that explicitly incentivizes chain-of-thought (CoT) reasoning in large language models. Built on DeepSeek‑V3‑Base (a MoE model pre-trained on ~14.8T tokens with MLA and Multi‑Token Prediction optimizations), the team trains R1 with a bespoke reinforcement learning framework—introducing GRPO (a reward-guided variant compared with PPO)—plus a reward model prompt, an 800K-example supervised fine-tuning (SFT) corpus, RL task datasets, cold-start recipes, and distillation to smaller checkpoints. The supplement covers hyperparameters, training cost, mitigation of reward‑gaming, a language‑consistency reward to avoid language mixing, and analyses of CoT length and self‑evolution of reasoning behaviors during training. Technically significant findings include GRPO outperforming PPO on math benchmarks (MATH), emergent advanced reasoning behaviours as RL progresses, and practical distillation strategies that retain reasoning gains in smaller models. The paper also provides an extensive safety report—risk-control architecture, benchmarked robustness to jailbreaks, and multilingual safety evaluations—plus ablation studies (reward hacking, language consistency) and open release plans for weights, code, and data. For the AI/ML community, DeepSeek‑R1 demonstrates a reproducible, RL-centered route to improve complex reasoning capabilities while documenting failure modes and safety controls important for deployment.