Mitigating Factual Hallucination in LRMs via Mixed-Mode Advantage Regularization (arxiv.org)

A recent study introduces MARGO (Mixed-Mode Advantage Regularization), a novel reinforcement learning framework aimed at enhancing factual accuracy in Large Reasoning Models (LRMs). These models, which generate explicit thinking traces prior to producing answers, can inadvertently contribute to what researchers call "thinking-induced hallucination," where the added cognitive process leads to incorrect conclusions instead of enhancing factuality. MARGO addresses this issue by integrating non-thinking rollouts into the model's decision-making process, allowing it to better gauge the factual benefits of explicit reasoning while minimizing misleading outputs. The significance of this development lies in MARGO's ability to improve the reliability of answers in factuality-oriented question answering tasks without sacrificing general reasoning capabilities. Experiments indicate that MARGO outperforms existing benchmarks in factual accuracy while maintaining strong performance in mathematical reasoning. This advancement presents a crucial step toward refining AI systems' reliability in real-world applications, offering a promising strategy to combat factual inaccuracies that hamper the deployment of language models in critical settings.