Is Capability a Liability? More Capable Language Models Make Worse Forecasts (arxiv.org)

Recent research reveals a counterintuitive phenomenon known as "inverse scaling" in large language models (LLMs) when applied to forecasting tasks, particularly in areas with superlinear growth and regime change risks like finance and epidemiology. The study found that more capable models, despite their advanced capabilities, produce less accurate distributional forecasts, particularly at the upper tail. This pattern was observed across various datasets, including synthetic SIR epidemics and real-world data on COVID-19 and housing markets. The findings suggest that while enhanced model scale and post-training methods contribute to this phenomenon, they do not lead to improved performance in forecasting critical extremes, indicating a crucial gap in traditional evaluation methods. This research has significant implications for the AI/ML community, particularly those involved in forecasting applications. It challenges the existing paradigm that greater model capabilities automatically enhance accuracy, highlighting the need for a shift in how model performance is evaluated. Specifically, the study advocates for the incorporation of continuous and unbounded measures of accuracy, which can capture upper-tail performance more effectively, instead of relying solely on conventional binary threshold metrics. This could lead to improved forecasting methodologies that better account for rare but impactful events, ultimately refining the reliability of AI systems used in critical decision-making fields.