The unreasonable difficulty of time series forecasting (suzyahyah.github.io)

Recent discussions in the AI/ML community have highlighted the challenging nature of time series forecasting compared to other machine learning tasks. A comprehensive benchmark study tested various models, including traditional statistical models, deep learning architectures like DLinear and NLinear, and zero-shot foundation models, across diverse datasets such as M4 and Bitcoin prices. Remarkably, simpler statistical models and zero-shot approaches often outperformed more complex models, revealing that sophisticated algorithms frequently failed to predict time series accurately, drifting off course with many predictions. The study underscores that time series forecasting is inherently difficult due to unique characteristics such as low signal-to-noise ratios, limited effective sample sizes from data being autocorrelated, and challenges related to distribution shifts during both training and inference. Unlike i.i.d. data, time series relies on a single realization of a process, complicating the learning of predictive functions. These insights are significant as they challenge the conventional wisdom on model performance, prompting a re-evaluation of how forecasting models are developed and trained, and emphasizing the importance of addressing the fundamental properties of the underlying data generating process for improved predictability.