Are time series foundation models ready to ship? (devetak.github.io)

Time series foundation models, particularly Mistral's newly released Timestral, are being evaluated for their effectiveness in predicting port traffic disruptions caused by severe weather events, exemplified by the recent impacts of Typhoon Saola and Typhoon Koinu on Hong Kong's operations. Despite the promising use case—a model designed to extrapolate patterns from past disturbances—the model struggled to accurately forecast the traffic patterns following these disruptions. For instance, while it was logical to anticipate a spike in activity after a closure, Timestral's predictions remained too conservative and did not adequately reflect the expected recovery dynamics. This limitation highlights a critical gap in current time series modeling: the lack of contextual metadata that could enhance the model’s predictive capabilities. The findings suggest that while time series models can capture seasonality, such as weekend effects, they may falter in recognizing the influence of external factors, like shipping decisions and geopolitical events. As the AI/ML community continues to refine these models, integrating richer contextual information may be essential for improving their performance and ensuring they can handle complexities inherent in real-world scenarios.