TServe – Open-source inference server for time-series foundation models (github.com)

TServe, an open-source inference server for time-series foundation models, has been launched to streamline the forecasting process in AI/ML applications. It efficiently loads and keeps various models, including Chronos and TimesFM, in memory, ready to respond to HTTP-based forecast requests. By using a unified interface through the `sktime` library, users can easily switch between different models with minimal changes to their pipelines, significantly enhancing flexibility and productivity for developers working with time-series data. This innovation is significant as it centralizes model serving while optimizing inference performance. TServe maintains model weights in memory after initial load, ensuring that subsequent requests incur only inference costs. The server accepts input in various formats like JSON or native Python tables (pandas, polars, or pyarrow) and offers a user-friendly browser dashboard for visualization. With over 100 pre-trained checkpoints and the option to deploy models using Docker, TServe provides a comprehensive and accessible solution for deploying scalable time-series forecasting, thus advancing the capabilities and practical applications of AI in analyzing time-based data.