HiGP: A high-performance Python package for Gaussian Process (arxiv.org)

HiGP is an innovative high-performance Python package aimed at enhancing the usability of Gaussian Processes (GPs) in machine learning, particularly for regression and classification tasks. The major challenge with traditional GPs is their high computational cost, which scales cubically with the size of the dataset due to kernel matrix operations. HiGP addresses this limitation by implementing advanced numerical methods, including H² matrices to achieve near-linear complexity in both computation and storage. This allows it to process spatial datasets efficiently and conduct on-the-fly kernel evaluations, significantly easing the burden of explicit storage in large-scale applications. The significance of HiGP for the AI/ML community lies in its ability to make Gaussian Processes more scalable and practical for real-world applications. Its C++ backend ensures high performance while maintaining seamless integration with popular machine learning workflows through Python interfaces. Furthermore, HiGP introduces a robust Adaptive Factorized Nyström preconditioner that enhances the convergence speed of iterative solvers across diverse kernel spectra. Additionally, it features analytically derived gradient computations that facilitate efficient hyperparameter optimization, minimizing the typical overhead associated with automatic differentiation. As a reusable numerical engine, HiGP complements existing frameworks like GPJax and KeOps, providing a reliable computational backbone for large-scale GP tasks.