Operational HPC lessons learnt training the Tessera geospatial foundation models [pdf] (www.cl.cam.ac.uk)

Researchers from the University of Cambridge have shared operational insights from training the Tessera geospatial foundation model, a pioneering framework that integrates satellite optical and radar data into unified annual embeddings for global land observations. This model stands out as it captures valuable environmental data at a resolution of 10 meters, facilitating numerous applications in ecology, agriculture, and weather prediction. Within just a year of its release, Tessera has gained traction, attracting usage from global researchers and organizations without necessitating repeated preprocessing or model inference. The significance of Tessera lies in its state-of-the-art approach to generating compact, reusable data representations from vast amounts of satellite imagery, enabling efficient access to high-quality environmental insights. The training process involved innovative use of hybrid computing resources across Intel, NVIDIA, and AMD platforms, ultimately resulting in the generation of a 21-million-parameter model that excels in diverse environmental tasks. This project not only highlights the complex infrastructure challenges encountered when working at planetary scales but also serves as a crucial guide for future computational practices in the AI/ML community, demonstrating how to manage heterogeneous systems effectively while focusing on sustainability and accessibility.