Show HN: We open-sourced Argus, our reproducibility protocol for ML workloads (github.com)

Argus, a newly open-sourced reproducibility protocol for machine learning (ML) workloads, has been launched to enhance the validation process in ML experiments. This tool is designed to track structural behavior changes under identical conditions, providing valuable insights into performance consistency without claiming absolute performance. Argus generates a Reproducible Observation Record that includes essential metrics, such as latency, throughput, and resource usage, which help researchers validate their experiments more reliably. The significance of Argus lies in its focus on reproducibility, a critical aspect of ML research that can often be overlooked due to varying environmental factors. By recording runtime context, including CPU load, memory usage, and GPU metrics, it alerts users to potential environmental noise that may affect results, and offers options for sanitizing data before sharing. This can foster greater collaboration and transparency in the AI/ML community, ensuring that results are not only achieved but also verifiable, thereby advancing the field's collective trust in performance claims.