Measure the energy and CO2 of local LLMs (github.com)

Tracarbon has announced a new Python library and CLI tool designed to measure energy consumption and estimate CO2 emissions of local devices, making it easier for developers to track the environmental impact of their machine learning models and applications. With automatic detection of device and location, users can simply run commands to collect data or integrate the tracking feature into their applications via a Python API. This tool not only provides real-time measurements but also supports multiple exporters like Prometheus and Datadog to streamline data reporting. The significance of Tracarbon for the AI/ML community lies in its potential to promote sustainability within machine learning practices, which are often energy-intensive and environmentally taxing. By incorporating energy tracking directly into the workflow, developers can make more informed decisions about model training and deployment, thereby reducing carbon footprints. Tracarbon supports various environments, including Mac, Windows, and Linux, with specific metrics for CPU and GPU power usage, ultimately encouraging a shift towards more eco-friendly AI solutions.