Show HN: Runtime data provenance for AI pipelines (github.com)

A new library called Origin has been introduced for tracking data provenance in AI training pipelines. This lightweight tool automatically records how data flows through the machine learning lifecycle, generating cryptographic fingerprints and documenting license metadata, which is increasingly important for compliance with regulations like the EU AI Act. As AI models are trained on large, often heterogeneous datasets, challenges such as ensuring data transparency, avoiding license conflicts, and maintaining an audit trail have become critical. Origin addresses these issues directly by creating an observation layer that does not alter the training process while documenting every aspect of data lineage. The significance of Origin lies in its contributions to the AI/ML community's transparency and reproducibility standards. By leveraging SHA-256 hashes and Merkle trees for data fingerprinting, Origin not only facilitates efficient verification and tamper detection but also ensures that organizations can provide audit-ready reports concerning the data used for training. The library has zero dependencies and operates locally, meaning users maintain control over their data with no external connectivity required. This focus on deterministic operations and compliance generates a safer pipeline for AI development, reducing legal risks associated with data usage while enhancing overall accountability in AI model training.