ModelDNA: Verifying the lineage of open-weight LLMs from weight fingerprints (arxiv.org)

Researchers have introduced modelDNA, a groundbreaking tool designed to verify the lineage of open-weight language models (LLMs) by using weight fingerprints. Traditionally, the lineage information on platforms like Hugging Face is often self-reported and unverified, leading to ambiguity about a model's origins. modelDNA circumvents this limitation by analyzing a model's fingerprint from a minimal 100-300 MB of data instead of extensive 15 GB downloads. The tool compares this fingerprint to a reference database, achieving an impressive area under the receiver operating characteristic curve (AUROC) of 1.0, indicating zero false positives and complete accuracy in identifying parent models. This development is significant for the AI/ML community, as it not only enhances transparency in model provenance but also addresses the contentious issue of lineage disputes more efficiently. Furthermore, it introduces a method for merge decomposition, enabling the recovery of mixture weights from model fingerprints without needing to download the actual models. This technique can accurately replicate existing merge methods, showcasing a strong correlation with established values. With all fingerprints, benchmarks, and an inferred lineage graph publicly available, modelDNA promises to foster trust and reproducibility in the burgeoning field of open-weight LLMs.