Measuring the Trustworthiness of Open-Source-Derived Models (cognition.com)

A new evaluation suite has been developed to measure the trustworthiness of AI models, particularly those derived from open-source bases such as Kimi K2.7 Code. This suite includes direct questioning to identify propaganda outputs and tests in realistic coding scenarios to ensure consistent model behavior across different contexts. The results are promising: the newly developed model, SWE-1.7, performs comparably or better than models from leading U.S. labs, showcasing the potential for open-source models to be reliable when developed with care and precision. This advancement is significant for the AI/ML community, particularly in addressing concerns around bias and security in models derived from open-source foundations, often associated with politically driven narratives. The evaluation methodology assesses both propaganda rates and compliance with politically motivated requests, revealing that while many open-source models may replicate biased narratives, SWE-1.7 exhibits a marked improvement in neutrality and safety. These findings encourage further exploration of open-source models for critical applications, suggesting that, if properly handled, they can rival proprietary models in terms of safety and performance.