Empero: A 9B that checks its own work (empero.org)

Empero, a new language model boasting 9 billion parameters, has been developed to autonomously validate its responses, marking a notable advancement in self-checking AI systems. Trained with a focus on smaller, more open models, Empero is incorporated with an innovative evaluation harness designed to refine its architecture and streamline its training data generation. The recent benchmarks reveal mixed results, with specific metrics like gpqa-diamond showing a slight decline of 0.05, while others remain consistent, indicating areas for further improvement. This development is significant for the AI/ML community as it emphasizes the potential for smaller models to perform robustly while maintaining self-correction capabilities. By building the entire framework—from the model evaluation harness to the data pipeline—Empero showcases a holistic approach to model training and performance validation, which could inspire future AI research. This self-checking mechanism could lead to more reliable AI applications, as models are increasingly relied upon for decision-making in critical sectors.