Rnj-1 – open-weight, dense models trained from scratch by Essential AI (huggingface.co)

Essential AI has launched Rnj-1, a new family of open-weight, 8 billion parameter dense models trained from scratch, designed specifically for programming and STEM tasks. This initiative is significant for the AI/ML community as it offers capabilities that rival state-of-the-art (SOTA) models, providing strong performance in code generation, mathematical problem-solving, and scientific reasoning. The Rnj-1 models, consisting of a base model and an instruction-tuned variant (Rnj-1-instruct), excel in agentic frameworks, showcasing superior tool-use capabilities and the potential for effective specialization in various domains. Technically, Rnj-1 is built with a global attention architecture, featuring 32 layers and an extended context length of up to 32,000 tokens after a mid-training phase. The models were pre-trained on 8.4 trillion tokens and utilize advanced optimization techniques. Evaluation findings reveal impressive benchmarks in code comprehension and generation across multiple languages, outperforming comparable models in tasks such as SWE-bench and algorithmic problem-solving. Both Rnj-1 and Rnj-1-instruct are designed to be extensible for the community, promoting further innovation and specialization in AI functionalities.