My Name Is Riemann – Prerelease (huggingface.co)

OpenPeer AI has announced the prerelease of "My Name Is Riemann," a groundbreaking 13.1 billion parameter decoder-only language model that showcases an innovative growth technique. Developed as part of the My Name Is Riemann line, this model builds upon the OpenPeerLLM architecture by sequentially expanding each checkpoint—growing wider and deeper while inheriting the learning from previous iterations. Fine-tuned with the OpenPeer-AI-NTK-Trainer, the model emphasizes text generation for code and multi-step reasoning, trained on a diverse 2.30 billion token corpus comprised mainly of coding transcripts, encyclopedic entries in Arabic and English, and source code examples. This release is significant as it pushes the envelope in language model training, particularly by employing a neural tangent kernel theory-based gate controller while ensuring the base weights remain frozen. The model’s current capabilities, demonstrated through a small subset of pretraining tokens (2.16 billion at this stage), reveal potential for further enhancements. Although touted for tasks like bilingual question-answering and code completion, users are cautioned about its limitations in factual reliability and safety, indicating a need for careful drafting and content filtering before deployment in sensitive environments. The prerelease serves as a foundation for future iterations and optimizations, inviting exploration and contributions from the AI/ML community.