FineDNA: Open data to train LLMs on DNA (huggingface.co)

Carbon-X has introduced FineDNA, a groundbreaking training corpus specifically designed to enhance language models' understanding of DNA sequences. The dataset integrates over 514 billion tokens from diverse sources, including genomic data from various organisms and biology-related texts. It has been used to train models such as Qwen3-14B, allowing them to process and interpret complex biological sequences through multiple training stages that include mid-training, supervised fine-tuning, and reinforcement learning. This multi-faceted approach equips Carbon-X with a robust ability to understand and manipulate biological information. The significance of FineDNA lies in its potential to bridge the gap between artificial intelligence and molecular biology, facilitating more accurate predictions and interactions with genetic data. By employing a mixture of DNA sequences, biology texts, and instructional dialogues, the model can cater to various biological applications, from genomic research to educational tools. This innovative dataset not only marks a step forward in machine learning applications in bioinformatics but also promotes accessibility to complex biological concepts, aiding researchers and educators in the field.