SciGPT: A LLM for Scientific Literature Understanding and Knowledge Discovery (arxiv.org)

Researchers introduced SciGPT, a domain-adapted large language model and a companion open-source benchmark, ScienceBench, designed to tackle the exploding volume and complexity of scientific literature. Built on the Qwen3 architecture, SciGPT targets the shortcomings of general-purpose LLMs — inability to handle technical jargon, methodological nuance, and long-document synthesis — and demonstrates better performance and robustness than GPT-4o across core scientific tasks such as sequence labeling, generation, and inference. ScienceBench provides a standardized way to evaluate models on scientific understanding and knowledge discovery, helping the community measure real-world research utility. Technically, SciGPT combines three innovations: a two-stage, low-cost domain distillation pipeline that preserves scientific signal while improving efficiency; a Sparse Mixture-of-Experts (SMoE) attention mechanism that cuts memory use by 55% for 32,000-token long-document reasoning (enabling practical, large-context processing of papers and datasets); and knowledge-aware adaptation that injects domain ontologies to better bridge interdisciplinary concepts. Together these advances make SciGPT both more capable on specialized scientific tasks and more efficient for long-form literature analysis, promising immediate impact on AI-augmented literature review, hypothesis generation, and cross-domain discovery workflows.