about
SciGPT: A LLM for Scientific Literature Understanding and Knowledge Discovery (arxiv.org)
3 points by PaulHoule on Sep 21, 2025 | hide | past | pdf | discuss on HN

In plain words: SciGPT is a language model trained on scientific papers, using a cheaper two-stage training process, a memory-saving attention design, and built-in knowledge maps to handle jargon across fields. It beat general-purpose GPT-4o on core scientific tasks and cut memory use 55% for long documents.

Abstract · SciGPT: A Large Language Model for Scientific Literature Understanding and Knowledge Discovery

Scientific literature is growing exponentially, creating a critical bottleneck for researchers to efficiently synthesize knowledge. While general-purpose Large Language Models (LLMs) show potential in text processing, they often fail to capture scientific domain-specific nuances (e.g., technical jargon, methodological rigor) and struggle with complex scientific tasks, limiting their utility for interdisciplinary research. To address these gaps, this paper presents SciGPT, a domain-adapted foundation model for scientific literature understanding and ScienceBench, an open source benchmark tailored to evaluate scientific LLMs. Built on the Qwen3 architecture, SciGPT incorporates three key innovations: (1) low-cost domain distillation via a two-stage pipeline to balance performance and efficiency; (2) a Sparse Mixture-of-Experts (SMoE) attention mechanism that cuts memory consumption by 55\% for 32,000-token long-document reasoning; and (3) knowledge-aware adaptation integrating domain ontologies to bridge interdisciplinary knowledge gaps. Experimental results on ScienceBench show that SciGPT outperforms GPT-4o in core scientific tasks including sequence labeling, generation, and inference. It also exhibits strong robustness in unseen scientific tasks, validating its potential to facilitate AI-augmented scientific discovery.

Fengyu She, Nan Wang, Hongfei Wu, Ziyi Wan, Jingmian Wang, Chang Wang
arXiv:2509.08032 · cs.CL · submitted Sep 9, 2025
abstract · pdf · html

add comment on HN