about
Quantized LLMss in Biomedical Natural Language Processing (arxiv.org)
1 point by PaulHoule on Sep 25, 2025 | hide | past | pdf | discuss on HN

In plain words: Quantization stores a model's numbers with less precision so it fits in far less memory. Testing on biomedical tasks showed it cut GPU memory by up to 75% with almost no drop in accuracy, letting big models run on ordinary local computers.

Abstract · Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation

Large language models have demonstrated remarkable capabilities in biomedical natural language processing, yet their rapid growth in size and computational requirements present a major barrier to adoption in healthcare settings where data privacy precludes cloud deployment and resources are limited. In this study, we systematically evaluated the impact of quantization on 12 state-of-the-art large language models, including both general-purpose and biomedical-specific models, across eight benchmark datasets covering four key tasks: named entity recognition, relation extraction, multi-label classification, and question answering. We show that quantization substantially reduces GPU memory requirements-by up to 75%-while preserving model performance across diverse tasks, enabling the deployment of 70B-parameter models on 40GB consumer-grade GPUs. In addition, domain-specific knowledge and responsiveness to advanced prompting methods are largely maintained. These findings provide significant practical and guiding value, highlighting quantization as a practical and effective strategy for enabling the secure, local deployment of large yet high-capacity language models in biomedical contexts, bridging the gap between technical advances in AI and real-world clinical translation.

Zaifu Zhan, Shuang Zhou, Min Zeng, Kai Yu, Meijia Song, Xiaoyi Chen, Jun Wang, Yu Hou, Rui Zhang
arXiv:2509.04534 · cs.CL, cs.AI · submitted Sep 4, 2025
abstract · pdf · html · 11 pages, 7 figures

add comment on HN