about
A Method for the Architecture of a Medical Vertical LLM Based on Deepseek R1 (arxiv.org)
1 point by PaulHoule on May 21, 2025 | hide | past | pdf | discuss on HN

In plain words: A big medical reasoning model teaches a much smaller one its knowledge, then the small model is shrunk further to run on ordinary hardware. It kept nearly the same accuracy on USMLE-style questions while using 64.7% less memory than the usual full-size setup.

Abstract · A Method for the Architecture of a Medical Vertical Large Language Model Based on Deepseek R1

Despite significant advances in foundation models like DeepSeek-R1 and ChatGPT, their deployment in medical settings faces critical challenges including computational requirements and professional knowledge barriers. This paper presents an efficient lightweight medical large language model architecture that systematically addresses these challenges through three-dimensional optimization: knowledge acquisition, model compression, and computational enhancement. We design a knowledge transfer pipeline from DeepSeek-R1-Distill-70B to DeepSeek-R1-Distill-7B using Low-Rank Adaptation (LoRA) for precise medical knowledge retention. Through 4-bit quantization and mixed-precision strategies, we achieve substantial model compression while preserving medical reasoning capabilities. The inference framework incorporates Flash Attention acceleration and continuous batching, complemented by specialized prompt templates for diverse medical queries. Experimental evaluation on medical benchmarks demonstrates that our approach maintains 92.1% accuracy on USMLE examinations while reducing memory consumption by 64.7% and inference latency by 12.4% compared to baseline models. This work provides a practical solution for deploying advanced language models in resource-constrained medical environments, enabling broader accessibility of AI-assisted healthcare.

Mingda Zhang, Jianglong Qin
arXiv:2505.00025 · cs.CL, cs.AI · submitted Apr 25, 2025 · updated Jul 22, 2025
abstract · pdf · html · 14 pages, 1 figures

add comment on HN