Lingshu-32B outperforms GPT-4.1 and Claude Sonnet 4 in most multimodal QA and report generation tasks.\n\n","updatedAt":"2025-06-10T02:18:48.102Z","author":{"_id":"604f67ef0fe8ff3ec13d71ef","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/604f67ef0fe8ff3ec13d71ef/KhUwWvZ3OJ9nEee3B-SXO.png","fullname":"Hou Pong (Ken) Chan","name":"kenchan0226","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":16,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.796971321105957},"editors":["kenchan0226"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/604f67ef0fe8ff3ec13d71ef/KhUwWvZ3OJ9nEee3B-SXO.png"],"reactions":[],"isReport":false}},{"id":"6847cf4bd0ad6bf06f935ddc","author":{"_id":"641433e38900ef6afa3095f1","avatarUrl":"/avatars/376ee167592ce03d723849f210788ebf.svg","fullname":"Henri Gelender","name":"MortFee","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false},"createdAt":"2025-06-10T06:23:07.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"\n\n","html":"
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- MEDMKG: Benchmarking Medical Knowledge Exploitation with Multimodal Knowledge Graph (2025) \n
- CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs (2025) \n
- Infi-Med: Low-Resource Medical MLLMs with Robust Reasoning Evaluation (2025) \n
- ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification (2025) \n
- Elicit and Enhance: Advancing Multimodal Reasoning in Medical Scenarios (2025) \n
- VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge (2025) \n
- MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM (2025) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Papers
arxiv:2506.07044
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Published on Jun 8, 2025
Abstract
A medical-specialized multimodal large language model, Lingshu, is introduced with enhanced data curation and reinforcement learning to address limitations in medical applications.
AI-generated summary
Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in understanding common visual elements, largely due to their large-scale datasets and advanced training strategies. However, their effectiveness in medical applications remains limited due to the inherent discrepancies between data and tasks in medical scenarios and those in the general domain. Concretely, existing medical MLLMs face the following critical limitations: (1) limited coverage of medical knowledge beyond imaging, (2) heightened susceptibility to hallucinations due to suboptimal data curation processes, (3) lack of reasoning capabilities tailored for complex medical scenarios. To address these challenges, we first propose a comprehensive data curation procedure that (1) efficiently acquires rich medical knowledge data not only from medical imaging but also from extensive medical texts and general-domain data; and (2) synthesizes accurate medical captions, visual question answering (VQA), and reasoning samples. As a result, we build a multimodal dataset enriched with extensive medical knowledge. Building on the curated data, we introduce our medical-specialized MLLM: Lingshu. Lingshu undergoes multi-stage training to embed medical expertise and enhance its task-solving capabilities progressively. Besides, we preliminarily explore the potential of applying reinforcement learning with verifiable rewards paradigm to enhance Lingshu's medical reasoning ability. Additionally, we develop MedEvalKit, a unified evaluation framework that consolidates leading multimodal and textual medical benchmarks for standardized, fair, and efficient model assessment. We evaluate the performance of Lingshu on three fundamental medical tasks, multimodal QA, text-based QA, and medical report generation. The results show that Lingshu consistently outperforms the existing open-source multimodal models on most tasks ...
Community
🌟 Highlights:
- Lingshu supports more than 12 medical imaging modalities, including X-Ray, CT Scan, MRI, Microscopy, Ultrasound, Histopathology, Dermoscopy, Fundus, OCT, Digital Photography, Endoscopy, and PET.
- Lingshu models achieve SOTA on most medical multimodal/textual QA and report generation tasks for 7B and 32 model sizes.
- Lingshu-32B outperforms GPT-4.1 and Claude Sonnet 4 in most multimodal QA and report generation tasks.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- MEDMKG: Benchmarking Medical Knowledge Exploitation with Multimodal Knowledge Graph (2025)
- CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs (2025)
- Infi-Med: Low-Resource Medical MLLMs with Robust Reasoning Evaluation (2025)
- ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification (2025)
- Elicit and Enhance: Advancing Multimodal Reasoning in Medical Scenarios (2025)
- VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge (2025)
- MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend