https://github.com/SkyworkAI/Skywork-R1V
$\"r1v_2.gif\"$

\n","updatedAt":"2025-04-28T06:49:19.241Z","author":{"_id":"6462b241b438438da3c25a5d","avatarUrl":"/avatars/606a67f1be639c9a5e36f293abd5f27a.svg","fullname":"Xuchen Song","name":"xuchensong","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.48046499490737915},"editors":["xuchensong"],"editorAvatarUrls":["/avatars/606a67f1be639c9a5e36f293abd5f27a.svg"],"reactions":[],"isReport":false}},{"id":"68102cbce09c7e5ee6a8678c","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-04-29T01:34:52.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought](https://huggingface.co/papers/2504.05599) (2025)\n* [R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization](https://huggingface.co/papers/2503.10615) (2025)\n* [VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning](https://huggingface.co/papers/2504.08837) (2025)\n* [MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning](https://huggingface.co/papers/2503.07365) (2025)\n* [Reasoning Beyond Limits: Advances and Open Problems for LLMs](https://huggingface.co/papers/2503.22732) (2025)\n* [Video-R1: Reinforcing Video Reasoning in MLLMs](https://huggingface.co/papers/2503.21776) (2025)\n* [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://huggingface.co/papers/2503.06749) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-04-29T01:34:52.064Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7146794199943542},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2504.16656","authors":[{"_id":"6809a4ac81a95c83f0c81c83","user":{"_id":"620f5a1c3f76c50e6458a9b6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg","isPro":true,"fullname":"Peiyu Wang","user":"OrlandoHugBot","type":"user"},"name":"Chris","status":"admin_assigned","statusLastChangedAt":"2025-05-09T12:34:22.418Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c84","name":"Yichen Wei","hidden":false},{"_id":"6809a4ac81a95c83f0c81c85","user":{"_id":"6321769fb8297a7899903f5c","avatarUrl":"/avatars/12c86bd9191cdfddf2cf693db3125be8.svg","isPro":false,"fullname":"PENG YI","user":"pengyi","type":"user"},"name":"Yi Peng","status":"admin_assigned","statusLastChangedAt":"2025-04-29T12:31:13.114Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c86","user":{"_id":"62be9b5aae56e75e4d689e7c","avatarUrl":"/avatars/6772bc09d6eeb4e86b1210481be91720.svg","isPro":false,"fullname":"wangxiaokun","user":"shawn0wang","type":"user"},"name":"Xiaokun Wang","status":"claimed_verified","statusLastChangedAt":"2025-05-09T07:22:46.116Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c87","user":{"_id":"660aab2c878289c5b34f9e97","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/660aab2c878289c5b34f9e97/yxx1-lR8x5o6KaEpZDXQq.jpeg","isPro":false,"fullname":"weijie qiu","user":"qiuwj","type":"user"},"name":"Weijie Qiu","status":"admin_assigned","statusLastChangedAt":"2025-04-29T12:30:41.500Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c88","name":"Wei Shen","hidden":false},{"_id":"6809a4ac81a95c83f0c81c89","user":{"_id":"63fdb1aa27abbe6b3ce098f5","avatarUrl":"/avatars/c22e3a77ff84b3b87c16cff2469f6d3d.svg","isPro":false,"fullname":"xietian","user":"sealical","type":"user"},"name":"Tianyidan Xie","status":"claimed_verified","statusLastChangedAt":"2025-04-28T12:50:25.445Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c8a","user":{"_id":"653df83feefdcdc9fd8a5b13","avatarUrl":"/avatars/f7f5bc67df7f8bf8b28e9bbd4fa6c397.svg","isPro":false,"fullname":"Jiangbo Pei","user":"jiangbop","type":"user"},"name":"Jiangbo Pei","status":"admin_assigned","statusLastChangedAt":"2025-04-29T12:30:09.986Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c8b","user":{"_id":"64235adf23b98d5ee6e9ca85","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64235adf23b98d5ee6e9ca85/m_7uwLaAkTXyt3qB2pPLA.png","isPro":false,"fullname":"Jianhao Zhang","user":"Heerozh","type":"user"},"name":"Jianhao Zhang","status":"admin_assigned","statusLastChangedAt":"2025-04-29T12:29:46.829Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c8c","user":{"_id":"653dd16277c2f09452ad37cd","avatarUrl":"/avatars/a95f9527722845a5414d86180c8e945d.svg","isPro":false,"fullname":"Yunzhuo Hao","user":"luckychao","type":"user"},"name":"Yunzhuo Hao","status":"admin_assigned","statusLastChangedAt":"2025-04-29T12:29:40.419Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c8d","user":{"_id":"6462b241b438438da3c25a5d","avatarUrl":"/avatars/606a67f1be639c9a5e36f293abd5f27a.svg","isPro":false,"fullname":"Xuchen Song","user":"xuchensong","type":"user"},"name":"Xuchen Song","status":"claimed_verified","statusLastChangedAt":"2025-04-25T08:35:17.241Z","hidden":false},{"_id":"6809a4ac81a95c83f0c81c8e","name":"Yang Liu","hidden":false},{"_id":"6809a4ac81a95c83f0c81c8f","name":"Yahui Zhou","hidden":false}],"publishedAt":"2025-04-23T12:24:10.000Z","submittedOnDailyAt":"2025-04-28T05:19:19.230Z","title":"Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning","submittedOnDailyBy":{"_id":"6462b241b438438da3c25a5d","avatarUrl":"/avatars/606a67f1be639c9a5e36f293abd5f27a.svg","isPro":false,"fullname":"Xuchen Song","user":"xuchensong","type":"user"},"summary":"We present Skywork R1V2, a next-generation multimodal reasoning model and a\nmajor leap forward from its predecessor, Skywork R1V. At its core, R1V2\nintroduces a hybrid reinforcement learning paradigm that harmonizes\nreward-model guidance with rule-based strategies, thereby addressing the\nlong-standing challenge of balancing sophisticated reasoning capabilities with\nbroad generalization. To further enhance training efficiency, we propose the\nSelective Sample Buffer (SSB) mechanism, which effectively counters the\n``Vanishing Advantages'' dilemma inherent in Group Relative Policy Optimization\n(GRPO) by prioritizing high-value samples throughout the optimization process.\nNotably, we observe that excessive reinforcement signals can induce visual\nhallucinations--a phenomenon we systematically monitor and mitigate through\ncalibrated reward thresholds throughout the training process. Empirical results\naffirm the exceptional capability of R1V2, with benchmark-leading performances\nsuch as 62.6 on OlympiadBench, 79.0 on AIME2024, 63.6 on LiveCodeBench, and\n74.0 on MMMU. These results underscore R1V2's superiority over existing\nopen-source models and demonstrate significant progress in closing the\nperformance gap with premier proprietary systems, including Gemini 2.5 and\nOpenAI o4-mini. The Skywork R1V2 model weights have been publicly released to\npromote openness and reproducibility\nhttps://huggingface.co/Skywork/Skywork-R1V2-38B.","upvotes":57,"discussionId":"6809a4ae81a95c83f0c81cda","projectPage":"https://huggingface.co/Skywork/Skywork-R1V2-38B","ai_summary":"Skywork R1V2 enhances multimodal reasoning through a hybrid reinforcement learning approach that balances reward-model guidance and rule-based strategies, improving training efficiency with the Selective Sample Buffer mechanism and mitigating visual hallucinations.","ai_keywords":["hybrid reinforcement learning","reward-model guidance","rule-based strategies","Selective Sample Buffer (SSB)","Vanishing Advantages","Group Relative Policy Optimization (GRPO)","visual hallucinations","OlympiadBench","AIME2024","LiveCodeBench","MMMU"],"organization":{"_id":"6522615d9334173c627b0efa","name":"Skywork","fullname":"Skywork","avatar":"https://cdn-uploads.huggingface.co/production/uploads/64535b71bcbd25618f7655da/AvtJ4GuPAyhLxl2-leVt6.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"60bc063208048a33ffdb1a6d","avatarUrl":"/avatars/33aabcf608d06426c259f3fcc57115dc.svg","isPro":true,"fullname":"Mwangi","user":"Benson","type":"user"},{"_id":"63b6f2e752c02ae8acbaa4d8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1672934038280-noauth.jpeg","isPro":false,"fullname":"Habibullah Akbar","user":"ChavyvAkvar","type":"user"},{"_id":"6462b241b438438da3c25a5d","avatarUrl":"/avatars/606a67f1be639c9a5e36f293abd5f27a.svg","isPro":false,"fullname":"Xuchen Song","user":"xuchensong","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"620f5a1c3f76c50e6458a9b6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg","isPro":true,"fullname":"Peiyu Wang","user":"OrlandoHugBot","type":"user"},{"_id":"653dd16277c2f09452ad37cd","avatarUrl":"/avatars/a95f9527722845a5414d86180c8e945d.svg","isPro":false,"fullname":"Yunzhuo Hao","user":"luckychao","type":"user"},{"_id":"660aab2c878289c5b34f9e97","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/660aab2c878289c5b34f9e97/yxx1-lR8x5o6KaEpZDXQq.jpeg","isPro":false,"fullname":"weijie qiu","user":"qiuwj","type":"user"},{"_id":"673f0a5bcdad8a9744d17df0","avatarUrl":"/avatars/413b0472c9790395a64aafe9294143bd.svg","isPro":false,"fullname":"Yichen Wei","user":"yichenchenchen","type":"user"},{"_id":"67f61e1459f6e8c3698a84a9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/eRTXFseUr13f-aaY3LXgn.png","isPro":false,"fullname":"peng bin","user":"pengdott","type":"user"},{"_id":"63d8773a635499f21644601b","avatarUrl":"/avatars/07efea204d172e2844dbacc418131da2.svg","isPro":false,"fullname":"Wei Tianwen","user":"weitianwen","type":"user"},{"_id":"63fdb1aa27abbe6b3ce098f5","avatarUrl":"/avatars/c22e3a77ff84b3b87c16cff2469f6d3d.svg","isPro":false,"fullname":"xietian","user":"sealical","type":"user"},{"_id":"619b03a080ebe7c9091fbf3c","avatarUrl":"/avatars/0b4be841601195cc73d984055ffab565.svg","isPro":false,"fullname":"Hu Dou Dou","user":"hl0737","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":2,"organization":{"_id":"6522615d9334173c627b0efa","name":"Skywork","fullname":"Skywork","avatar":"https://cdn-uploads.huggingface.co/production/uploads/64535b71bcbd25618f7655da/AvtJ4GuPAyhLxl2-leVt6.jpeg"}}">

Papers

arxiv:2504.16656

Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning

Published on Apr 23, 2025

· Submitted by

Xuchen Song on Apr 28, 2025

#2 Paper of the day

Skywork

Upvote

Authors:

Chris ,

Yi Peng ,

Xiaokun Wang ,

Weijie Qiu ,

Tianyidan Xie ,

Jiangbo Pei ,

Jianhao Zhang ,

Yunzhuo Hao ,

Xuchen Song ,

Abstract

Skywork R1V2 enhances multimodal reasoning through a hybrid reinforcement learning approach that balances reward-model guidance and rule-based strategies, improving training efficiency with the Selective Sample Buffer mechanism and mitigating visual hallucinations.

AI-generated summary

We present Skywork R1V2, a next-generation multimodal reasoning model and a major leap forward from its predecessor, Skywork R1V. At its core, R1V2 introduces a hybrid reinforcement learning paradigm that harmonizes reward-model guidance with rule-based strategies, thereby addressing the long-standing challenge of balancing sophisticated reasoning capabilities with broad generalization. To further enhance training efficiency, we propose the Selective Sample Buffer (SSB) mechanism, which effectively counters the ``Vanishing Advantages'' dilemma inherent in Group Relative Policy Optimization (GRPO) by prioritizing high-value samples throughout the optimization process. Notably, we observe that excessive reinforcement signals can induce visual hallucinations--a phenomenon we systematically monitor and mitigate through calibrated reward thresholds throughout the training process. Empirical results affirm the exceptional capability of R1V2, with benchmark-leading performances such as 62.6 on OlympiadBench, 79.0 on AIME2024, 63.6 on LiveCodeBench, and 74.0 on MMMU. These results underscore R1V2's superiority over existing open-source models and demonstrate significant progress in closing the performance gap with premier proprietary systems, including Gemini 2.5 and OpenAI o4-mini. The Skywork R1V2 model weights have been publicly released to promote openness and reproducibility https://huggingface.co/Skywork/Skywork-R1V2-38B.