https://ucsc-vlaa.github.io/ViLBench/\n","updatedAt":"2025-03-27T07:33:39.778Z","author":{"_id":"604ae011caabafacfa48e3de","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1615519738679-noauth.jpeg","fullname":"Haoqin Tu","name":"PahaII","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.24655406177043915},"editors":["PahaII"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1615519738679-noauth.jpeg"],"reactions":[],"isReport":false}},{"id":"67e5fc923f41353ad4c1612b","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-03-28T01:34:10.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models](https://huggingface.co/papers/2502.14191) (2025)\n* [Demystifying Multilingual Chain-of-Thought in Process Reward Modeling](https://huggingface.co/papers/2502.12663) (2025)\n* [MM-RLHF: The Next Step Forward in Multimodal LLM Alignment](https://huggingface.co/papers/2502.10391) (2025)\n* [Process-based Self-Rewarding Language Models](https://huggingface.co/papers/2503.03746) (2025)\n* [Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models](https://huggingface.co/papers/2501.18533) (2025)\n* [Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning](https://huggingface.co/papers/2502.14356) (2025)\n* [R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization](https://huggingface.co/papers/2503.12937) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models (2025) \n
- Demystifying Multilingual Chain-of-Thought in Process Reward Modeling (2025) \n
- MM-RLHF: The Next Step Forward in Multimodal LLM Alignment (2025) \n
- Process-based Self-Rewarding Language Models (2025) \n
- Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models (2025) \n
- Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning (2025) \n
- R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization (2025) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Papers
arxiv:2503.20271
ViLBench: A Suite for Vision-Language Process Reward Modeling
Published on Mar 26, 2025
Abstract
Evaluation of process-supervised reward models on vision-language benchmarks reveals variability in performance and introduces a challenging benchmark, ViLBench, where current models show significant room for improvement.
AI-generated summary
Process-supervised reward models serve as a fine-grained function that provides detailed step-wise feedback to model responses, facilitating effective selection of reasoning trajectories for complex tasks. Despite its advantages, evaluation on PRMs remains less explored, especially in the multimodal domain. To address this gap, this paper first benchmarks current vision large language models (VLLMs) as two types of reward models: output reward models (ORMs) and process reward models (PRMs) on multiple vision-language benchmarks, which reveal that neither ORM nor PRM consistently outperforms across all tasks, and superior VLLMs do not necessarily yield better rewarding performance. To further advance evaluation, we introduce ViLBench, a vision-language benchmark designed to require intensive process reward signals. Notably, OpenAI's GPT-4o with Chain-of-Thought (CoT) achieves only 27.3% accuracy, indicating the benchmark's challenge for current VLLMs. Lastly, we preliminarily showcase a promising pathway towards bridging the gap between general VLLMs and reward models -- by collecting 73.6K vision-language process reward data using an enhanced tree-search algorithm, our 3B model is able to achieve an average improvement of 3.3% over standard CoT and up to 2.5% compared to its untrained counterpart on ViLBench by selecting OpenAI o1's generations. We release the implementations at https://ucsc-vlaa.github.io/ViLBench with our code, model, and data.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models (2025)
- Demystifying Multilingual Chain-of-Thought in Process Reward Modeling (2025)
- MM-RLHF: The Next Step Forward in Multimodal LLM Alignment (2025)
- Process-based Self-Rewarding Language Models (2025)
- Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models (2025)
- Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning (2025)
- R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2503.20271 in a model README.md to link it from this page.
Datasets citing this paper 2
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2503.20271 in a Space README.md to link it from this page.