The following papers were recommended by the Semantic Scholar API
\n- \n
- CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoning (2025) \n
- Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse (2025) \n
- MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources (2025) \n
- Staying in the Sweet Spot: Responsive Reasoning Evolution via Capability-Adaptive Hint Scaffolding (2025) \n
- COPO: Consistency-Aware Policy Optimization (2025) \n
- DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (2025) \n
- Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR (2025) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs
Abstract
CurES, a reinforcement learning-based method, improves the training efficiency of large language models by optimizing prompt selection and rollout allocation, leading to faster convergence and reduced computational overhead.
Curriculum learning plays a crucial role in enhancing the training efficiency of large language models (LLMs) on reasoning tasks. However, existing methods often fail to adequately account for variations in prompt difficulty or rely on simplistic filtering mechanisms to select prompt datasets within a narrow criterion range, resulting in significant computational waste. In this work, we approach the problem from the perspective of reinforcement learning gradient optimization, offering a systematic and theoretical investigation into how to improve the training efficiency of LLMs. We identify two key factors influencing training efficiency: the selection of training prompts and the allocation of rollout quantities across different prompts. Our theoretical analysis reveals that the sampling distribution of prompts dictates the convergence rate of gradient descent, while the allocation of the rollout quantity influences the consistency and stability of overall gradient updates. Based on these insights, we propose CurES, an efficient training method that accelerates convergence and employs Bayesian posterior estimation to minimize computational overhead. Experiments demonstrate that our CurES outperforms Group Relative Policy Optimization (GRPO) by +3.30 points and +4.82 points with 1.5B and 7B models, respectively. Additionally, CurES exhibits faster convergence compared to baselines, including GRPO.
Community
This paper proposes a gradient analysis based budget allocation method for reinforcement learning in LLM reasoning.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoning (2025)
- Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse (2025)
- MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources (2025)
- Staying in the Sweet Spot: Responsive Reasoning Evolution via Capability-Adaptive Hint Scaffolding (2025)
- COPO: Consistency-Aware Policy Optimization (2025)
- DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (2025)
- Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper