The following papers were recommended by the Semantic Scholar API
\n- \n
- DISPO: Enhancing Training Efficiency and Stability in Reinforcement Learning for Large Language Model Mathematical Reasoning (2026) \n
- Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026) \n
- Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning (2026) \n
- Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning (2026) \n
- Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models (2026) \n
- Thickening-to-Thinning: Reward Shaping via Human-Inspired Learning Dynamics for LLM Reasoning (2026) \n
- TACLer: Tailored Curriculum Reinforcement Learning for Efficient Reasoning (2026) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
On the Optimal Reasoning Length for RL-Trained Language Models
Abstract
Length control methods in reinforcement learning-trained language models affect reasoning performance and computational efficiency, with optimal output lengths balancing these factors.
Reinforcement learning substantially improves reasoning in large language models, but it also tends to lengthen chain of thought outputs and increase computational cost during both training and inference. Though length control methods have been proposed, it remains unclear what the optimal output length is for balancing efficiency and performance. In this work, we compare several length control methods on two models, Qwen3-1.7B Base and DeepSeek-R1-Distill-Qwen-1.5B. Our results indicate that length penalties may hinder reasoning acquisition, while properly tuned length control can improve efficiency for models with strong prior reasoning. By extending prior work to RL trained policies, we identify two failure modes, 1) long outputs increase dispersion, and 2) short outputs lead to under-thinking.
Community
RL-trained reasoning models often produce longer CoT, increasing test-time cost. We compare several length-control methods on Qwen3-1.7B-Base and DeepSeek-R1-Distill-Qwen-1.5B, and characterize when length penalties hurt reasoning acquisition vs when tuned control improves efficiency. We also highlight two failure modes: overly long outputs increase dispersion, while overly short outputs cause under-thinking.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- DISPO: Enhancing Training Efficiency and Stability in Reinforcement Learning for Large Language Model Mathematical Reasoning (2026)
- Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026)
- Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning (2026)
- Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning (2026)
- Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models (2026)
- Thickening-to-Thinning: Reward Shaping via Human-Inspired Learning Dynamics for LLM Reasoning (2026)
- TACLer: Tailored Curriculum Reinforcement Learning for Efficient Reasoning (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper