https://github.com/kanishkg/cognitive-behaviors :)\n","updatedAt":"2025-03-04T05:09:04.433Z","author":{"_id":"63e6a880f2e9a8f22c5a1630","avatarUrl":"/avatars/53b57690fe052ce6882bbfc87b11567c.svg","fullname":"Kanishk Gandhi","name":"obiwan96","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":8,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6801269054412842},"editors":["obiwan96"],"editorAvatarUrls":["/avatars/53b57690fe052ce6882bbfc87b11567c.svg"],"reactions":[{"reaction":"👍","users":["sugatoray"],"count":1}],"isReport":false}},{"id":"67c7aa002b26bf87ecb70796","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-03-05T01:33:52.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [LADDER: Self-Improving LLMs Through Recursive Problem Decomposition](https://huggingface.co/papers/2503.00735) (2025)\n* [AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO](https://huggingface.co/papers/2502.14669) (2025)\n* [Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling](https://huggingface.co/papers/2501.11651) (2025)\n* [Self-rewarding correction for mathematical reasoning](https://huggingface.co/papers/2502.19613) (2025)\n* [Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search](https://huggingface.co/papers/2502.02508) (2025)\n* [S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning](https://huggingface.co/papers/2502.12853) (2025)\n* [Self-Training Elicits Concise Reasoning in Large Language Models](https://huggingface.co/papers/2502.20122) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- LADDER: Self-Improving LLMs Through Recursive Problem Decomposition (2025) \n
- AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO (2025) \n
- Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (2025) \n
- Self-rewarding correction for mathematical reasoning (2025) \n
- Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search (2025) \n
- S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning (2025) \n
- Self-Training Elicits Concise Reasoning in Large Language Models (2025) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
We made a deep dive video for this paper: https://www.youtube.com/watch?v=b3UEEbvmZHM. Happy learning together!
Papers
arxiv:2503.01307
Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs
Published on Mar 3, 2025
Abstract
Research reveals that effective self-improvement in language models is linked to their ability to perform specific reasoning behaviors, such as verification and backward chaining, which can be primed through appropriate training.
AI-generated summary
Test-time inference has emerged as a powerful paradigm for enabling language models to ``think'' longer and more carefully about complex challenges, much like skilled human experts. While reinforcement learning (RL) can drive self-improvement in language models on verifiable tasks, some models exhibit substantial gains while others quickly plateau. For instance, we find that Qwen-2.5-3B far exceeds Llama-3.2-3B under identical RL training for the game of Countdown. This discrepancy raises a critical question: what intrinsic properties enable effective self-improvement? We introduce a framework to investigate this question by analyzing four key cognitive behaviors -- verification, backtracking, subgoal setting, and backward chaining -- that both expert human problem solvers and successful language models employ. Our study reveals that Qwen naturally exhibits these reasoning behaviors, whereas Llama initially lacks them. In systematic experimentation with controlled behavioral datasets, we find that priming Llama with examples containing these reasoning behaviors enables substantial improvements during RL, matching or exceeding Qwen's performance. Importantly, the presence of reasoning behaviors, rather than correctness of answers, proves to be the critical factor -- models primed with incorrect solutions containing proper reasoning patterns achieve comparable performance to those trained on correct solutions. Finally, leveraging continued pretraining with OpenWebMath data, filtered to amplify reasoning behaviors, enables the Llama model to match Qwen's self-improvement trajectory. Our findings establish a fundamental relationship between initial reasoning behaviors and the capacity for improvement, explaining why some language models effectively utilize additional computation while others plateau.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- LADDER: Self-Improving LLMs Through Recursive Problem Decomposition (2025)
- AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO (2025)
- Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (2025)
- Self-rewarding correction for mathematical reasoning (2025)
- Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search (2025)
- S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning (2025)
- Self-Training Elicits Concise Reasoning in Large Language Models (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
We made a deep dive video for this paper: https://www.youtube.com/watch?v=b3UEEbvmZHM. Happy learning together!
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2503.01307 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2503.01307 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2503.01307 in a Space README.md to link it from this page.