https://github.com/LINs-lab/LIE
🔗 Paper: https://arxiv.org/abs/2602.11748

\n","updatedAt":"2026-02-13T06:16:21.491Z","author":{"_id":"644915c5e87a77e872e61350","avatarUrl":"/avatars/46ba7bdf04ad4c1b0ad79155010dc684.svg","fullname":"Luo","name":"ramiroluo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6584978103637695},"editors":["ramiroluo"],"editorAvatarUrls":["/avatars/46ba7bdf04ad4c1b0ad79155010dc684.svg"],"reactions":[],"isReport":false}},{"id":"698f223cdf22f6e2af5c0680","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2026-02-13T13:08:12.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Thickening-to-Thinning: Reward Shaping via Human-Inspired Learning Dynamics for LLM Reasoning](https://huggingface.co/papers/2602.04265) (2026)\n* [Beyond Mode Elicitation: Diversity-Preserving Reinforcement Learning via Latent Diffusion Reasoner](https://huggingface.co/papers/2602.01705) (2026)\n* [Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models](https://huggingface.co/papers/2601.03969) (2026)\n* [Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning](https://huggingface.co/papers/2602.02099) (2026)\n* [Restoring Exploration after Post-Training: Latent Exploration Decoding for Large Reasoning Models](https://huggingface.co/papers/2602.01698) (2026)\n* [Diversity or Precision? A Deep Dive into Next Token Prediction](https://huggingface.co/papers/2512.22955) (2025)\n* [Step Potential Advantage Estimation: Harnessing Intermediate Confidence and Correctness for Efficient Mathematical Reasoning](https://huggingface.co/papers/2601.03823) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2026-02-13T13:08:12.292Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7137289643287659},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[{"reaction":"👍","users":["sehun"],"count":1}],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2602.11748","authors":[{"_id":"698ec188cace060ff123aed7","name":"Futing Wang","hidden":false},{"_id":"698ec188cace060ff123aed8","name":"Jianhao Yan","hidden":false},{"_id":"698ec188cace060ff123aed9","name":"Yun Luo","hidden":false},{"_id":"698ec188cace060ff123aeda","name":"Ganqu Cui","hidden":false},{"_id":"698ec188cace060ff123aedb","name":"Zhi Wang","hidden":false},{"_id":"698ec188cace060ff123aedc","name":"Xiaoye Qu","hidden":false},{"_id":"698ec188cace060ff123aedd","name":"Yue Zhang","hidden":false},{"_id":"698ec188cace060ff123aede","name":"Yu Cheng","hidden":false},{"_id":"698ec188cace060ff123aedf","name":"Tao Lin","hidden":false}],"publishedAt":"2026-02-12T09:24:32.000Z","submittedOnDailyAt":"2026-02-13T03:46:21.481Z","title":"Think Longer to Explore Deeper: Learn to Explore In-Context via Length-Incentivized Reinforcement Learning","submittedOnDailyBy":{"_id":"644915c5e87a77e872e61350","avatarUrl":"/avatars/46ba7bdf04ad4c1b0ad79155010dc684.svg","isPro":false,"fullname":"Luo","user":"ramiroluo","type":"user"},"summary":"Achieving effective test-time scaling requires models to engage in In-Context Exploration -- the intrinsic ability to generate, verify, and refine multiple reasoning hypotheses within a single continuous context.\n Grounded in State Coverage theory, our analysis identifies a critical bottleneck to enabling this capability: while broader state coverage requires longer reasoning trajectories, the probability of sampling such sequences decays exponentially during autoregressive generation, a phenomenon we term the ``Shallow Exploration Trap''.\n To bridge this gap, we propose Length-Incentivized Exploration(\\method).\n This simple yet effective recipe explicitly encourages models to explore more via a length-based reward coupled with a redundancy penalty, thereby maximizing state coverage in two-step manner.\n Comprehensive experiments across different models (Qwen3, Llama) demonstrate that \\method effectively incentivize in-context exploration.\n As a result, our method achieves an average improvement of 4.4\\% on in-domain tasks and a 2.7\\% gain on out-of-domain benchmarks.","upvotes":30,"discussionId":"698ec188cace060ff123aee0","githubRepo":"https://github.com/LINs-lab/LIE","githubRepoAddedBy":"user","ai_summary":"Models require in-context exploration capabilities to scale effectively at test time, but autoregressive generation faces exponential decay in sampling long sequences, which is addressed by a length-incentivized exploration method that improves performance on both in-domain and out-of-domain tasks.","ai_keywords":["In-Context Exploration","State Coverage theory","Shallow Exploration Trap","Length-Incentivized Exploration","autoregressive generation","redundancy penalty","state coverage"],"githubStars":15,"organization":{"_id":"643cb10025681c3afab0f1a6","name":"Westlake-University","fullname":"Westlake University","avatar":"https://cdn-uploads.huggingface.co/production/uploads/6437eca0819f3ab20d162e14/SQRCHUyjPRyqdtV3um42X.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"641a7e524182690729c8c846","avatarUrl":"/avatars/769e950174c92f76d1bc25492de9731a.svg","isPro":false,"fullname":"Xiaoye Qu","user":"XiaoYee","type":"user"},{"_id":"6086838b19137b3a6ba760e7","avatarUrl":"/avatars/d63eea3e39b22c6e65b82c28192696f1.svg","isPro":false,"fullname":"Jianhao Yan","user":"Elliott","type":"user"},{"_id":"644915c5e87a77e872e61350","avatarUrl":"/avatars/46ba7bdf04ad4c1b0ad79155010dc684.svg","isPro":false,"fullname":"Luo","user":"ramiroluo","type":"user"},{"_id":"691b0f528411a45dc9ee9de8","avatarUrl":"/avatars/261c28f7e616a8482970f50c1f8919fd.svg","isPro":false,"fullname":"Haiyuan Wan","user":"HY-Wan","type":"user"},{"_id":"645b4819f9d4ec91fdd54852","avatarUrl":"/avatars/e12efb8e030688a0afcc72176b453fb3.svg","isPro":false,"fullname":"Jiawei Gu","user":"kuvvi","type":"user"},{"_id":"62495cb96ee7ee6b646db130","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/62495cb96ee7ee6b646db130/UwBXmvcMq7LMvBWUw0xo3.jpeg","isPro":false,"fullname":"Runzhe Zhan","user":"rzzhan","type":"user"},{"_id":"67247adb73d1eb17b6bfd27c","avatarUrl":"/avatars/57bdbb7362f9854c87dd0a71ae071652.svg","isPro":false,"fullname":"Zefeng He","user":"yhx12","type":"user"},{"_id":"63f3502a520c14618925825a","avatarUrl":"/avatars/e986a2a6625e7be6890616a417f908d2.svg","isPro":false,"fullname":"Yafu Li","user":"yaful","type":"user"},{"_id":"671b3d82101c0d9f78749256","avatarUrl":"/avatars/902c1006280c6552885b0b946cd2bb16.svg","isPro":false,"fullname":"Zhanchao Zhou","user":"Zcchill","type":"user"},{"_id":"6418228b83957c4eaaad4d01","avatarUrl":"/avatars/b6af01d09bba5d5ba7bf4a62914ca468.svg","isPro":false,"fullname":"wang","user":"astrid01052","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"681a5957e46aef487017a935","avatarUrl":"/avatars/f683ed5b35bda86de5abc8cf27d6b256.svg","isPro":false,"fullname":"PeterPP","user":"ZhSh1230","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0,"organization":{"_id":"643cb10025681c3afab0f1a6","name":"Westlake-University","fullname":"Westlake University","avatar":"https://cdn-uploads.huggingface.co/production/uploads/6437eca0819f3ab20d162e14/SQRCHUyjPRyqdtV3um42X.jpeg"}}">

Papers

arxiv:2602.11748

Think Longer to Explore Deeper: Learn to Explore In-Context via Length-Incentivized Reinforcement Learning

Published on Feb 12

· Submitted by

Luo on Feb 13

Westlake University

Upvote

Authors:

Abstract

Models require in-context exploration capabilities to scale effectively at test time, but autoregressive generation faces exponential decay in sampling long sequences, which is addressed by a length-incentivized exploration method that improves performance on both in-domain and out-of-domain tasks.

AI-generated summary

Achieving effective test-time scaling requires models to engage in In-Context Exploration -- the intrinsic ability to generate, verify, and refine multiple reasoning hypotheses within a single continuous context. Grounded in State Coverage theory, our analysis identifies a critical bottleneck to enabling this capability: while broader state coverage requires longer reasoning trajectories, the probability of sampling such sequences decays exponentially during autoregressive generation, a phenomenon we term the ``Shallow Exploration Trap''. To bridge this gap, we propose Length-Incentivized Exploration(\method). This simple yet effective recipe explicitly encourages models to explore more via a length-based reward coupled with a redundancy penalty, thereby maximizing state coverage in two-step manner. Comprehensive experiments across different models (Qwen3, Llama) demonstrate that \method effectively incentivize in-context exploration. As a result, our method achieves an average improvement of 4.4\% on in-domain tasks and a 2.7\% gain on out-of-domain benchmarks.