\n","updatedAt":"2025-01-22T04:42:44.766Z","author":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","fullname":"AK","name":"akhaliq","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":9179,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.4335770010948181},"editors":["akhaliq"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg"],"reactions":[{"reaction":"🔥","users":["mkurman"],"count":1}],"isReport":false}},{"id":"67919c8e7a57fe99eb5edd64","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-01-23T01:34:06.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning](https://huggingface.co/papers/2412.15797) (2024)\n* [Towards Large Reasoning Models: A Survey on Scaling LLM Reasoning Capabilities](https://huggingface.co/papers/2501.09686) (2025)\n* [Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimization](https://huggingface.co/papers/2412.18279) (2024)\n* [REL: Working out is all you need](https://huggingface.co/papers/2412.04645) (2024)\n* [RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement](https://huggingface.co/papers/2412.12881) (2024)\n* [Semantic Exploration with Adaptive Gating for Efficient Problem Solving with Language Models](https://huggingface.co/papers/2501.05752) (2025)\n* [Offline Reinforcement Learning for LLM Multi-Step Reasoning](https://huggingface.co/papers/2412.16145) (2024)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

\n

The following papers were recommended by the Semantic Scholar API

\n

\n
• Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning (2024)
\n
• Towards Large Reasoning Models: A Survey on Scaling LLM Reasoning Capabilities (2025)
\n
• Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimization (2024)
\n
• REL: Working out is all you need (2024)
\n
• RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement (2024)
\n
• Semantic Exploration with Adaptive Gating for Efficient Problem Solving with Language Models (2025)
\n
• Offline Reinforcement Learning for LLM Multi-Step Reasoning (2024)
\n

\n

Please give a thumbs up to this comment if you found it helpful!

\n

If you want recommendations for any Paper on Hugging Face checkout this Space

\n

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-01-23T01:34:06.421Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7379752993583679},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[{"reaction":"👍","users":["JunaidMB"],"count":1}],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2501.11223","authors":[{"_id":"6790772b8d7df822f1fb4405","user":{"_id":"6613cdc25736e7f44f94df65","avatarUrl":"/avatars/f5b398b4da03d7833e20ddb3ce4211be.svg","isPro":false,"fullname":"Maciej Besta","user":"bestam","type":"user"},"name":"Maciej Besta","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:48:29.913Z","hidden":false},{"_id":"6790772b8d7df822f1fb4406","name":"Julia Barth","hidden":false},{"_id":"6790772b8d7df822f1fb4407","user":{"_id":"6712941f745634a65d916056","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/T5RZTWwb4FLZESYB5tu6m.png","isPro":false,"fullname":"Eric Schreiber","user":"eschreibe1","type":"user"},"name":"Eric Schreiber","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:48:45.137Z","hidden":false},{"_id":"6790772b8d7df822f1fb4408","user":{"_id":"64baf20d12d00c4589bb12f7","avatarUrl":"/avatars/2dbc10d369788c0ea048e1be97f0c5e6.svg","isPro":false,"fullname":"Ales Kubicek","user":"aleskubicek","type":"user"},"name":"Ales Kubicek","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:48:52.532Z","hidden":false},{"_id":"6790772b8d7df822f1fb4409","user":{"_id":"66fc32aa787008467cfe20cb","avatarUrl":"/avatars/b1c77ce8c7deaacf546a264467078673.svg","isPro":false,"fullname":"Afonso Catarino","user":"AfonsoC","type":"user"},"name":"Afonso Catarino","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:49:00.804Z","hidden":false},{"_id":"6790772b8d7df822f1fb440a","user":{"_id":"65a91420c46ce42ef5da96af","avatarUrl":"/avatars/a9fa3a7973d0030292b1e23172112a1e.svg","isPro":false,"fullname":"Robert Gerstenberger","user":"rgersten","type":"user"},"name":"Robert Gerstenberger","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:49:11.846Z","hidden":false},{"_id":"6790772b8d7df822f1fb440b","user":{"_id":"63f89b579f87cc3e645d96f9","avatarUrl":"/avatars/116be80348aa8f8be95b6ea774ecb65d.svg","isPro":false,"fullname":"Piotr Nyczyk","user":"pnyczyk","type":"user"},"name":"Piotr Nyczyk","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:49:18.866Z","hidden":false},{"_id":"6790772b8d7df822f1fb440c","name":"Patrick Iff","hidden":false},{"_id":"6790772b8d7df822f1fb440d","user":{"_id":"641c07c4bbdbe642a79914df","avatarUrl":"/avatars/5c89ee5f5cdf93e1d5ead6e46ae6c774.svg","isPro":false,"fullname":"Yueling Li","user":"liy140","type":"user"},"name":"Yueling Li","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:49:35.973Z","hidden":false},{"_id":"6790772b8d7df822f1fb440e","user":{"_id":"66af74ddd59c09785e02d1e0","avatarUrl":"/avatars/df0fec7228c35052b88cb5905bea809e.svg","isPro":false,"fullname":"Sam Houliston","user":"samhouliston","type":"user"},"name":"Sam Houliston","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:49:43.387Z","hidden":false},{"_id":"6790772b8d7df822f1fb440f","user":{"_id":"6535b041b66f4bf689267d91","avatarUrl":"/avatars/6ef4a183ff08ec3f3595f0866f3129ac.svg","isPro":false,"fullname":"Tomasz Sternal","user":"tsternal","type":"user"},"name":"Tomasz Sternal","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:49:51.528Z","hidden":false},{"_id":"6790772b8d7df822f1fb4410","name":"Marcin Copik","hidden":false},{"_id":"6790772b8d7df822f1fb4411","name":"Grzegorz Kwaśniewski","hidden":false},{"_id":"6790772b8d7df822f1fb4412","name":"Jürgen Müller","hidden":false},{"_id":"6790772b8d7df822f1fb4413","name":"Łukasz Flis","hidden":false},{"_id":"6790772b8d7df822f1fb4414","user":{"_id":"62f80cbc04de855c35e32fdb","avatarUrl":"/avatars/ab2600b96fe8b1787ad1eddaa45ad9ae.svg","isPro":false,"fullname":"Hannes Eberhard","user":"HannesE","type":"user"},"name":"Hannes Eberhard","status":"admin_assigned","statusLastChangedAt":"2025-01-22T15:50:47.492Z","hidden":false},{"_id":"6790772b8d7df822f1fb4415","name":"Hubert Niewiadomski","hidden":false},{"_id":"6790772b8d7df822f1fb4416","name":"Torsten Hoefler","hidden":false}],"publishedAt":"2025-01-20T02:16:19.000Z","submittedOnDailyAt":"2025-01-22T02:12:44.747Z","title":"Reasoning Language Models: A Blueprint","submittedOnDailyBy":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","isPro":false,"fullname":"AK","user":"akhaliq","type":"user"},"summary":"Reasoning language models (RLMs), also known as Large Reasoning Models\n(LRMs), such as OpenAI's o1 and o3, DeepSeek-V3, and Alibaba's QwQ, have\nredefined AI's problem-solving capabilities by extending large language models\n(LLMs) with advanced reasoning mechanisms. Yet, their high costs, proprietary\nnature, and complex architectures - uniquely combining Reinforcement Learning\n(RL), search heuristics, and LLMs - present accessibility and scalability\nchallenges. To address these, we propose a comprehensive blueprint that\norganizes RLM components into a modular framework, based on a survey and\nanalysis of all RLM works. This blueprint incorporates diverse reasoning\nstructures (chains, trees, graphs, and nested forms), reasoning strategies\n(e.g., Monte Carlo Tree Search, Beam Search), RL concepts (policy, value models\nand others), and supervision schemes (Output-Based and Process-Based\nSupervision). We also provide detailed mathematical formulations and\nalgorithmic specifications to simplify RLM implementation. By showing how\nschemes like LLaMA-Berry, QwQ, Journey Learning, and Graph of Thoughts fit as\nspecial cases, we demonstrate the blueprint's versatility and unifying\npotential. To illustrate its utility, we introduce x1, a modular implementation\nfor rapid RLM prototyping and experimentation. Using x1 and a literature\nreview, we provide key insights, such as multi-phase training for policy and\nvalue models, and the importance of familiar training distributions. Finally,\nwe outline how RLMs can integrate with a broader LLM ecosystem, including tools\nand databases. Our work demystifies RLM construction, democratizes advanced\nreasoning capabilities, and fosters innovation, aiming to mitigate the gap\nbetween \"rich AI\" and \"poor AI\" by lowering barriers to RLM development and\nexperimentation.","upvotes":33,"discussionId":"6790772d8d7df822f1fb4493","githubRepo":"https://github.com/spcl/x1","githubRepoAddedBy":"auto","ai_summary":"A comprehensive blueprint for modularizing reasoning language models (RLMs) to enhance accessibility and scalability, incorporating diverse reasoning structures, RL concepts, and supervision schemes.","ai_keywords":["reasoning language models","RLMs","Large Reasoning Models","LRMs","Reinforcement Learning","RL","Monte Carlo Tree Search","Beam Search","policy","value models","Output-Based Supervision","Process-Based Supervision","modular framework","RLM construction","x1","LLaMA-Berry","QwQ","Journey Learning","Graph of Thoughts","LLM ecosystem"],"githubStars":94},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64747f7e33192631bacd8831","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64747f7e33192631bacd8831/dstkZJ4sHJSeqLesV5cOC.jpeg","isPro":false,"fullname":"Taufiq Dwi Purnomo","user":"taufiqdp","type":"user"},{"_id":"63082bb7bc0a2a5ee2253523","avatarUrl":"/avatars/6cf8d12d16d15db1070fbea89b5b3967.svg","isPro":false,"fullname":"Kuo-Hsin Tu","user":"dapumptu","type":"user"},{"_id":"6527f92ca4c1d9d0aee7e766","avatarUrl":"/avatars/11e92f503e373a3523544ab0c086ba6e.svg","isPro":false,"fullname":"Aram Dovlatyan","user":"aramdov","type":"user"},{"_id":"64d86d66d7e30889c6a2e955","avatarUrl":"/avatars/222fcbe4af3bd897f260d019a54cfb6d.svg","isPro":false,"fullname":"ziyu zhu","user":"edward2021","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6560d75d6ff1b91e28e3cd7b","avatarUrl":"/avatars/bf205b47c71b197c56414ad1aaae3453.svg","isPro":false,"fullname":"js","user":"rldy","type":"user"},{"_id":"648eb1eb59c4e5c87dc116e0","avatarUrl":"/avatars/c636cea39c2c0937f01398c94ead5dad.svg","isPro":false,"fullname":"fdsqefsgergd","user":"T-representer","type":"user"},{"_id":"658e4851c0b1372b2e69aaaa","avatarUrl":"/avatars/ff073c7bb5229279e188e356da6481ae.svg","isPro":false,"fullname":"wang","user":"wangxbx","type":"user"},{"_id":"6776340dd3ceb4493fda0c6e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6776340dd3ceb4493fda0c6e/JzUAaFFPICKhZLgJR3pgP.png","isPro":false,"fullname":"Ruben Roy","user":"rubenroy","type":"user"},{"_id":"6366313c361a96184dbadff8","avatarUrl":"/avatars/9b83c5aedc02267d9596b19c20fbe593.svg","isPro":false,"fullname":"HAN JUNGU","user":"JUNGU","type":"user"},{"_id":"65059c6e14302b1d76960153","avatarUrl":"/avatars/7e03bf27f0c16a0e3f9fc475db32184c.svg","isPro":false,"fullname":"Jiwoong Park","user":"jwpark33","type":"user"},{"_id":"63732ebbbd81fae2b3aaf3fb","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1669551186189-63732ebbbd81fae2b3aaf3fb.jpeg","isPro":false,"fullname":"Knut Jägersberg","user":"KnutJaegersberg","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">