https://github.com/facebookresearch/swe-rl

\n","updatedAt":"2025-02-26T03:03:08.526Z","author":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","fullname":"AK","name":"akhaliq","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":9179,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.535057783126831},"editors":["akhaliq"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg"],"reactions":[{"reaction":"👍","users":["inkasaras","lingming","sugatoray","dtanow"],"count":4}],"isReport":false}},{"id":"67beaccfa2efb58ea981a052","author":{"_id":"67189ee42dd03d8869b93735","avatarUrl":"/avatars/e9fb6ddbe4bb79ed2183df96c74e0fb8.svg","fullname":"Keaton Elvins","name":"keatone","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false},"createdAt":"2025-02-26T05:55:27.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Will the training code be provided as well? There's a big ask for good GRPO multi-node infra in the OSS community right now.","html":"

Will the training code be provided as well? There's a big ask for good GRPO multi-node infra in the OSS community right now.

\n","updatedAt":"2025-02-26T05:55:27.883Z","author":{"_id":"67189ee42dd03d8869b93735","avatarUrl":"/avatars/e9fb6ddbe4bb79ed2183df96c74e0fb8.svg","fullname":"Keaton Elvins","name":"keatone","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9592596292495728},"editors":["keatone"],"editorAvatarUrls":["/avatars/e9fb6ddbe4bb79ed2183df96c74e0fb8.svg"],"reactions":[{"reaction":"👍","users":["inkasaras","maveriq","amxv","askender"],"count":4}],"isReport":false}},{"id":"67befaa4534941b5fe5ca82f","author":{"_id":"63d83e6c6ac3104e50c8dcd6","avatarUrl":"/avatars/2a4b82aa3fcd0617387c6504da17b071.svg","fullname":"Boris Yangel","name":"hr0nix","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false},"createdAt":"2025-02-26T11:27:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This paper completely failed to mention this work that lead to a similar result on swebench-verified a few months ago: https://nebius.com/blog/posts/training-and-search-for-software-engineering-agents","html":"

This paper completely failed to mention this work that lead to a similar result on swebench-verified a few months ago: https://nebius.com/blog/posts/training-and-search-for-software-engineering-agents

\n","updatedAt":"2025-02-26T11:27:32.804Z","author":{"_id":"63d83e6c6ac3104e50c8dcd6","avatarUrl":"/avatars/2a4b82aa3fcd0617387c6504da17b071.svg","fullname":"Boris Yangel","name":"hr0nix","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9320974946022034},"editors":["hr0nix"],"editorAvatarUrls":["/avatars/2a4b82aa3fcd0617387c6504da17b071.svg"],"reactions":[],"isReport":false}},{"id":"67bfc17f9e4a5ffaa49ba49e","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-02-27T01:35:59.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://huggingface.co/papers/2501.12948) (2025)\n* [Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning](https://huggingface.co/papers/2502.14768) (2025)\n* [Challenges in Ensuring AI Safety in DeepSeek-R1 Models: The Shortcomings of Reinforcement Learning Strategies](https://huggingface.co/papers/2501.17030) (2025)\n* [The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning](https://huggingface.co/papers/2502.15214) (2025)\n* [Training Software Engineering Agents and Verifiers with SWE-Gym](https://huggingface.co/papers/2412.21139) (2024)\n* [Rule-Bottleneck Reinforcement Learning: Joint Explanation and Decision Optimization for Resource Allocation with Language Agents](https://huggingface.co/papers/2502.10732) (2025)\n* [EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning](https://huggingface.co/papers/2502.12486) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-02-27T01:35:59.436Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7540298700332642},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}},{"id":"67c34ae163f1a3ac3e9483f7","author":{"_id":"665edfcf2b842ec980842bd4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/665edfcf2b842ec980842bd4/GJHNPJ3ULIMEMq6VGxZaI.png","fullname":"AI Papers Academy","name":"aipapersacademy","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false},"createdAt":"2025-03-01T17:58:57.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"A written & video summary can be found here - https://aipapersacademy.com/swe-rl/","html":"

A written & video summary can be found here - https://aipapersacademy.com/swe-rl/

\n","updatedAt":"2025-03-01T17:58:57.780Z","author":{"_id":"665edfcf2b842ec980842bd4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/665edfcf2b842ec980842bd4/GJHNPJ3ULIMEMq6VGxZaI.png","fullname":"AI Papers Academy","name":"aipapersacademy","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9133083820343018},"editors":["aipapersacademy"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/665edfcf2b842ec980842bd4/GJHNPJ3ULIMEMq6VGxZaI.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2502.18449","authors":[{"_id":"67be845a8a5a80542314579f","user":{"_id":"632a176259950c1d279d5ea7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/632a176259950c1d279d5ea7/xsSGhBXalt9RaKzSKY8uk.jpeg","isPro":false,"fullname":"Yuxiang Wei","user":"yuxiang630","type":"user"},"name":"Yuxiang Wei","status":"admin_assigned","statusLastChangedAt":"2025-02-26T08:50:44.837Z","hidden":false},{"_id":"67be845a8a5a8054231457a0","name":"Olivier Duchenne","hidden":false},{"_id":"67be845a8a5a8054231457a1","user":{"_id":"6481e0ac50b759c75d5fdad0","avatarUrl":"/avatars/49f08d989ca505ae01bce5578a94f6fe.svg","isPro":false,"fullname":"Jade Copet","user":"JadeCopet","type":"user"},"name":"Jade Copet","status":"admin_assigned","statusLastChangedAt":"2025-02-26T08:50:58.290Z","hidden":false},{"_id":"67be845a8a5a8054231457a2","name":"Quentin Carbonneaux","hidden":false},{"_id":"67be845a8a5a8054231457a3","user":{"_id":"656f473c14fa8cfccd14559e","avatarUrl":"/avatars/8f4fef3d835a7a11c2ab66dbf04f3424.svg","isPro":false,"fullname":"Lingming Zhang","user":"lingming","type":"user"},"name":"Lingming Zhang","status":"admin_assigned","statusLastChangedAt":"2025-02-26T08:51:10.640Z","hidden":false},{"_id":"67be845a8a5a8054231457a4","name":"Daniel Fried","hidden":false},{"_id":"67be845a8a5a8054231457a5","user":{"_id":"630eac7931970d1cd4fbacf2","avatarUrl":"/avatars/b7ccbddfa745db854dc342be1327cd53.svg","isPro":false,"fullname":"Gabriel Synnaeve","user":"gsynnaeve","type":"user"},"name":"Gabriel Synnaeve","status":"admin_assigned","statusLastChangedAt":"2025-02-26T08:51:21.641Z","hidden":false},{"_id":"67be845a8a5a8054231457a6","user":{"_id":"6597e5a6420dcc68501a69e9","avatarUrl":"/avatars/da48b13e07c367ecd5c891abfd6c3ded.svg","isPro":false,"fullname":"Rishabh Singh","user":"RishabhSingh021","type":"user"},"name":"Rishabh Singh","status":"admin_assigned","statusLastChangedAt":"2025-02-26T08:51:28.321Z","hidden":false},{"_id":"67be845a8a5a8054231457a7","name":"Sida I. Wang","hidden":false}],"publishedAt":"2025-02-25T18:45:04.000Z","submittedOnDailyAt":"2025-02-26T00:33:08.515Z","title":"SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open\n Software Evolution","submittedOnDailyBy":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","isPro":false,"fullname":"AK","user":"akhaliq","type":"user"},"summary":"The recent DeepSeek-R1 release has demonstrated the immense potential of\nreinforcement learning (RL) in enhancing the general reasoning capabilities of\nlarge language models (LLMs). While DeepSeek-R1 and other follow-up work\nprimarily focus on applying RL to competitive coding and math problems, this\npaper introduces SWE-RL, the first approach to scale RL-based LLM reasoning for\nreal-world software engineering. Leveraging a lightweight rule-based reward\n(e.g., the similarity score between ground-truth and LLM-generated solutions),\nSWE-RL enables LLMs to autonomously recover a developer's reasoning processes\nand solutions by learning from extensive open-source software evolution data --\nthe record of a software's entire lifecycle, including its code snapshots, code\nchanges, and events such as issues and pull requests. Trained on top of Llama\n3, our resulting reasoning model, Llama3-SWE-RL-70B, achieves a 41.0% solve\nrate on SWE-bench Verified -- a human-verified collection of real-world GitHub\nissues. To our knowledge, this is the best performance reported for\nmedium-sized (<100B) LLMs to date, even comparable to leading proprietary LLMs\nlike GPT-4o. Surprisingly, despite performing RL solely on software evolution\ndata, Llama3-SWE-RL has even emerged with generalized reasoning skills. For\nexample, it shows improved results on five out-of-domain tasks, namely,\nfunction coding, library use, code reasoning, mathematics, and general language\nunderstanding, whereas a supervised-finetuning baseline even leads to\nperformance degradation on average. Overall, SWE-RL opens up a new direction to\nimprove the reasoning capabilities of LLMs through reinforcement learning on\nmassive software engineering data.","upvotes":75,"discussionId":"67be845b8a5a8054231457d6","githubRepo":"https://github.com/facebookresearch/swe-rl","githubRepoAddedBy":"auto","ai_summary":"SWE-RL leverages reinforcement learning to enhance large language models' software engineering reasoning, achieving state-of-the-art performance and generalized reasoning skills by learning from extensive open-source software evolution data.","ai_keywords":["reinforcement learning","large language models","RL-based LLM reasoning","lightweight rule-based reward","open-source software evolution data","SWE-bench Verified","solve rate","Llama 3","function coding","library use","code reasoning","mathematics","general language understanding"],"githubStars":677},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"61ca43f194afcb20ceb764d7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1640645587885-noauth.jpeg","isPro":false,"fullname":"Raúl Garrido","user":"happybydefault","type":"user"},{"_id":"632a176259950c1d279d5ea7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/632a176259950c1d279d5ea7/xsSGhBXalt9RaKzSKY8uk.jpeg","isPro":false,"fullname":"Yuxiang Wei","user":"yuxiang630","type":"user"},{"_id":"644b584a9279988e0cbeb664","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/644b584a9279988e0cbeb664/fhWCI_Q26tTruhdFkjejw.jpeg","isPro":false,"fullname":"Jiawei Liu","user":"ganler","type":"user"},{"_id":"64d98ef7a4839890b25eb78b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64d98ef7a4839890b25eb78b/215-CSVLl81z6CAq0ECWU.jpeg","isPro":true,"fullname":"Fangyuan Yu","user":"Ksgk-fy","type":"user"},{"_id":"65df33cc172353c169dceaaf","avatarUrl":"/avatars/938f4fcbea0305279ca9ec37ce7eaa65.svg","isPro":false,"fullname":"Ubec","user":"hrw","type":"user"},{"_id":"66f612b934b8ac9ffa44f084","avatarUrl":"/avatars/6836c122e19c66c90f1673f28b30d7f0.svg","isPro":false,"fullname":"Tang","user":"tommysally","type":"user"},{"_id":"60078446e55258e41786a959","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/60078446e55258e41786a959/UGPCE4YqG9BVMSf0YauxL.png","isPro":false,"fullname":"Motoki Wu","user":"tokestermw","type":"user"},{"_id":"640ae2aeb871d7117d5f8135","avatarUrl":"/avatars/af0820f65b8db3180075bbdcb15624e0.svg","isPro":false,"fullname":"Trangle Heshvp","user":"Trangle","type":"user"},{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","isPro":false,"fullname":"Asankhaya Sharma","user":"codelion","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"6270324ebecab9e2dcf245de","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6270324ebecab9e2dcf245de/cMbtWSasyNlYc9hvsEEzt.jpeg","isPro":false,"fullname":"Kye Gomez","user":"kye","type":"user"},{"_id":"650be23ec4e52db6a4db63ef","avatarUrl":"/avatars/03af548029b38bee49ec295fefe74f9a.svg","isPro":false,"fullname":"Haoling Li","user":"Ringo1110","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":1}">

Papers

arxiv:2502.18449

SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution

Published on Feb 25, 2025

· Submitted by

AK on Feb 26, 2025

#1 Paper of the day

Upvote

Authors:

Yuxiang Wei ,

Jade Copet ,

Lingming Zhang ,

Gabriel Synnaeve ,

Rishabh Singh ,

Abstract

SWE-RL leverages reinforcement learning to enhance large language models' software engineering reasoning, achieving state-of-the-art performance and generalized reasoning skills by learning from extensive open-source software evolution data.

AI-generated summary

The recent DeepSeek-R1 release has demonstrated the immense potential of reinforcement learning (RL) in enhancing the general reasoning capabilities of large language models (LLMs). While DeepSeek-R1 and other follow-up work primarily focus on applying RL to competitive coding and math problems, this paper introduces SWE-RL, the first approach to scale RL-based LLM reasoning for real-world software engineering. Leveraging a lightweight rule-based reward (e.g., the similarity score between ground-truth and LLM-generated solutions), SWE-RL enables LLMs to autonomously recover a developer's reasoning processes and solutions by learning from extensive open-source software evolution data -- the record of a software's entire lifecycle, including its code snapshots, code changes, and events such as issues and pull requests. Trained on top of Llama 3, our resulting reasoning model, Llama3-SWE-RL-70B, achieves a 41.0% solve rate on SWE-bench Verified -- a human-verified collection of real-world GitHub issues. To our knowledge, this is the best performance reported for medium-sized (<100B) LLMs to date, even comparable to leading proprietary LLMs like GPT-4o. Surprisingly, despite performing RL solely on software evolution data, Llama3-SWE-RL has even emerged with generalized reasoning skills. For example, it shows improved results on five out-of-domain tasks, namely, function coding, library use, code reasoning, mathematics, and general language understanding, whereas a supervised-finetuning baseline even leads to performance degradation on average. Overall, SWE-RL opens up a new direction to improve the reasoning capabilities of LLMs through reinforcement learning on massive software engineering data.