https://github.com/allenai/open-instruct\n","updatedAt":"2024-11-25T06:10:46.655Z","author":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","fullname":"AK","name":"akhaliq","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":9178,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5899043083190918},"editors":["akhaliq"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg"],"reactions":[{"reaction":"👍","users":["dashfunnydashdash","Azily","yizhongw"],"count":3}],"isReport":false}},{"id":"67452598334ae6264ba23d73","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2024-11-26T01:34:16.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [A Post-Training Enhanced Optimization Approach for Small Language Models](https://huggingface.co/papers/2411.02939) (2024)\n* [CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement](https://huggingface.co/papers/2411.05199) (2024)\n* [NeKo: Toward Post Recognition Generative Correction Large Language Models with Task-Oriented Experts](https://huggingface.co/papers/2411.05945) (2024)\n* [SAG: Style-Aligned Article Generation via Model Collaboration](https://huggingface.co/papers/2410.03137) (2024)\n* [How to Train Long-Context Language Models (Effectively)](https://huggingface.co/papers/2410.02660) (2024)\n* [SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe](https://huggingface.co/papers/2410.05248) (2024)\n* [Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpus](https://huggingface.co/papers/2410.14815) (2024)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- A Post-Training Enhanced Optimization Approach for Small Language Models (2024) \n
- CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement (2024) \n
- NeKo: Toward Post Recognition Generative Correction Large Language Models with Task-Oriented Experts (2024) \n
- SAG: Style-Aligned Article Generation via Model Collaboration (2024) \n
- How to Train Long-Context Language Models (Effectively) (2024) \n
- SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe (2024) \n
- Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpus (2024) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Abstract
T\"ULU 3, an open-source family of post-trained language models, introduces transparent training data, recipes, and advanced techniques to match or surpass proprietary models in performance.
AI-generated summary
Language model post-training is applied to refine behaviors and unlock new skills across a wide range of recent language models, but open recipes for applying these techniques lag behind proprietary ones. The underlying training data and recipes for post-training are simultaneously the most important pieces of the puzzle and the portion with the least transparency. To bridge this gap, we introduce T\"ULU 3, a family of fully-open state-of-the-art post-trained models, alongside its data, code, and training recipes, serving as a comprehensive guide for modern post-training techniques. T\"ULU 3, which builds on Llama 3.1 base models, achieves results surpassing the instruct versions of Llama 3.1, Qwen 2.5, Mistral, and even closed models such as GPT-4o-mini and Claude 3.5-Haiku. The training algorithms for our models include supervised finetuning (SFT), Direct Preference Optimization (DPO), and a novel method we call Reinforcement Learning with Verifiable Rewards (RLVR). With T\"ULU 3, we introduce a multi-task evaluation scheme for post-training recipes with development and unseen evaluations, standard benchmark implementations, and substantial decontamination of existing open datasets on said benchmarks. We conclude with analysis and discussion of training methods that did not reliably improve performance. In addition to the T\"ULU 3 model weights and demo, we release the complete recipe -- including datasets for diverse core skills, a robust toolkit for data curation and evaluation, the training code and infrastructure, and, most importantly, a detailed report for reproducing and further adapting the T\"ULU 3 approach to more domains.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- A Post-Training Enhanced Optimization Approach for Small Language Models (2024)
- CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement (2024)
- NeKo: Toward Post Recognition Generative Correction Large Language Models with Task-Oriented Experts (2024)
- SAG: Style-Aligned Article Generation via Model Collaboration (2024)
- How to Train Long-Context Language Models (Effectively) (2024)
- SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe (2024)
- Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpus (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend