\n\t\t\n\t\n\t\n\t\tAya 23: The Next Leap in Multilingual Language Models\n\t\n\n

\n\n

\n\t\n\t\t\n\t\n\t\n\t\tLinks 🔗:\n\t\n

👉 Subscribe: https://www.youtube.com/@Arxflix
👉 Twitter: https://x.com/arxflix
👉 LMNT (Partner): https://lmnt.com/

\n","updatedAt":"2024-06-09T05:08:21.332Z","author":{"_id":"6186ddf6a7717cb375090c01","avatarUrl":"/avatars/716b6a7d1094c8036b2a8a7b9063e8aa.svg","fullname":"Julien BLANCHON","name":"blanchon","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":176,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5411456227302551},"editors":["blanchon"],"editorAvatarUrls":["/avatars/716b6a7d1094c8036b2a8a7b9063e8aa.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2405.15032","authors":[{"_id":"6653ed0583fac8ccdc09c6fd","user":{"_id":"633e9b78acd351fc37d69560","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633e9b78acd351fc37d69560/rCk8z-C4oOlrksp-QE2lV.jpeg","isPro":false,"fullname":"Viraat Aryabumi","user":"viraat","type":"user"},"name":"Viraat Aryabumi","status":"extracted_confirmed","statusLastChangedAt":"2024-05-27T04:13:50.260Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c6fe","user":{"_id":"65c581dfc3fa039f843991f6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65c581dfc3fa039f843991f6/O4ywGgQQrzps5JYeR3o0y.jpeg","isPro":false,"fullname":"John Dang","user":"johndang-cohere","type":"user"},"name":"John Dang","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:49:56.573Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c6ff","name":"Dwarak Talupuru","hidden":false},{"_id":"6653ed0583fac8ccdc09c700","user":{"_id":"63adeb79a31e9ea8298ad4ac","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63adeb79a31e9ea8298ad4ac/a_kEJXPLkfaHDUN3XcTYA.jpeg","isPro":false,"fullname":"Saurabh Dash","user":"saurabhdash","type":"user"},"name":"Saurabh Dash","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:50:22.027Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c701","user":{"_id":"64674f1a8334813a7ae33023","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64674f1a8334813a7ae33023/VjxQYzaXW7zlH7ktPBsEX.jpeg","isPro":false,"fullname":"David Cairuz","user":"davidcairuz","type":"user"},"name":"David Cairuz","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:52:57.723Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c702","name":"Hangyu Lin","hidden":false},{"_id":"6653ed0583fac8ccdc09c703","user":{"_id":"65e8e96868cd34406483a639","avatarUrl":"/avatars/77fbe9ebd1fccd2e15abb403e2139652.svg","isPro":false,"fullname":"Bharat venkitesh","user":"bharatv007","type":"user"},"name":"Bharat Venkitesh","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:53:51.118Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c704","user":{"_id":"6351689040dffad651f0d26b","avatarUrl":"/avatars/65808f9bf9243c7e8bf7130cdf86166b.svg","isPro":false,"fullname":"Madeline Smith","user":"madelzzz","type":"user"},"name":"Madeline Smith","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:53:58.583Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c705","user":{"_id":"63bc6313d8d676a229a2b9c8","avatarUrl":"/avatars/d11fa232a03d5d0ac88c177ef1cf187d.svg","isPro":false,"fullname":"Kelly Marchisio","user":"kellymarchisio","type":"user"},"name":"Kelly Marchisio","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:54:17.273Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c706","user":{"_id":"624575789dc26c59bae6a19e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1648719221694-noauth.jpeg","isPro":false,"fullname":"Sebastian Ruder","user":"ruder","type":"user"},"name":"Sebastian Ruder","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:54:26.270Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c707","user":{"_id":"60d370b613f774189902f556","avatarUrl":"/avatars/717bd7955f30eb76310d2074fd51afce.svg","isPro":false,"fullname":"Acyr Locatelli","user":"acyrl","type":"user"},"name":"Acyr Locatelli","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:54:35.114Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c708","user":{"_id":"6544e43b12da508864c38f96","avatarUrl":"/avatars/76f0cd55b4bf9c03d2686e146c6f795f.svg","isPro":false,"fullname":"Julia Kreutzer","user":"JuliaKreutzerCohere","type":"user"},"name":"Julia Kreutzer","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:54:49.294Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c709","user":{"_id":"6402473006c715b934011e0d","avatarUrl":"/avatars/e312516c1e0edbf65418b5285b3164d4.svg","isPro":false,"fullname":"nick frosst","user":"nickfrosst","type":"user"},"name":"Nick Frosst","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:54:57.514Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c70a","name":"Phil Blunsom","hidden":false},{"_id":"6653ed0583fac8ccdc09c70b","user":{"_id":"6441042d5d600fb0951a5f99","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6441042d5d600fb0951a5f99/4CbOaYcEz99BtVAQvnGTn.jpeg","isPro":false,"fullname":"Marzieh Fadaee","user":"MarziehFadaee","type":"user"},"name":"Marzieh Fadaee","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:55:08.278Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c70c","user":{"_id":"60d35d7ad7b174177faabd5b","avatarUrl":"/avatars/4e5403b9d4a845a2d21e8217dc3c16d2.svg","isPro":false,"fullname":"Ahmet Üstün","user":"ahmetu","type":"user"},"name":"Ahmet Üstün","status":"admin_assigned","statusLastChangedAt":"2024-05-27T08:55:16.611Z","hidden":false},{"_id":"6653ed0583fac8ccdc09c70d","user":{"_id":"63434eb76f59b79da07dbddf","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63434eb76f59b79da07dbddf/BEwmVjqPNYlqmutXG0G6e.jpeg","isPro":false,"fullname":"Sara Hooker","user":"sarahooker","type":"user"},"name":"Sara Hooker","status":"extracted_confirmed","statusLastChangedAt":"2024-05-27T03:43:46.321Z","hidden":false}],"publishedAt":"2024-05-23T20:10:38.000Z","submittedOnDailyAt":"2024-05-27T00:46:38.679Z","title":"Aya 23: Open Weight Releases to Further Multilingual Progress","submittedOnDailyBy":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","isPro":false,"fullname":"AK","user":"akhaliq","type":"user"},"summary":"This technical report introduces Aya 23, a family of multilingual language\nmodels. Aya 23 builds on the recent release of the Aya model (\\\"Ust\\\"un et al.,\n2024), focusing on pairing a highly performant pre-trained model with the\nrecently released Aya collection (Singh et al., 2024). The result is a powerful\nmultilingual large language model serving 23 languages, expanding state-of-art\nlanguage modeling capabilities to approximately half of the world's population.\nThe Aya model covered 101 languages whereas Aya 23 is an experiment in depth vs\nbreadth, exploring the impact of allocating more capacity to fewer languages\nthat are included during pre-training. Aya 23 outperforms both previous\nmassively multilingual models like Aya 101 for the languages it covers, as well\nas widely used models like Gemma, Mistral and Mixtral on an extensive range of\ndiscriminative and generative tasks. We release the open weights for both the\n8B and 35B models as part of our continued commitment for expanding access to\nmultilingual progress.","upvotes":32,"discussionId":"6653ed0683fac8ccdc09c79b","ai_summary":"Aya 23 is a powerful multilingual language model for 23 languages, outperforming previous models on a wide range of tasks.","ai_keywords":["multilingual language models","pre-trained model","Aya model","Aya collection","state-of-art language modeling","Gemma","Mistral","Mixtral","discriminative tasks","generative tasks"]},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"64747f7e33192631bacd8831","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64747f7e33192631bacd8831/dstkZJ4sHJSeqLesV5cOC.jpeg","isPro":false,"fullname":"Taufiq Dwi Purnomo","user":"taufiqdp","type":"user"},{"_id":"648eb1eb59c4e5c87dc116e0","avatarUrl":"/avatars/c636cea39c2c0937f01398c94ead5dad.svg","isPro":false,"fullname":"fdsqefsgergd","user":"T-representer","type":"user"},{"_id":"60cb51422424f83d03935470","avatarUrl":"/avatars/f071939cfd6fdd08a8399bf9a7a96fa2.svg","isPro":false,"fullname":"Oleksiy Syvokon","user":"osyvokon","type":"user"},{"_id":"62a4ac6fd83c3facafa50892","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/62a4ac6fd83c3facafa50892/qFpobw9B5XaLZvwn0XbmB.jpeg","isPro":false,"fullname":"Mohammed Brıman","user":"mohammedbriman","type":"user"},{"_id":"63a369d98c0c89dcae3b8329","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63a369d98c0c89dcae3b8329/AiH2zjy1cnt9OADAAZMLD.jpeg","isPro":false,"fullname":"Adina Yakefu","user":"AdinaY","type":"user"},{"_id":"6032802e1f993496bc14d9e3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png","isPro":false,"fullname":"Omar Sanseviero","user":"osanseviero","type":"user"},{"_id":"647f3aece9c81260ff86c843","avatarUrl":"/avatars/04489491bf435e19bfdcbdfd80204eb1.svg","isPro":false,"fullname":"Chao Zhou","user":"ASHIDAKA","type":"user"},{"_id":"63ddc7b80f6d2d6c3efe3600","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ddc7b80f6d2d6c3efe3600/RX5q9T80Jl3tn6z03ls0l.jpeg","isPro":false,"fullname":"J","user":"dashfunnydashdash","type":"user"},{"_id":"655ac762cb17ec19ef82719b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/655ac762cb17ec19ef82719b/1kDncYrGLYS_2SR8cNdAL.png","isPro":false,"fullname":"Welcome to matlok","user":"matlok","type":"user"},{"_id":"64b778fbfdb702b3d8664673","avatarUrl":"/avatars/1f599ed6f11c0af1336f6addb3265fc4.svg","isPro":false,"fullname":"DamianImrich","user":"imrichD","type":"user"},{"_id":"60ba519750effef3a58beac3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1660312628256-60ba519750effef3a58beac3.png","isPro":false,"fullname":"Matthew Carrigan","user":"Rocketknight1","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

Papers

arxiv:2405.15032

Aya 23: Open Weight Releases to Further Multilingual Progress

Published on May 23, 2024

· Submitted by

AK on May 27, 2024

Upvote

Authors:

Viraat Aryabumi ,

John Dang ,

Saurabh Dash ,

David Cairuz ,

Bharat Venkitesh ,

Madeline Smith ,

Kelly Marchisio ,

Sebastian Ruder ,

Acyr Locatelli ,

Julia Kreutzer ,

Nick Frosst ,

Marzieh Fadaee ,

Ahmet Üstün ,

Sara Hooker

Abstract

Aya 23 is a powerful multilingual language model for 23 languages, outperforming previous models on a wide range of tasks.

AI-generated summary

This technical report introduces Aya 23, a family of multilingual language models. Aya 23 builds on the recent release of the Aya model (\"Ust\"un et al., 2024), focusing on pairing a highly performant pre-trained model with the recently released Aya collection (Singh et al., 2024). The result is a powerful multilingual large language model serving 23 languages, expanding state-of-art language modeling capabilities to approximately half of the world's population. The Aya model covered 101 languages whereas Aya 23 is an experiment in depth vs breadth, exploring the impact of allocating more capacity to fewer languages that are included during pre-training. Aya 23 outperforms both previous massively multilingual models like Aya 101 for the languages it covers, as well as widely used models like Gemma, Mistral and Mixtral on an extensive range of discriminative and generative tasks. We release the open weights for both the 8B and 35B models as part of our continued commitment for expanding access to multilingual progress.