Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Qwen2 Technical Report (2024)
GEB-1.3B: Open Lightweight Large Language Model (2024)
H2O-Danube3 Technical Report (2024)
LIONs: An Empirically Optimized Approach to Align Language Models (2024)
A Teacher Is Worth A Million Instructions (2024)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2024-08-03T01:32:14.105Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.747871458530426},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[{"reaction":"👍","users":["sidunai"],"count":1}],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2408.00118","authors":[{"_id":"66aca7a3eb69d5b0eba59951","name":"Gemma Team","hidden":false},{"_id":"66aca7a3eb69d5b0eba59952","user":{"_id":"66953d345d5615345518ca31","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66953d345d5615345518ca31/rIpTxlrMxVNB9Td9fJijl.png","isPro":false,"fullname":"Morgane Riviere","user":"morugan","type":"user"},"name":"Morgane Riviere","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:21:44.445Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59953","user":{"_id":"667125edfc615a6257be29bc","avatarUrl":"/avatars/c1b191d6443b7a4ac52866a746ca5717.svg","isPro":false,"fullname":"Shreya Pathak","user":"shreyapathak","type":"user"},"name":"Shreya Pathak","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:21:51.931Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59954","user":{"_id":"667a7876becec8fc5112cf1f","avatarUrl":"/avatars/6874f0e5a7edca4145c57d7908f89840.svg","isPro":false,"fullname":"Pier Giuseppe Sessa","user":"piergs","type":"user"},"name":"Pier Giuseppe Sessa","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:22:01.722Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59955","user":{"_id":"663a5ec5a4152b77b6cf96fd","avatarUrl":"/avatars/a51901b21d8670e26b2ac57fd482bb63.svg","isPro":false,"fullname":"Cassidy Hardin","user":"casshardin","type":"user"},"name":"Cassidy Hardin","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:22:07.910Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59956","user":{"_id":"65d7644d19bf1e60614dee6e","avatarUrl":"/avatars/422a8d28fa9d9ff2c3ec1406352cb346.svg","isPro":false,"fullname":"Surya Bhupatiraju","user":"suryabhupa","type":"user"},"name":"Surya Bhupatiraju","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:22:14.718Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59957","user":{"_id":"667a7b2e1764931987803b46","avatarUrl":"/avatars/da2ee162f33c681e3d1c4d1e6d44dcbd.svg","isPro":false,"fullname":"Léonard Hussenot","user":"leonardhussenot","type":"user"},"name":"Léonard Hussenot","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:22:24.134Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59958","user":{"_id":"6639adab8eb8c9c62ac3fb15","avatarUrl":"/avatars/83133b04922143be1d8a1fb71fb2938f.svg","isPro":false,"fullname":"Thomas Mesnard","user":"thomasmesnard","type":"user"},"name":"Thomas Mesnard","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:22:33.910Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59959","name":"Bobak Shahriari","hidden":false},{"_id":"66aca7a3eb69d5b0eba5995a","user":{"_id":"63c94ede00104ea998de19a6","avatarUrl":"/avatars/273959d87f0c67747588cf0700d64039.svg","isPro":false,"fullname":"Alexandre Rame","user":"alexrame","type":"user"},"name":"Alexandre Ramé","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:22:46.308Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5995b","user":{"_id":"65afb7dbdd6bdfd73cd8e609","avatarUrl":"/avatars/b21069bc2d7ee4cc1508008e3c8ade64.svg","isPro":false,"fullname":"Johan Ferret","user":"ferretj","type":"user"},"name":"Johan Ferret","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:22:53.748Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5995c","name":"Peter Liu","hidden":false},{"_id":"66aca7a3eb69d5b0eba5995d","user":{"_id":"6558091ecafc48de36f848a8","avatarUrl":"/avatars/3b7e8327d369436d02fe8c58dde036c2.svg","isPro":false,"fullname":"Pouya Tafti","user":"pdt","type":"user"},"name":"Pouya Tafti","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:36:32.170Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5995e","name":"Abe Friesen","hidden":false},{"_id":"66aca7a3eb69d5b0eba5995f","user":{"_id":"662806c1167ee93ec8e503a4","avatarUrl":"/avatars/8dcf47f91328a38833636068686a8a5a.svg","isPro":false,"fullname":"Michelle Casbon","user":"michellecasbon","type":"user"},"name":"Michelle Casbon","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:36:42.087Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59960","name":"Sabela Ramos","hidden":false},{"_id":"66aca7a3eb69d5b0eba59961","user":{"_id":"6317476e106740aaad50e2cd","avatarUrl":"/avatars/7f066cb7b99f24b57f87598bc9ba7417.svg","isPro":false,"fullname":"Ravin Kumar","user":"canyon289","type":"user"},"name":"Ravin Kumar","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:36:52.026Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59962","name":"Charline Le Lan","hidden":false},{"_id":"66aca7a3eb69d5b0eba59963","name":"Sammy Jerome","hidden":false},{"_id":"66aca7a3eb69d5b0eba59964","user":{"_id":"63ab8e5c8a357d5057ae659e","avatarUrl":"/avatars/0eed94fcdb6d7c86259bc5b72b5bd41e.svg","isPro":false,"fullname":"Anton Tsitsulin","user":"xgfs","type":"user"},"name":"Anton Tsitsulin","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:37:07.767Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59965","name":"Nino Vieillard","hidden":false},{"_id":"66aca7a3eb69d5b0eba59966","name":"Piotr Stanczyk","hidden":false},{"_id":"66aca7a3eb69d5b0eba59967","name":"Sertan Girgin","hidden":false},{"_id":"66aca7a3eb69d5b0eba59968","name":"Nikola Momchev","hidden":false},{"_id":"66aca7a3eb69d5b0eba59969","name":"Matt Hoffman","hidden":false},{"_id":"66aca7a3eb69d5b0eba5996a","name":"Shantanu Thakoor","hidden":false},{"_id":"66aca7a3eb69d5b0eba5996b","name":"Jean-Bastien Grill","hidden":false},{"_id":"66aca7a3eb69d5b0eba5996c","name":"Behnam Neyshabur","hidden":false},{"_id":"66aca7a3eb69d5b0eba5996d","name":"Alanna Walton","hidden":false},{"_id":"66aca7a3eb69d5b0eba5996e","user":{"_id":"66ac94b19c108b7a2af462c5","avatarUrl":"/avatars/dc1b38b62dc58b8a0f19fd4978f7bb8c.svg","isPro":false,"fullname":"Aliaksei Severyn","user":"aseveryn","type":"user"},"name":"Aliaksei Severyn","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:38:40.223Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5996f","user":{"_id":"659d71eaf0ae1a57ae140831","avatarUrl":"/avatars/b9eb8c70b4fe5dbd0f1555602ae41b65.svg","isPro":false,"fullname":"Alicia Parrish","user":"avparrish","type":"user"},"name":"Alicia Parrish","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:38:46.584Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59970","name":"Aliya Ahmad","hidden":false},{"_id":"66aca7a3eb69d5b0eba59971","name":"Allen Hutchison","hidden":false},{"_id":"66aca7a3eb69d5b0eba59972","user":{"_id":"66aaa23c08609915b3855f0b","avatarUrl":"/avatars/6e6c105d921c41477b61af7a4a1b6495.svg","isPro":false,"fullname":"Alvin Abdagic","user":"aabdagic","type":"user"},"name":"Alvin Abdagic","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:39:19.565Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59973","name":"Amanda Carl","hidden":false},{"_id":"66aca7a3eb69d5b0eba59974","name":"Amy Shen","hidden":false},{"_id":"66aca7a3eb69d5b0eba59975","name":"Andy Brock","hidden":false},{"_id":"66aca7a3eb69d5b0eba59976","user":{"_id":"63b334ff922f26a27e79cf1e","avatarUrl":"/avatars/a3110658bae1a7c9591c096c6ac457f6.svg","isPro":false,"fullname":"Andy Coenen","user":"cannoneyed","type":"user"},"name":"Andy Coenen","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:40:04.549Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59977","user":{"_id":"642cd49253e76b4c22872896","avatarUrl":"/avatars/ff2bab2077064d16082f0a7592e821ba.svg","isPro":false,"fullname":"Anthony Laforge","user":"aglaforge","type":"user"},"name":"Anthony Laforge","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:40:12.260Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59978","name":"Antonia Paterson","hidden":false},{"_id":"66aca7a3eb69d5b0eba59979","name":"Ben Bastian","hidden":false},{"_id":"66aca7a3eb69d5b0eba5997a","name":"Bilal Piot","hidden":false},{"_id":"66aca7a3eb69d5b0eba5997b","name":"Bo Wu","hidden":false},{"_id":"66aca7a3eb69d5b0eba5997c","user":{"_id":"64e4f2bfccb9ee90b10ffdba","avatarUrl":"/avatars/49a28f87b7280c47d2b10cf0fd35d332.svg","isPro":false,"fullname":"Brandon Royal","user":"broyal","type":"user"},"name":"Brandon Royal","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:40:39.045Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5997d","name":"Charlie Chen","hidden":false},{"_id":"66aca7a3eb69d5b0eba5997e","name":"Chintu Kumar","hidden":false},{"_id":"66aca7a3eb69d5b0eba5997f","user":{"_id":"630cf8e33dc31beba6eeadfc","avatarUrl":"/avatars/f66eaecb7e710e58f0c06687503c106d.svg","isPro":false,"fullname":"Chris Perry","user":"cperry","type":"user"},"name":"Chris Perry","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:41:43.466Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59980","name":"Chris Welty","hidden":false},{"_id":"66aca7a3eb69d5b0eba59981","user":{"_id":"649c5307e89a7650158f4965","avatarUrl":"/avatars/3c71b501db27c86ed686ae26ad1391cd.svg","isPro":false,"fullname":"Christopher A. Choquette-Choo","user":"cchoquette","type":"user"},"name":"Christopher A. Choquette-Choo","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:41:55.951Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59982","user":{"_id":"667a6ddc400b4b8da62488f1","avatarUrl":"/avatars/65772bd92bbbe832eeb51298a231f57a.svg","isPro":false,"fullname":"Danila Sinopalnikov","user":"sinopalnikov","type":"user"},"name":"Danila Sinopalnikov","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:42:04.581Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59983","name":"David Weinberger","hidden":false},{"_id":"66aca7a3eb69d5b0eba59984","name":"Dimple Vijaykumar","hidden":false},{"_id":"66aca7a3eb69d5b0eba59985","name":"Dominika Rogozińska","hidden":false},{"_id":"66aca7a3eb69d5b0eba59986","name":"Dustin Herbison","hidden":false},{"_id":"66aca7a3eb69d5b0eba59987","name":"Elisa Bandy","hidden":false},{"_id":"66aca7a3eb69d5b0eba59988","name":"Emma Wang","hidden":false},{"_id":"66aca7a3eb69d5b0eba59989","name":"Eric Noland","hidden":false},{"_id":"66aca7a3eb69d5b0eba5998a","name":"Erica Moreira","hidden":false},{"_id":"66aca7a3eb69d5b0eba5998b","user":{"_id":"62a47ec0fdd7a13aac264904","avatarUrl":"/avatars/f2ff23645203ed2399cfe8b447af437a.svg","isPro":false,"fullname":"Evan Senter","user":"evansenter","type":"user"},"name":"Evan Senter","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:43:51.964Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5998c","user":{"_id":"638b4f0a3bbf29e589029a2e","avatarUrl":"/avatars/d39fcf06b93affb6d07f534b952c6c16.svg","isPro":false,"fullname":"Evgenii Eltyshev","user":"EvgeniiEltyshev","type":"user"},"name":"Evgenii Eltyshev","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:43:59.687Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5998d","user":{"_id":"6891bd04a98f679b467dd2d2","avatarUrl":"/avatars/103aca91893e3cec16d3cbc4a80e2077.svg","isPro":false,"fullname":"Francesco Visin","user":"fvisin","type":"user"},"name":"Francesco Visin","status":"claimed_verified","statusLastChangedAt":"2025-09-26T12:30:22.040Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5998e","user":{"_id":"63174c76c92fd6fee319e7f5","avatarUrl":"/avatars/a2a71b9eed5aa97925740290b2c41dc6.svg","isPro":false,"fullname":"Gabriel Rasskin","user":"grasskin","type":"user"},"name":"Gabriel Rasskin","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:44:10.319Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5998f","user":{"_id":"66a138cc627519df4aa4b7ab","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66a138cc627519df4aa4b7ab/XdIE8H9v22kvsWweMDF6d.jpeg","isPro":false,"fullname":"Gary Wei","user":"cdwei-gg","type":"user"},"name":"Gary Wei","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:44:43.648Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59990","name":"Glenn Cameron","hidden":false},{"_id":"66aca7a3eb69d5b0eba59991","user":{"_id":"637a4a31e20e8765c5220b77","avatarUrl":"/avatars/eb8e5b3bf2c1c99dad4f35ac24788669.svg","isPro":false,"fullname":"Luiz Gustavo Martins","user":"gusthema","type":"user"},"name":"Gus Martins","status":"claimed_verified","statusLastChangedAt":"2025-09-26T12:30:16.962Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59992","user":{"_id":"64da4f7578864cd2221af0c1","avatarUrl":"/avatars/428d3fd717654893984f9d01f3d6667d.svg","isPro":false,"fullname":"Hadi Hashemi","user":"hadi-hashemi","type":"user"},"name":"Hadi Hashemi","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:45:20.378Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59993","name":"Hanna Klimczak-Plucińska","hidden":false},{"_id":"66aca7a3eb69d5b0eba59994","name":"Harleen Batra","hidden":false},{"_id":"66aca7a3eb69d5b0eba59995","name":"Harsh Dhand","hidden":false},{"_id":"66aca7a3eb69d5b0eba59996","user":{"_id":"6280f4b052815a0dc7399902","avatarUrl":"/avatars/e098c20e16d17e687d5d0c78745cdc7d.svg","isPro":false,"fullname":"Ivan Nardini","user":"IlNardo92","type":"user"},"name":"Ivan Nardini","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:45:56.728Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59997","name":"Jacinda Mein","hidden":false},{"_id":"66aca7a3eb69d5b0eba59998","name":"Jack Zhou","hidden":false},{"_id":"66aca7a3eb69d5b0eba59999","user":{"_id":"62d33483c20456076a2a2c06","avatarUrl":"/avatars/e1ac677101381030a3174cd859604afa.svg","isPro":false,"fullname":"James Svensson","user":"Sven84","type":"user"},"name":"James Svensson","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:46:11.238Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5999a","name":"Jeff Stanway","hidden":false},{"_id":"66aca7a3eb69d5b0eba5999b","user":{"_id":"6670e8966d34fe6262885205","avatarUrl":"/avatars/f5f7845a54c2bf8e418989f113272578.svg","isPro":false,"fullname":"Jetha Chan","user":"jethac","type":"user"},"name":"Jetha Chan","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:46:31.465Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba5999c","name":"Jin Zhou","hidden":false},{"_id":"66aca7a3eb69d5b0eba5999d","name":"Joana Carrasqueira","hidden":false},{"_id":"66aca7a3eb69d5b0eba5999e","name":"Joana Iljazi","hidden":false},{"_id":"66aca7a3eb69d5b0eba5999f","name":"Jocelyn Becker","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a0","name":"Joe Fernandez","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a1","name":"Joost van Amersfoort","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a2","name":"Josh Gordon","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a3","name":"Josh Lipschultz","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a4","name":"Josh Newlan","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a5","name":"Ju-yeong Ji","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a6","name":"Kareem Mohamed","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a7","name":"Kartikeya Badola","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a8","name":"Kat Black","hidden":false},{"_id":"66aca7a3eb69d5b0eba599a9","name":"Katie Millican","hidden":false},{"_id":"66aca7a3eb69d5b0eba599aa","user":{"_id":"62b9e7c04dde8074b78be449","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1675986645321-62b9e7c04dde8074b78be449.jpeg","isPro":false,"fullname":"Keelin McDonell","user":"keelin","type":"user"},"name":"Keelin McDonell","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:56:57.135Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ab","name":"Kelvin Nguyen","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ac","name":"Kiranbir Sodhia","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ad","name":"Kish Greene","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ae","name":"Lars Lowe Sjoesund","hidden":false},{"_id":"66aca7a3eb69d5b0eba599af","user":{"_id":"63dd3ddea8877129a15a534c","avatarUrl":"/avatars/96bdadf176162856d1687f8298915484.svg","isPro":false,"fullname":"lauren usui","user":"lusui","type":"user"},"name":"Lauren Usui","status":"admin_assigned","statusLastChangedAt":"2024-08-02T13:00:24.875Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b0","name":"Laurent Sifre","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b1","name":"Lena Heuermann","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b2","name":"Leticia Lago","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b3","name":"Lilly McNealus","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b4","name":"Livio Baldini Soares","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b5","name":"Logan Kilpatrick","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b6","user":{"_id":"64c25a9c2b1ffc2eecbce31d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64c25a9c2b1ffc2eecbce31d/uTp3yOPdmCSAmhzWwKUjK.png","isPro":false,"fullname":"Lucas Dixon","user":"iislucas","type":"user"},"name":"Lucas Dixon","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:57:55.078Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b7","name":"Luciano Martins","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b8","name":"Machel Reid","hidden":false},{"_id":"66aca7a3eb69d5b0eba599b9","user":{"_id":"64dc01ab3764a5137e8fbd70","avatarUrl":"/avatars/807bef46ed74bbcc6bd9cac9b9acf82b.svg","isPro":false,"fullname":"Manvinder Singh","user":"manvinder","type":"user"},"name":"Manvinder Singh","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:57:17.021Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ba","name":"Mark Iverson","hidden":false},{"_id":"66aca7a3eb69d5b0eba599bb","name":"Martin Görner","hidden":false},{"_id":"66aca7a3eb69d5b0eba599bc","name":"Mat Velloso","hidden":false},{"_id":"66aca7a3eb69d5b0eba599bd","name":"Mateo Wirth","hidden":false},{"_id":"66aca7a3eb69d5b0eba599be","name":"Matt Davidow","hidden":false},{"_id":"66aca7a3eb69d5b0eba599bf","name":"Matt Miller","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c0","user":{"_id":"6691892b389abe9d474d80da","avatarUrl":"/avatars/7b2fa384c7f1fcaef1e99d297b9a624b.svg","isPro":false,"fullname":"Matthew Rahtz","user":"gdm-mrahtz","type":"user"},"name":"Matthew Rahtz","status":"admin_assigned","statusLastChangedAt":"2024-08-02T13:00:57.136Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c1","name":"Matthew Watson","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c2","user":{"_id":"626b23fc822e3d85324babed","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/626b23fc822e3d85324babed/CbRuisnyy5JsCeA5uxYL7.jpeg","isPro":false,"fullname":"Meg Risdal","user":"megrisdal","type":"user"},"name":"Meg Risdal","status":"admin_assigned","statusLastChangedAt":"2024-08-02T13:00:40.049Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c3","name":"Mehran Kazemi","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c4","user":{"_id":"65aeae63563e362f5521e32a","avatarUrl":"/avatars/ad3a9f68cae8cbaf639084e31adffc54.svg","isPro":false,"fullname":"Michael Moynihan","user":"mmoynihan","type":"user"},"name":"Michael Moynihan","status":"admin_assigned","statusLastChangedAt":"2024-08-02T13:01:19.758Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c5","name":"Ming Zhang","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c6","user":{"_id":"6301f3fb4ccccaa23d38fb7f","avatarUrl":"/avatars/4226111991d1788d3d442ffb2f33ea48.svg","isPro":false,"fullname":"Minsuk Kahng","user":"minsukkahng","type":"user"},"name":"Minsuk Kahng","status":"admin_assigned","statusLastChangedAt":"2024-08-02T13:23:27.543Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c7","name":"Minwoo Park","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c8","user":{"_id":"65d5f883ba0ad765ea02bb9c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65d5f883ba0ad765ea02bb9c/Z0lPfLx6p4i-NAyCslZkN.png","isPro":false,"fullname":"mofi rahman","user":"mofirahman","type":"user"},"name":"Mofi Rahman","status":"admin_assigned","statusLastChangedAt":"2024-08-02T13:01:42.741Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599c9","name":"Mohit Khatwani","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ca","name":"Natalie Dao","hidden":false},{"_id":"66aca7a3eb69d5b0eba599cb","user":{"_id":"65d618051a95fdcf7c59086d","avatarUrl":"/avatars/05256144ae27b46b0bd2ff4eb0ea6642.svg","isPro":false,"fullname":"Nenshad Bardoliwalla","user":"nenshadbardoliwalla","type":"user"},"name":"Nenshad Bardoliwalla","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:58:17.909Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599cc","name":"Nesh Devanathan","hidden":false},{"_id":"66aca7a3eb69d5b0eba599cd","name":"Neta Dumai","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ce","user":{"_id":"66a7935b7959dfd1e912afca","avatarUrl":"/avatars/3d31bfeaf0c008b2c9dae99b8d0298b7.svg","isPro":false,"fullname":"Nilay Chauhan","user":"nilayc","type":"user"},"name":"Nilay Chauhan","status":"admin_assigned","statusLastChangedAt":"2024-08-02T13:00:17.063Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599cf","name":"Oscar Wahltinez","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d0","name":"Pankil Botarda","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d1","name":"Parker Barnes","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d2","name":"Paul Barham","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d3","name":"Paul Michel","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d4","name":"Pengchong Jin","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d5","name":"Petko Georgiev","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d6","user":{"_id":"627a9aa0f967ef475abe8037","avatarUrl":"/avatars/47cf94d6c015577d0a860b617c9ea2b8.svg","isPro":false,"fullname":"Phil Culliton","user":"PhilCulliton","type":"user"},"name":"Phil Culliton","status":"claimed_verified","statusLastChangedAt":"2025-05-29T20:18:12.540Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d7","name":"Pradeep Kuppala","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d8","name":"Ramona Comanescu","hidden":false},{"_id":"66aca7a3eb69d5b0eba599d9","name":"Ramona Merhej","hidden":false},{"_id":"66aca7a3eb69d5b0eba599da","name":"Reena Jana","hidden":false},{"_id":"66aca7a3eb69d5b0eba599db","name":"Reza Ardeshir Rokni","hidden":false},{"_id":"66aca7a3eb69d5b0eba599dc","name":"Rishabh Agarwal","hidden":false},{"_id":"66aca7a3eb69d5b0eba599dd","user":{"_id":"65a402a53522df7a27125823","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65a402a53522df7a27125823/tVR6lZlKmjPqi7dTZBGoS.jpeg","isPro":false,"fullname":"Ryan Mullins","user":"RyanMullins","type":"user"},"name":"Ryan Mullins","status":"claimed_verified","statusLastChangedAt":"2025-03-28T08:39:13.010Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599de","name":"Samaneh Saadat","hidden":false},{"_id":"66aca7a3eb69d5b0eba599df","name":"Sara Mc Carthy","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e0","name":"Sarah Perrin","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e1","name":"Sébastien Arnold","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e2","name":"Sebastian Krause","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e3","name":"Shengyang Dai","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e4","name":"Shruti Garg","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e5","name":"Shruti Sheth","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e6","name":"Sue Ronstrom","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e7","name":"Susan Chan","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e8","name":"Timothy Jordan","hidden":false},{"_id":"66aca7a3eb69d5b0eba599e9","name":"Ting Yu","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ea","name":"Tom Eccles","hidden":false},{"_id":"66aca7a3eb69d5b0eba599eb","name":"Tom Hennigan","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ec","name":"Tomas Kocisky","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ed","name":"Tulsee Doshi","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ee","name":"Vihan Jain","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ef","name":"Vikas Yadav","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f0","user":{"_id":"62639745200163ffba79a4b2","avatarUrl":"/avatars/587e13f75ec044868d0ec5a5969af60d.svg","isPro":false,"fullname":"Vilobh Meshram","user":"vilobhmm","type":"user"},"name":"Vilobh Meshram","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:52:52.733Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f1","name":"Vishal Dharmadhikari","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f2","name":"Warren Barkley","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f3","name":"Wei Wei","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f4","user":{"_id":"654b22fa2d2fcd6bf2a0dc03","avatarUrl":"/avatars/272404d9c935d9b9fbd5fb56fbc85b89.svg","isPro":false,"fullname":"Wenming Ye","user":"yewboulder","type":"user"},"name":"Wenming Ye","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:52:32.891Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f5","user":{"_id":"657b20f3aa07c6e014ba61da","avatarUrl":"/avatars/84a59b1dd99116793e1e411a5b44e94b.svg","isPro":false,"fullname":"Woohyun Han","user":"tink-expo","type":"user"},"name":"Woohyun Han","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:52:24.780Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f6","name":"Woosuk Kwon","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f7","name":"Xiang Xu","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f8","name":"Zhe Shen","hidden":false},{"_id":"66aca7a3eb69d5b0eba599f9","name":"Zhitao Gong","hidden":false},{"_id":"66aca7a3eb69d5b0eba599fa","name":"Zichuan Wei","hidden":false},{"_id":"66aca7a3eb69d5b0eba599fb","user":{"_id":"667458e1a5f9723c7699207a","avatarUrl":"/avatars/3cfb56465b3e59f6ca87b4458423f6d2.svg","isPro":false,"fullname":"Victor Cotruta","user":"victorinox25","type":"user"},"name":"Victor Cotruta","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:51:22.088Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba599fc","name":"Phoebe Kirk","hidden":false},{"_id":"66aca7a3eb69d5b0eba599fd","name":"Anand Rao","hidden":false},{"_id":"66aca7a3eb69d5b0eba599fe","name":"Minh Giang","hidden":false},{"_id":"66aca7a3eb69d5b0eba599ff","user":{"_id":"60e473dcdefbf317ecd88035","avatarUrl":"/avatars/e5c069e6cfc73ea5d3d6c2a437490cdd.svg","isPro":false,"fullname":"LUDOVIC PERAN","user":"LudovicPeran","type":"user"},"name":"Ludovic Peran","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:50:46.824Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a00","user":{"_id":"64d51be85e5f05485ca50751","avatarUrl":"/avatars/5802fcc5a5f1e089df41232892f8f502.svg","isPro":false,"fullname":"Tris Warkentin","user":"trisfromgoogle","type":"user"},"name":"Tris Warkentin","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:50:34.079Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a01","name":"Eli Collins","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a02","name":"Joelle Barral","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a03","name":"Zoubin Ghahramani","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a04","name":"Raia Hadsell","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a05","name":"D. Sculley","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a06","name":"Jeanine Banks","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a07","name":"Anca Dragan","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a08","name":"Slav Petrov","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a09","name":"Oriol Vinyals","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a0a","name":"Jeff Dean","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a0b","name":"Demis Hassabis","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a0c","name":"Koray Kavukcuoglu","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a0d","user":{"_id":"638a689ef560ea995b57aad4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1670015109688-noauth.jpeg","isPro":false,"fullname":"Clement Farabet","user":"clmnt","type":"user"},"name":"Clement Farabet","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:48:30.187Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a0e","user":{"_id":"65da0ecba37e8a543ab732ca","avatarUrl":"/avatars/90889950d8c5e0089ada6b3707104a87.svg","isPro":false,"fullname":"Elena Buchatskaya","user":"ebuchatskaya","type":"user"},"name":"Elena Buchatskaya","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:48:22.767Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a0f","name":"Sebastian Borgeaud","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a10","user":{"_id":"62f82ebd870a3f98bbf9e62a","avatarUrl":"/avatars/3b03299e128abf2c8ba82db704855de7.svg","isPro":false,"fullname":"Noah Fiedel","user":"nfiedel","type":"user"},"name":"Noah Fiedel","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:48:06.968Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a11","name":"Armand Joulin","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a12","user":{"_id":"666b337c3fc0651814997536","avatarUrl":"/avatars/4d6198300c286c5144794bfd2d1fa653.svg","isPro":false,"fullname":"Kathleen Kenealy","user":"kathleenatgoogle","type":"user"},"name":"Kathleen Kenealy","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:47:54.480Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a13","user":{"_id":"65fc7c0da57b88a765aea493","avatarUrl":"/avatars/440f88769da1113d6158fe7e0514ead3.svg","isPro":false,"fullname":"Robert Dadashi","user":"ddsh","type":"user"},"name":"Robert Dadashi","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:47:45.872Z","hidden":false},{"_id":"66aca7a3eb69d5b0eba59a14","user":{"_id":"65d667858ac5cf899e3b66c4","avatarUrl":"/avatars/340468ee8cd9125eceae23a9b8e536a5.svg","isPro":false,"fullname":"Alek Andreev","user":"alekandreev","type":"user"},"name":"Alek Andreev","status":"admin_assigned","statusLastChangedAt":"2024-08-02T12:47:36.757Z","hidden":false}],"publishedAt":"2024-07-31T19:13:07.000Z","submittedOnDailyAt":"2024-08-02T08:02:31.376Z","title":"Gemma 2: Improving Open Language Models at a Practical Size","submittedOnDailyBy":{"_id":"6032802e1f993496bc14d9e3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png","isPro":false,"fullname":"Omar Sanseviero","user":"osanseviero","type":"user"},"summary":"In this work, we introduce Gemma 2, a new addition to the Gemma family of\nlightweight, state-of-the-art open models, ranging in scale from 2 billion to\n27 billion parameters. In this new version, we apply several known technical\nmodifications to the Transformer architecture, such as interleaving\nlocal-global attentions (Beltagy et al., 2020a) and group-query attention\n(Ainslie et al., 2023). We also train the 2B and 9B models with knowledge\ndistillation (Hinton et al., 2015) instead of next token prediction. The\nresulting models deliver the best performance for their size, and even offer\ncompetitive alternatives to models that are 2-3 times bigger. We release all\nour models to the community.","upvotes":78,"discussionId":"66aca7a4eb69d5b0eba59a65","ai_summary":"Gemma 2 introduces improvements in the Transformer architecture through interleaving local-global attentions and group-query attention, showcasing superior performance relative to its size.","ai_keywords":["Transformer architecture","interleaving local-global attentions","group-query attention","knowledge distillation"]},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6032802e1f993496bc14d9e3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png","isPro":false,"fullname":"Omar Sanseviero","user":"osanseviero","type":"user"},{"_id":"64203551f837b31c1cfa566d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64203551f837b31c1cfa566d/xK5H1z7Zy1mQswgXRZjg1.jpeg","isPro":false,"fullname":"Nathan Sarrazin","user":"nsarrazin","type":"user"},{"_id":"5dd96eb166059660ed1ee413","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg","isPro":true,"fullname":"Julien Chaumond","user":"julien-c","type":"user"},{"_id":"61b85ce86eb1f2c5e6233736","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1655385361868-61b85ce86eb1f2c5e6233736.jpeg","isPro":false,"fullname":"Vaibhav Srivastav","user":"reach-vb","type":"user"},{"_id":"64bbe9b236eb058cd9d6a5b9","avatarUrl":"/avatars/c7c01a3fa8809e73800392679abff6d5.svg","isPro":false,"fullname":"Kai Zuberbühler","user":"kaizuberbuehler","type":"user"},{"_id":"648eb1eb59c4e5c87dc116e0","avatarUrl":"/avatars/c636cea39c2c0937f01398c94ead5dad.svg","isPro":false,"fullname":"fdsqefsgergd","user":"T-representer","type":"user"},{"_id":"63ab8e5c8a357d5057ae659e","avatarUrl":"/avatars/0eed94fcdb6d7c86259bc5b72b5bd41e.svg","isPro":false,"fullname":"Anton Tsitsulin","user":"xgfs","type":"user"},{"_id":"62a4ac6fd83c3facafa50892","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/62a4ac6fd83c3facafa50892/qFpobw9B5XaLZvwn0XbmB.jpeg","isPro":false,"fullname":"Mohammed Brıman","user":"mohammedbriman","type":"user"},{"_id":"61b253b7ac5ecaae3d1efe0c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61b253b7ac5ecaae3d1efe0c/hwiQ0uvz3t-L5a-NtBIO6.png","isPro":false,"fullname":"Joshua","user":"Xenova","type":"user"},{"_id":"63a369d98c0c89dcae3b8329","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63a369d98c0c89dcae3b8329/AiH2zjy1cnt9OADAAZMLD.jpeg","isPro":false,"fullname":"Adina Yakefu","user":"AdinaY","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"6374ff24cc5cc31768847b8c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6374ff24cc5cc31768847b8c/7_hKYVzSE226qzB5wJ6gw.jpeg","isPro":false,"fullname":"Minghui Jia","user":"Maxwell-Jia","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":2}">

Papers

arxiv:2408.00118

Gemma 2: Improving Open Language Models at a Practical Size

Published on Jul 31, 2024

· Submitted by

Omar Sanseviero on Aug 2, 2024

#2 Paper of the day

Upvote

Authors:

Morgane Riviere ,

Shreya Pathak ,

Pier Giuseppe Sessa ,

Cassidy Hardin ,

Surya Bhupatiraju ,

Léonard Hussenot ,

Thomas Mesnard ,

Alexandre Ramé ,

Johan Ferret ,

Pouya Tafti ,

Michelle Casbon ,

Ravin Kumar ,

Anton Tsitsulin ,

Abstract

Gemma 2 introduces improvements in the Transformer architecture through interleaving local-global attentions and group-query attention, showcasing superior performance relative to its size.

AI-generated summary

In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.

View arXiv page View PDF Add to collection