Awesome-Token-level-Model-Compression\", which collects 200+ papers about recent awesome token compression works! Feel free to contribute your suggestions!

\n","updatedAt":"2025-05-27T08:06:39.458Z","author":{"_id":"66a0caa1a7a6ed88ad1c0ddf","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66a0caa1a7a6ed88ad1c0ddf/WoOP24-ruuHy4ryNhRp0D.jpeg","fullname":"Xuyang Liu","name":"xuyang-liu16","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":5,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.6735106706619263},"editors":["xuyang-liu16"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/66a0caa1a7a6ed88ad1c0ddf/WoOP24-ruuHy4ryNhRp0D.jpeg"],"reactions":[{"reaction":"😎","users":["xuyang-liu16","coderchen01","liweijia","zichenwen"],"count":4},{"reaction":"🚀","users":["coderchen01","liweijia","xuyang-liu16","zichenwen"],"count":4}],"isReport":false}},{"id":"683668d725d0c6bd7c856c49","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-05-28T01:37:27.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search](https://huggingface.co/papers/2505.16838) (2025)\n* [Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention](https://huggingface.co/papers/2505.15774) (2025)\n* [SmolVLM: Redefining small and efficient multimodal models](https://huggingface.co/papers/2504.05299) (2025)\n* [Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures](https://huggingface.co/papers/2505.09343) (2025)\n* [CrossLMM: Decoupling Long Video Sequences from LMMs via Dual Cross-Attention Mechanisms](https://huggingface.co/papers/2505.17020) (2025)\n* [Efficient Pretraining Length Scaling](https://huggingface.co/papers/2504.14992) (2025)\n* [Token Sequence Compression for Efficient Multimodal Computing](https://huggingface.co/papers/2504.17892) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

\n

The following papers were recommended by the Semantic Scholar API

\n

\n
• R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search (2025)
\n
• Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention (2025)
\n
• SmolVLM: Redefining small and efficient multimodal models (2025)
\n
• Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures (2025)
\n
• CrossLMM: Decoupling Long Video Sequences from LMMs via Dual Cross-Attention Mechanisms (2025)
\n
• Efficient Pretraining Length Scaling (2025)
\n
• Token Sequence Compression for Efficient Multimodal Computing (2025)
\n

\n

Please give a thumbs up to this comment if you found it helpful!

\n

If you want recommendations for any Paper on Hugging Face checkout this Space

\n

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-05-28T01:37:27.104Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7174310088157654},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}},{"id":"6837d0f34ae8da638a45b3d2","author":{"_id":"6813ee19c9b224a738fea856","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/g1uPHIKEgWe1ftHGHbo_U.png","fullname":"YJ","name":"yjh415","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2025-05-29T03:13:55.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"\n\nthanks to AI now we can turn academic papers into manga! hahaha!\n\naudio overview thanks to notebookllm :D\nhttps://youtu.be/uaSGgWHIeM8?si=Pa_LD6JDnLmBEMX8","html":"

thanks to AI now we can turn academic papers into manga! hahaha!

\n

audio overview thanks to notebookllm :D
https://youtu.be/uaSGgWHIeM8?si=Pa_LD6JDnLmBEMX8

\n","updatedAt":"2025-05-29T16:29:14.118Z","author":{"_id":"6813ee19c9b224a738fea856","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/g1uPHIKEgWe1ftHGHbo_U.png","fullname":"YJ","name":"yjh415","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.7206955552101135},"editors":["yjh415"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/g1uPHIKEgWe1ftHGHbo_U.png"],"reactions":[{"reaction":"😎","users":["xuyang-liu16","linfengZ"],"count":2},{"reaction":"🔥","users":["coderchen01","linfengZ"],"count":2}],"isReport":false},"replies":[{"id":"68397b009224d8ad56d545d4","author":{"_id":"66a0caa1a7a6ed88ad1c0ddf","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66a0caa1a7a6ed88ad1c0ddf/WoOP24-ruuHy4ryNhRp0D.jpeg","fullname":"Xuyang Liu","name":"xuyang-liu16","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":5,"isUserFollowing":false},"createdAt":"2025-05-30T09:31:44.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"That is interesting! I'm very happy to see such amazing generation results.","html":"

That is interesting! I'm very happy to see such amazing generation results.

\n","updatedAt":"2025-05-30T09:31:44.434Z","author":{"_id":"66a0caa1a7a6ed88ad1c0ddf","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66a0caa1a7a6ed88ad1c0ddf/WoOP24-ruuHy4ryNhRp0D.jpeg","fullname":"Xuyang Liu","name":"xuyang-liu16","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":5,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9759975075721741},"editors":["xuyang-liu16"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/66a0caa1a7a6ed88ad1c0ddf/WoOP24-ruuHy4ryNhRp0D.jpeg"],"reactions":[{"reaction":"🚀","users":["coderchen01","zichenwen"],"count":2}],"isReport":false,"parentCommentId":"6837d0f34ae8da638a45b3d2"}}]},{"id":"694aeef44f715c34acd47873","author":{"_id":"65243980050781c16f234f1f","avatarUrl":"/avatars/743a009681d5d554c27e04300db9f267.svg","fullname":"Avi","name":"avahal","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false},"createdAt":"2025-12-23T19:35:16.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"arXiv lens breakdown of this paper 👉 https://arxivlens.com/PaperView/Details/shifting-ai-efficiency-from-model-centric-to-data-centric-compression-2173-5358afcf\n- Executive Summary\n- Detailed Breakdown\n- Practical Applications","html":"

arXiv lens breakdown of this paper 👉 https://arxivlens.com/PaperView/Details/shifting-ai-efficiency-from-model-centric-to-data-centric-compression-2173-5358afcf

\n

\n
• Executive Summary
\n
• Detailed Breakdown
\n
• Practical Applications
\n

\n","updatedAt":"2025-12-23T19:35:16.166Z","author":{"_id":"65243980050781c16f234f1f","avatarUrl":"/avatars/743a009681d5d554c27e04300db9f267.svg","fullname":"Avi","name":"avahal","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7194615602493286},"editors":["avahal"],"editorAvatarUrls":["/avatars/743a009681d5d554c27e04300db9f267.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2505.19147","authors":[{"_id":"68353258d005e45149d2d384","user":{"_id":"66a0caa1a7a6ed88ad1c0ddf","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66a0caa1a7a6ed88ad1c0ddf/WoOP24-ruuHy4ryNhRp0D.jpeg","isPro":false,"fullname":"Xuyang Liu","user":"xuyang-liu16","type":"user"},"name":"Xuyang Liu","status":"claimed_verified","statusLastChangedAt":"2025-05-27T08:23:05.932Z","hidden":false},{"_id":"68353258d005e45149d2d385","user":{"_id":"653b8c3e97a4d71d950e2f20","avatarUrl":"/avatars/b68880022e14556d0be58c69615db3be.svg","isPro":false,"fullname":"Zichen Wen","user":"zichenwen","type":"user"},"name":"Zichen Wen","status":"claimed_verified","statusLastChangedAt":"2025-05-27T07:50:45.710Z","hidden":false},{"_id":"68353258d005e45149d2d386","user":{"_id":"66968099c952e09a4cb29f78","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66968099c952e09a4cb29f78/n90NI2R3E9_RqCyMjDCQF.webp","isPro":false,"fullname":"Wang","user":"Steven-Shaobo","type":"user"},"name":"Shaobo Wang","status":"claimed_verified","statusLastChangedAt":"2025-05-27T07:50:41.853Z","hidden":false},{"_id":"68353258d005e45149d2d387","user":{"_id":"652f8642338c761caf474169","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/mq5jjqqNaFxVboWGDEocJ.jpeg","isPro":false,"fullname":"Junjie Chen","user":"coderchen01","type":"user"},"name":"Junjie Chen","status":"claimed_verified","statusLastChangedAt":"2025-05-27T07:50:48.148Z","hidden":false},{"_id":"68353258d005e45149d2d388","user":{"_id":"679f280ffb07d74f084520b6","avatarUrl":"/avatars/b378000f68c7faf8d4fee8074dd2db5b.svg","isPro":false,"fullname":"Zhishan Tao","user":"Pppeach33","type":"user"},"name":"Zhishan Tao","status":"claimed_verified","statusLastChangedAt":"2025-05-27T07:50:43.758Z","hidden":false},{"_id":"68353258d005e45149d2d389","name":"Yubo Wang","hidden":false},{"_id":"68353258d005e45149d2d38a","user":{"_id":"64abcbfde144ba0eb9bb8419","avatarUrl":"/avatars/6ccea0e755bad384aaabd5c455bd962e.svg","isPro":false,"fullname":"Xiangqi Jin","user":"Lueci4er","type":"user"},"name":"Xiangqi Jin","status":"admin_assigned","statusLastChangedAt":"2025-05-27T08:21:21.961Z","hidden":false},{"_id":"68353258d005e45149d2d38b","user":{"_id":"6623417a9c9b00aaff64de33","avatarUrl":"/avatars/58c108ebf486d90866d8626bbb69312d.svg","isPro":false,"fullname":"Chang Zou","user":"Shenyizou","type":"user"},"name":"Chang Zou","status":"claimed_verified","statusLastChangedAt":"2025-07-21T08:31:06.699Z","hidden":false},{"_id":"68353258d005e45149d2d38c","name":"Yiyu Wang","hidden":false},{"_id":"68353258d005e45149d2d38d","user":{"_id":"6806464ed918f6d2fee2bc8b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6806464ed918f6d2fee2bc8b/rgpG2oO0m6PT0KltCF_Wf.jpeg","isPro":false,"fullname":"Chenfei Liao","user":"Chenfei-Liao","type":"user"},"name":"Chenfei Liao","status":"claimed_verified","statusLastChangedAt":"2025-05-28T09:00:32.765Z","hidden":false},{"_id":"68353258d005e45149d2d38e","user":{"_id":"65961a7daa677adc97d8e33f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65961a7daa677adc97d8e33f/TA0Vz7CDmYZEdrhnsj_1b.jpeg","isPro":false,"fullname":"Xu Zheng","user":"xz287","type":"user"},"name":"Xu Zheng","status":"claimed_verified","statusLastChangedAt":"2025-06-12T12:43:38.659Z","hidden":false},{"_id":"68353258d005e45149d2d38f","name":"Honggang Chen","hidden":false},{"_id":"68353258d005e45149d2d390","name":"Weijia Li","hidden":false},{"_id":"68353258d005e45149d2d391","name":"Xuming Hu","hidden":false},{"_id":"68353258d005e45149d2d392","user":{"_id":"63f9fca8d4349b157a109eec","avatarUrl":"/avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg","isPro":false,"fullname":"Conghui He","user":"conghui","type":"user"},"name":"Conghui He","status":"admin_assigned","statusLastChangedAt":"2025-05-27T08:22:15.329Z","hidden":false},{"_id":"68353258d005e45149d2d393","user":{"_id":"642ec9831d1737803dc1c30a","avatarUrl":"/avatars/c9ded838bad09004c15a27200e66a108.svg","isPro":false,"fullname":"linfeng zhang","user":"linfengZ","type":"user"},"name":"Linfeng Zhang","status":"admin_assigned","statusLastChangedAt":"2025-05-27T08:22:07.787Z","hidden":false}],"publishedAt":"2025-05-25T13:51:17.000Z","submittedOnDailyAt":"2025-05-27T02:06:05.849Z","title":"Shifting AI Efficiency From Model-Centric to Data-Centric Compression","submittedOnDailyBy":{"_id":"653b8c3e97a4d71d950e2f20","avatarUrl":"/avatars/b68880022e14556d0be58c69615db3be.svg","isPro":false,"fullname":"Zichen Wen","user":"zichenwen","type":"user"},"summary":"The rapid advancement of large language models (LLMs) and multi-modal LLMs\n(MLLMs) has historically relied on model-centric scaling through increasing\nparameter counts from millions to hundreds of billions to drive performance\ngains. However, as we approach hardware limits on model size, the dominant\ncomputational bottleneck has fundamentally shifted to the quadratic cost of\nself-attention over long token sequences, now driven by ultra-long text\ncontexts, high-resolution images, and extended videos. In this position paper,\nwe argue that the focus of research for efficient AI is shifting from\nmodel-centric compression to data-centric compression. We position token\ncompression as the new frontier, which improves AI efficiency via reducing the\nnumber of tokens during model training or inference. Through comprehensive\nanalysis, we first examine recent developments in long-context AI across\nvarious domains and establish a unified mathematical framework for existing\nmodel efficiency strategies, demonstrating why token compression represents a\ncrucial paradigm shift in addressing long-context overhead. Subsequently, we\nsystematically review the research landscape of token compression, analyzing\nits fundamental benefits and identifying its compelling advantages across\ndiverse scenarios. Furthermore, we provide an in-depth analysis of current\nchallenges in token compression research and outline promising future\ndirections. Ultimately, our work aims to offer a fresh perspective on AI\nefficiency, synthesize existing research, and catalyze innovative developments\nto address the challenges that increasing context lengths pose to the AI\ncommunity's advancement.","upvotes":145,"discussionId":"68353259d005e45149d2d3c0","projectPage":"https://github.com/xuyang-liu16/Awesome-Token-level-Model-Compression","githubRepo":"https://github.com/xuyang-liu16/Awesome-Token-level-Model-Compression","githubRepoAddedBy":"user","ai_summary":"The focus in AI research is shifting from model-centric to data-centric compression, with token compression identified as key to improving efficiency in handling long-context scenarios.","ai_keywords":["large language models","multi-modal LLMs","self-attention","token compression","long-context AI","mathematical framework","model efficiency","long-context overhead","current challenges","future directions"],"githubStars":190,"organization":{"_id":"63e5ef7bf2e9a8f22c515654","name":"SJTU","fullname":"Shanghai Jiao Tong University","avatar":"https://cdn-uploads.huggingface.co/production/uploads/1676013394657-63e5ee22b6a40bf941da0928.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"653b8c3e97a4d71d950e2f20","avatarUrl":"/avatars/b68880022e14556d0be58c69615db3be.svg","isPro":false,"fullname":"Zichen Wen","user":"zichenwen","type":"user"},{"_id":"67833c6e1ad778871bd02ce8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Upm1TXwluVTlp49km876u.png","isPro":false,"fullname":"yibo","user":"Alex-liuyibo","type":"user"},{"_id":"665c476f052479b276a7239d","avatarUrl":"/avatars/f1ac6fa099efd6141a7382c6ecfced96.svg","isPro":false,"fullname":"yuwei zhang ","user":"zyw2002","type":"user"},{"_id":"66a0caa1a7a6ed88ad1c0ddf","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66a0caa1a7a6ed88ad1c0ddf/WoOP24-ruuHy4ryNhRp0D.jpeg","isPro":false,"fullname":"Xuyang Liu","user":"xuyang-liu16","type":"user"},{"_id":"6720d9fd72358ddfaf90c42e","avatarUrl":"/avatars/f7a06b8d131af375b0c62c4d35c47b6d.svg","isPro":false,"fullname":"GM","user":"MgGladys","type":"user"},{"_id":"66968099c952e09a4cb29f78","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66968099c952e09a4cb29f78/n90NI2R3E9_RqCyMjDCQF.webp","isPro":false,"fullname":"Wang","user":"Steven-Shaobo","type":"user"},{"_id":"682d748c97a31e5ac840ce48","avatarUrl":"/avatars/bd9bed53963a82b4d23b0c5fbb2a08f8.svg","isPro":false,"fullname":"Yuzhe","user":"AidenChenyu","type":"user"},{"_id":"652a589f1a3250bbfe3a3a11","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/652a589f1a3250bbfe3a3a11/_TtCceOAdlRbs1L-ZMvJm.jpeg","isPro":false,"fullname":"Ashley","user":"AshleyLuo","type":"user"},{"_id":"652f8642338c761caf474169","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/mq5jjqqNaFxVboWGDEocJ.jpeg","isPro":false,"fullname":"Junjie Chen","user":"coderchen01","type":"user"},{"_id":"65fd82762bf2cd20ddaa193f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/yBYbWp_mT7UusYdkqtAvw.png","isPro":false,"fullname":"Siteng Huang","user":"huangsiteng","type":"user"},{"_id":"67e60670bfead911e97ddb2e","avatarUrl":"/avatars/7ec68214323791282549196a0db79943.svg","isPro":false,"fullname":"Jingyun Wang","user":"bzh666","type":"user"},{"_id":"681da87ee27a030c96adc0d1","avatarUrl":"/avatars/1598f12fbe514503617b008ca5fb1926.svg","isPro":false,"fullname":"Liangyu Xu","user":"GFZShiwai","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":2,"organization":{"_id":"63e5ef7bf2e9a8f22c515654","name":"SJTU","fullname":"Shanghai Jiao Tong University","avatar":"https://cdn-uploads.huggingface.co/production/uploads/1676013394657-63e5ee22b6a40bf941da0928.png"}}">