Very interesting 👍
\nThe single symbol function name is a neat little trick, feels so obvious in hindsight.
\nIt would be very interesting to see just how much information we can encapsulate in a single symbol... And then re-use those symbols to increase throughput
\nA kind of meta-language that represents a topology of layers of abstraction, on top of layers of abstraction
\nEach time a new concept is learned, it gets a symbol, and that symbol is then used to further train the model. This new alphabet would effectively represent a map of the knowledge in the model.
\nKind of like database normalisation for embeddings.
\n","updatedAt":"2024-04-03T12:51:03.343Z","author":{"_id":"644e1b1d9b4e87c31bab0a14","avatarUrl":"/avatars/88bb4c4a67dc8958069e9014f5e73a0b.svg","fullname":"Michael Barry","name":"MichaelBarryUK","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":7,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.92112135887146},"editors":["MichaelBarryUK"],"editorAvatarUrls":["/avatars/88bb4c4a67dc8958069e9014f5e73a0b.svg"],"reactions":[{"reaction":"🤯","users":["julien-c","juandavidgf","DoodleBear","BlakeRain","MichaelBarryUK"],"count":5},{"reaction":"🔥","users":["maveriq"],"count":1}],"isReport":false},"replies":[{"id":"660d6ea1341f0fa3de96cf78","author":{"_id":"5dd96eb166059660ed1ee413","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg","fullname":"Julien Chaumond","name":"julien-c","type":"user","isPro":true,"isHf":true,"isHfAdmin":true,"isMod":false,"followerCount":3842,"isUserFollowing":false},"createdAt":"2024-04-03T14:58:41.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"🤯","html":"🤯
\n","updatedAt":"2024-04-03T14:58:41.345Z","author":{"_id":"5dd96eb166059660ed1ee413","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg","fullname":"Julien Chaumond","name":"julien-c","type":"user","isPro":true,"isHf":true,"isHfAdmin":true,"isMod":false,"followerCount":3842,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"oc","probability":0.10113807767629623},"editors":["julien-c"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg"],"reactions":[],"isReport":false,"parentCommentId":"660d50b77bed164f7ec2e32b"}},{"id":"6618087e0796ae11aad36ca3","author":{"_id":"65c082c626c639b0009a3c82","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65c082c626c639b0009a3c82/w_Q_5iLOM-nh_drd9Tu4X.jpeg","fullname":"Alex Chen","name":"alexchen4ai","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":21,"isUserFollowing":false},"createdAt":"2024-04-11T15:57:50.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Hi Michael, thanks for your sharing. I also agree that a single symbol could be powerful. Previously, we've seen many special tokens like the `EOS` token or `Hi Michael, thanks for your sharing. I also agree that a single symbol could be powerful. Previously, we've seen many special tokens like the EOS token or <image> in Llava. According to our more experiments, function as a token could be pretty effective when we want the model to perform function call well. We can also let the model get feedback from the functions to be executed.
I think the idea of using special tokens makes a lot of sense. I think we under appreciate the power and expressiveness of token-space in LLMs.
\nIf you look at techniques LlaVa, ViTs need registers and Prompt Fine Tuning, all of these effectively hack the expressiveness of token-space. With long-context the opportunity to use token-space is larger. If you look at models like Bert, almost 30% of the model is embedding weight, but unlike most layers the ability to add just a single new token can be extremely efficient and extremely powerful. I think as a research community there is a lot of exciting stuff on the horizon here.
\n","updatedAt":"2024-04-12T07:54:50.385Z","author":{"_id":"635f9fd1ae7144a6674c839b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1667211208219-noauth.jpeg","fullname":"Marcus Gawronsky","name":"marcusinthesky","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":11,"isUserFollowing":false}},"numEdits":3,"identifiedLanguage":{"language":"en","probability":0.9335092306137085},"editors":["marcusinthesky"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1667211208219-noauth.jpeg"],"reactions":[],"isReport":false}},{"id":"660e013dffd736d867fb1488","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2024-04-04T01:24:13.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following](https://huggingface.co/papers/2403.03129) (2024)\n* [MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases](https://huggingface.co/papers/2402.14905) (2024)\n* [AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls](https://huggingface.co/papers/2402.04253) (2024)\n* [SwissNYF: Tool Grounded LLM Agents for Black Box Setting](https://huggingface.co/papers/2402.10051) (2024)\n* [A Survey of using Large Language Models for Generating Infrastructure as Code](https://huggingface.co/papers/2404.00227) (2024)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following (2024) \n
- MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases (2024) \n
- AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls (2024) \n
- SwissNYF: Tool Grounded LLM Agents for Black Box Setting (2024) \n
- A Survey of using Large Language Models for Generating Infrastructure as Code (2024) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Will the dataset for this be released in full as open source?
\n","updatedAt":"2024-04-04T02:15:48.116Z","author":{"_id":"641dfddf3bae5a77636817c5","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/641dfddf3bae5a77636817c5/2IwNwh9kK98eCHUmOGoWD.png","fullname":"wing lian","name":"winglian","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4396,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.977699875831604},"editors":["winglian"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/641dfddf3bae5a77636817c5/2IwNwh9kK98eCHUmOGoWD.png"],"reactions":[{"reaction":"🔥","users":["huangyt"],"count":1}],"isReport":false}},{"id":"66659269efbf2cc81e10fa0f","author":{"_id":"6186ddf6a7717cb375090c01","avatarUrl":"/avatars/716b6a7d1094c8036b2a8a7b9063e8aa.svg","fullname":"Julien BLANCHON","name":"blanchon","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":176,"isUserFollowing":false},"createdAt":"2024-06-09T11:30:49.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"# Octopus v2: Revolutionizing On-Device AI for Super Agents!\n\nhttps://cdn-uploads.huggingface.co/production/uploads/6186ddf6a7717cb375090c01/Ef76SslELhzhJE4bVVAw5.mp4 \n\n## Links 🔗:\n👉 Subscribe: https://www.youtube.com/@Arxflix\n👉 Twitter: https://x.com/arxflix\n👉 LMNT (Partner): https://lmnt.com/\n\n\nBy Arxflix\n","html":"\n\t\n\t\t\n\t\n\t\n\t\tOctopus v2: Revolutionizing On-Device AI for Super Agents!\n\t\n
\n \n\n\n\t\n\t\t\n\t\n\t\n\t\tLinks 🔗:\n\t\n
\n👉 Subscribe: https://www.youtube.com/@Arxflix
👉 Twitter: https://x.com/arxflix
👉 LMNT (Partner): https://lmnt.com/
By Arxflix
Octopus v2: On-device language model for super agent
Abstract
A new on-device method improves the performance of a large language model, matching GPT-4's accuracy and latency while reducing context length and enhancing deployment on edge devices.
Language models have shown effectiveness in a variety of software applications, particularly in tasks related to automatic workflow. These models possess the crucial ability to call functions, which is essential in creating AI agents. Despite the high performance of large-scale language models in cloud environments, they are often associated with concerns over privacy and cost. Current on-device models for function calling face issues with latency and accuracy. Our research presents a new method that empowers an on-device model with 2 billion parameters to surpass the performance of GPT-4 in both accuracy and latency, and decrease the context length by 95\%. When compared to Llama-7B with a RAG-based function calling mechanism, our method enhances latency by 35-fold. This method reduces the latency to levels deemed suitable for deployment across a variety of edge devices in production environments, aligning with the performance requisites for real-world applications.
Community
Very interesting 👍
The single symbol function name is a neat little trick, feels so obvious in hindsight.
It would be very interesting to see just how much information we can encapsulate in a single symbol... And then re-use those symbols to increase throughput
A kind of meta-language that represents a topology of layers of abstraction, on top of layers of abstraction
Each time a new concept is learned, it gets a symbol, and that symbol is then used to further train the model. This new alphabet would effectively represent a map of the knowledge in the model.
Kind of like database normalisation for embeddings.
🤯
I think the idea of using special tokens makes a lot of sense. I think we under appreciate the power and expressiveness of token-space in LLMs.
If you look at techniques LlaVa, ViTs need registers and Prompt Fine Tuning, all of these effectively hack the expressiveness of token-space. With long-context the opportunity to use token-space is larger. If you look at models like Bert, almost 30% of the model is embedding weight, but unlike most layers the ability to add just a single new token can be extremely efficient and extremely powerful. I think as a research community there is a lot of exciting stuff on the horizon here.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following (2024)
- MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases (2024)
- AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls (2024)
- SwissNYF: Tool Grounded LLM Agents for Black Box Setting (2024)
- A Survey of using Large Language Models for Generating Infrastructure as Code (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Will the dataset for this be released in full as open source?
Octopus v2: Revolutionizing On-Device AI for Super Agents!
Links 🔗:
👉 Subscribe: https://www.youtube.com/@Arxflix
👉 Twitter: https://x.com/arxflix
👉 LMNT (Partner): https://lmnt.com/
By Arxflix
Models citing this paper 7
Browse 7 models citing this paperDatasets citing this paper 0
No dataset linking this paper