https://baai-agents.github.io/EgoActor/

\n","updatedAt":"2026-02-05T03:05:43.380Z","author":{"_id":"61e52be53d6dbb1da842316a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61e52be53d6dbb1da842316a/gx0WGPcOCClXPymoKglc4.jpeg","fullname":"Börje Karlsson","name":"tellarin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":27,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7109494209289551},"editors":["tellarin"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/61e52be53d6dbb1da842316a/gx0WGPcOCClXPymoKglc4.jpeg"],"reactions":[],"isReport":false}},{"id":"69854670e2be52b98782a6bb","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2026-02-06T01:40:00.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning](https://huggingface.co/papers/2512.08639) (2025)\n* [PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence](https://huggingface.co/papers/2512.16793) (2025)\n* [Action-Sketcher: From Reasoning to Action via Visual Sketches for Long-Horizon Robotic Manipulation](https://huggingface.co/papers/2601.01618) (2026)\n* [TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments](https://huggingface.co/papers/2602.02459) (2026)\n* [GSR: Learning Structured Reasoning for Embodied Manipulation](https://huggingface.co/papers/2602.01693) (2026)\n* [CoINS: Counterfactual Interactive Navigation via Skill-Aware VLM](https://huggingface.co/papers/2601.03956) (2026)\n* [Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videos](https://huggingface.co/papers/2512.13080) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2026-02-06T01:40:00.533Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7496412396430969},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}},{"id":"69874c2ec481ea7b01b1077e","author":{"_id":"65243980050781c16f234f1f","avatarUrl":"/avatars/743a009681d5d554c27e04300db9f267.svg","fullname":"Avi","name":"avahal","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false},"createdAt":"2026-02-07T14:29:02.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"arXivLens breakdown of this paper 👉 https://arxivlens.com/PaperView/Details/egoactor-grounding-task-planning-into-spatial-aware-egocentric-actions-for-humanoid-robots-via-visual-language-models-3425-0b9ead12\n- Executive Summary\n- Detailed Breakdown\n- Practical Applications","html":"

arXivLens breakdown of this paper 👉 https://arxivlens.com/PaperView/Details/egoactor-grounding-task-planning-into-spatial-aware-egocentric-actions-for-humanoid-robots-via-visual-language-models-3425-0b9ead12

Executive Summary
Detailed Breakdown
Practical Applications

\n","updatedAt":"2026-02-07T14:29:02.558Z","author":{"_id":"65243980050781c16f234f1f","avatarUrl":"/avatars/743a009681d5d554c27e04300db9f267.svg","fullname":"Avi","name":"avahal","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5830453038215637},"editors":["avahal"],"editorAvatarUrls":["/avatars/743a009681d5d554c27e04300db9f267.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2602.04515","authors":[{"_id":"69840866e34659da7e1f4d86","name":"Yu Bai","hidden":false},{"_id":"69840866e34659da7e1f4d87","name":"MingMing Yu","hidden":false},{"_id":"69840866e34659da7e1f4d88","name":"Chaojie Li","hidden":false},{"_id":"69840866e34659da7e1f4d89","name":"Ziyi Bai","hidden":false},{"_id":"69840866e34659da7e1f4d8a","name":"Xinlong Wang","hidden":false},{"_id":"69840866e34659da7e1f4d8b","user":{"_id":"61e52be53d6dbb1da842316a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61e52be53d6dbb1da842316a/gx0WGPcOCClXPymoKglc4.jpeg","isPro":false,"fullname":"Börje Karlsson","user":"tellarin","type":"user"},"name":"Börje F. Karlsson","status":"claimed_verified","statusLastChangedAt":"2026-02-05T10:54:26.009Z","hidden":false}],"publishedAt":"2026-02-04T13:04:56.000Z","submittedOnDailyAt":"2026-02-05T00:35:43.372Z","title":"EgoActor: Grounding Task Planning into Spatial-aware Egocentric Actions for Humanoid Robots via Visual-Language Models","submittedOnDailyBy":{"_id":"61e52be53d6dbb1da842316a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61e52be53d6dbb1da842316a/gx0WGPcOCClXPymoKglc4.jpeg","isPro":false,"fullname":"Börje Karlsson","user":"tellarin","type":"user"},"summary":"Deploying humanoid robots in real-world settings is fundamentally challenging, as it demands tight integration of perception, locomotion, and manipulation under partial-information observations and dynamically changing environments. As well as transitioning robustly between sub-tasks of different types. Towards addressing these challenges, we propose a novel task - EgoActing, which requires directly grounding high-level instructions into various, precise, spatially aware humanoid actions. We further instantiate this task by introducing EgoActor, a unified and scalable vision-language model (VLM) that can predict locomotion primitives (e.g., walk, turn, move sideways, change height), head movements, manipulation commands, and human-robot interactions to coordinate perception and execution in real-time. We leverage broad supervision over egocentric RGB-only data from real-world demonstrations, spatial reasoning question-answering, and simulated environment demonstrations, enabling EgoActor to make robust, context-aware decisions and perform fluent action inference (under 1s) with both 8B and 4B parameter models. Extensive evaluations in both simulated and real-world environments demonstrate that EgoActor effectively bridges abstract task planning and concrete motor execution, while generalizing across diverse tasks and unseen environments.","upvotes":38,"discussionId":"69840866e34659da7e1f4d8c","projectPage":"https://baai-agents.github.io/RoboNoid/EgoActor/","ai_summary":"EgoActor is a unified vision-language model that translates high-level instructions into precise humanoid robot actions through integrated perception and execution across simulated and real-world environments.","ai_keywords":["vision-language model","locomotion primitives","head movements","manipulation commands","human-robot interactions","egocentric RGB-only data","spatial reasoning","question-answering","simulated environment demonstrations","action inference","motor execution"],"organization":{"_id":"61be9739d2f9358e24ca0a4f","name":"BAAI","fullname":"Beijing Academy of Artificial Intelligence","avatar":"https://cdn-uploads.huggingface.co/production/uploads/1664511063789-632c234f42c386ebd2710434.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"61e52be53d6dbb1da842316a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61e52be53d6dbb1da842316a/gx0WGPcOCClXPymoKglc4.jpeg","isPro":false,"fullname":"Börje Karlsson","user":"tellarin","type":"user"},{"_id":"64b8a904ee98fa835421c84e","avatarUrl":"/avatars/96d6c35b54b04293722d43a166e6cff5.svg","isPro":false,"fullname":"Chengxi","user":"chriszhangcx","type":"user"},{"_id":"67cfd3ad27c7f0b2db24eb2c","avatarUrl":"/avatars/6bd886cd7cbceab378ce18e5343ac00a.svg","isPro":false,"fullname":"Bo Li","user":"HopeLee","type":"user"},{"_id":"65dc919c58ea0eec69f264e8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65dc919c58ea0eec69f264e8/bht0MKY5Anz3L8N3xBISN.jpeg","isPro":false,"fullname":"ZM_J","user":"ZM-J","type":"user"},{"_id":"65b36a28ed21070756e8a458","avatarUrl":"/avatars/1726d27aa0f37b162d54492b04fcdaa0.svg","isPro":false,"fullname":"Ziyi Bai","user":"ByZ0e","type":"user"},{"_id":"64b964d31043081735692158","avatarUrl":"/avatars/209ae30594e39f9fbfa2972689b1a8b1.svg","isPro":false,"fullname":"Yu Bai","user":"woodenwhite","type":"user"},{"_id":"65397ffa1ad9b3ba7c4775e1","avatarUrl":"/avatars/9c3c76629a94b4c6990e5934f92eb6af.svg","isPro":false,"fullname":"Yuying Zhu","user":"yuyingz","type":"user"},{"_id":"6516a5115b041d575d30b921","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/9zN09EW3ZnJQnbJvwIDpP.png","isPro":false,"fullname":"Yifan Yu","user":"MarkYu98","type":"user"},{"_id":"6463554dd2044cd1d7c6e0bf","avatarUrl":"/avatars/d7653623117268c545a7063fec69664b.svg","isPro":false,"fullname":"Bingzheng Wei","user":"Bingzheng","type":"user"},{"_id":"6312cab05beb528b5c1500e3","avatarUrl":"/avatars/a328e8cc99fb031b2d5c911c4b577e7e.svg","isPro":false,"fullname":"Fu-En Yang","user":"FuEnYang","type":"user"},{"_id":"6438a9027de34e8ea7e4b257","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6438a9027de34e8ea7e4b257/vib8QSd1AWMr_bR9ig_xJ.jpeg","isPro":false,"fullname":"Jaward Sesay","user":"Jaward","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0,"organization":{"_id":"61be9739d2f9358e24ca0a4f","name":"BAAI","fullname":"Beijing Academy of Artificial Intelligence","avatar":"https://cdn-uploads.huggingface.co/production/uploads/1664511063789-632c234f42c386ebd2710434.png"}}">

Papers

arxiv:2602.04515

EgoActor: Grounding Task Planning into Spatial-aware Egocentric Actions for Humanoid Robots via Visual-Language Models

Published on Feb 4

· Submitted by

Börje Karlsson on Feb 5

Beijing Academy of Artificial Intelligence

Upvote

Authors:

Börje F. Karlsson

Abstract

EgoActor is a unified vision-language model that translates high-level instructions into precise humanoid robot actions through integrated perception and execution across simulated and real-world environments.

AI-generated summary

Deploying humanoid robots in real-world settings is fundamentally challenging, as it demands tight integration of perception, locomotion, and manipulation under partial-information observations and dynamically changing environments. As well as transitioning robustly between sub-tasks of different types. Towards addressing these challenges, we propose a novel task - EgoActing, which requires directly grounding high-level instructions into various, precise, spatially aware humanoid actions. We further instantiate this task by introducing EgoActor, a unified and scalable vision-language model (VLM) that can predict locomotion primitives (e.g., walk, turn, move sideways, change height), head movements, manipulation commands, and human-robot interactions to coordinate perception and execution in real-time. We leverage broad supervision over egocentric RGB-only data from real-world demonstrations, spatial reasoning question-answering, and simulated environment demonstrations, enabling EgoActor to make robust, context-aware decisions and perform fluent action inference (under 1s) with both 8B and 4B parameter models. Extensive evaluations in both simulated and real-world environments demonstrate that EgoActor effectively bridges abstract task planning and concrete motor execution, while generalizing across diverse tasks and unseen environments.