Librarian Bot. I found the following papers similar to this paper.

\n

The following papers were recommended by the Semantic Scholar API

\n

\n
• ECBench: Can Multi-modal Foundation Models Understand the Egocentric World? A Holistic Embodied Cognition Benchmark (2025)
\n
• From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons (2024)
\n
• A Multimodal Social Agent (2024)
\n
• Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (2024)
\n
• DriveMM: All-in-One Large Multimodal Model for Autonomous Driving (2024)
\n
• ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing (2024)
\n
• Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts (2025)
\n

\n

Please give a thumbs up to this comment if you found it helpful!

\n

If you want recommendations for any Paper on Hugging Face checkout this Space

\n

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-01-25T01:33:26.260Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7409428358078003},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2501.11858","authors":[{"_id":"67943e877aa0faad86e4d750","name":"Zhili Cheng","hidden":false},{"_id":"67943e877aa0faad86e4d751","name":"Yuge Tu","hidden":false},{"_id":"67943e877aa0faad86e4d752","name":"Ran Li","hidden":false},{"_id":"67943e877aa0faad86e4d753","name":"Shiqi Dai","hidden":false},{"_id":"67943e877aa0faad86e4d754","user":{"_id":"637a06580a77f602dc4ac922","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/637a06580a77f602dc4ac922/qOJAhHOEE2N-HzRcZOu1L.jpeg","isPro":false,"fullname":"Jinyi Hu","user":"JamesHujy","type":"user"},"name":"Jinyi Hu","status":"claimed_verified","statusLastChangedAt":"2025-01-26T11:38:36.572Z","hidden":false},{"_id":"67943e877aa0faad86e4d755","name":"Shengding Hu","hidden":false},{"_id":"67943e877aa0faad86e4d756","name":"Jiahao Li","hidden":false},{"_id":"67943e877aa0faad86e4d757","user":{"_id":"673c7319d11b1c2e246ead9c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/673c7319d11b1c2e246ead9c/IjFIO--N7Hm_BOEafhEQv.jpeg","isPro":false,"fullname":"Yang Shi","user":"DogNeverSleep","type":"user"},"name":"Yang Shi","status":"claimed_verified","statusLastChangedAt":"2025-02-18T09:35:10.394Z","hidden":false},{"_id":"67943e877aa0faad86e4d758","user":{"_id":"64abc4aa6cadc7aca585dddf","avatarUrl":"/avatars/736afea979cd0021c7a37f68731524ea.svg","isPro":false,"fullname":"Tianyu Yu","user":"Yirany","type":"user"},"name":"Tianyu Yu","status":"claimed_verified","statusLastChangedAt":"2025-06-24T12:43:17.142Z","hidden":false},{"_id":"67943e877aa0faad86e4d759","name":"Weize Chen","hidden":false},{"_id":"67943e877aa0faad86e4d75a","name":"Lei Shi","hidden":false},{"_id":"67943e877aa0faad86e4d75b","name":"Maosong Sun","hidden":false}],"publishedAt":"2025-01-21T03:22:10.000Z","submittedOnDailyAt":"2025-01-24T23:00:01.730Z","title":"EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents","submittedOnDailyBy":{"_id":"637a06580a77f602dc4ac922","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/637a06580a77f602dc4ac922/qOJAhHOEE2N-HzRcZOu1L.jpeg","isPro":false,"fullname":"Jinyi Hu","user":"JamesHujy","type":"user"},"summary":"Multimodal Large Language Models (MLLMs) have shown significant advancements,\nproviding a promising future for embodied agents. Existing benchmarks for\nevaluating MLLMs primarily utilize static images or videos, limiting\nassessments to non-interactive scenarios. Meanwhile, existing embodied AI\nbenchmarks are task-specific and not diverse enough, which do not adequately\nevaluate the embodied capabilities of MLLMs. To address this, we propose\nEmbodiedEval, a comprehensive and interactive evaluation benchmark for MLLMs\nwith embodied tasks. EmbodiedEval features 328 distinct tasks within 125 varied\n3D scenes, each of which is rigorously selected and annotated. It covers a\nbroad spectrum of existing embodied AI tasks with significantly enhanced\ndiversity, all within a unified simulation and evaluation framework tailored\nfor MLLMs. The tasks are organized into five categories: navigation, object\ninteraction, social interaction, attribute question answering, and spatial\nquestion answering to assess different capabilities of the agents. We evaluated\nthe state-of-the-art MLLMs on EmbodiedEval and found that they have a\nsignificant shortfall compared to human level on embodied tasks. Our analysis\ndemonstrates the limitations of existing MLLMs in embodied capabilities,\nproviding insights for their future development. We open-source all evaluation\ndata and simulation framework at https://github.com/thunlp/EmbodiedEval.","upvotes":7,"discussionId":"67943e8a7aa0faad86e4d9f7","githubRepo":"https://github.com/thunlp/embodiedeval","githubRepoAddedBy":"auto","ai_summary":"EmbodiedEval is a comprehensive benchmark for assessing the embodied capabilities of multimodal large language models using diverse interactive tasks in 3D scenes.","ai_keywords":["multimodal large language models","embodied AI","static images","videos","non-interactive scenarios","task-specific benchmarks","embodiedEval","328 distinct tasks","125 varied 3D scenes","navigation","object interaction","social interaction","attribute question answering","spatial question answering","state-of-the-art MLLMs"],"githubStars":57},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"637a06580a77f602dc4ac922","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/637a06580a77f602dc4ac922/qOJAhHOEE2N-HzRcZOu1L.jpeg","isPro":false,"fullname":"Jinyi Hu","user":"JamesHujy","type":"user"},{"_id":"63b2a92e18e5cf2cdd333492","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63b2a92e18e5cf2cdd333492/GxnngJG0u7d0jYTEFOrfe.png","isPro":false,"fullname":"Jaehyun Jun","user":"btjhjeon","type":"user"},{"_id":"635563ce525beaee688c4fc8","avatarUrl":"/avatars/f74eec56da76dd77a23649b76b857d8d.svg","isPro":false,"fullname":"Zhili Cheng","user":"chengzl18","type":"user"},{"_id":"6054c9adc26fe34ec2e63d72","avatarUrl":"/avatars/a6cf48befd61b49d24286350f3c894c4.svg","isPro":false,"fullname":"Kazuaki Hiraga","user":"kazuakey","type":"user"},{"_id":"613b11ebd52a92d5044a16c7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/613b11ebd52a92d5044a16c7/zANxb3CeKcomWrXPouPVs.jpeg","isPro":false,"fullname":"Yan Varakin","user":"ZDPLI","type":"user"},{"_id":"6537ad851c6cecb1f935fe7d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/DABH07FpX5z1qhbzn3bJ8.jpeg","isPro":false,"fullname":"Ran Li","user":"Ran-Li","type":"user"},{"_id":"673c7319d11b1c2e246ead9c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/673c7319d11b1c2e246ead9c/IjFIO--N7Hm_BOEafhEQv.jpeg","isPro":false,"fullname":"Yang Shi","user":"DogNeverSleep","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">