https://connoriginal.github.io/MEMENTO/

\n","updatedAt":"2025-05-27T03:50:10.075Z","author":{"_id":"636b529ef796304dd67d139c","avatarUrl":"/avatars/7a64d5095fcb1da558b52ad48177ad76.svg","fullname":"Taeyoon Kwon","name":"Connoriginal","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6972867846488953},"editors":["Connoriginal"],"editorAvatarUrls":["/avatars/7a64d5095fcb1da558b52ad48177ad76.svg"],"reactions":[],"isReport":false}},{"id":"6836686cf5740a450b1a541f","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-05-28T01:35:40.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Reasoning LLMs for User-Aware Multimodal Conversational Agents](https://huggingface.co/papers/2504.01700) (2025)\n* [Reasoning Meets Personalization: Unleashing the Potential of Large Reasoning Model for Personalized Generation](https://huggingface.co/papers/2505.17571) (2025)\n* [LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Robotics](https://huggingface.co/papers/2504.21716) (2025)\n* [Aux-Think: Exploring Reasoning Strategies for Data-Efficient Vision-Language Navigation](https://huggingface.co/papers/2505.11886) (2025)\n* [Beyond Needle(s) in the Embodied Haystack: Environment, Architecture, and Training Considerations for Long Context Reasoning](https://huggingface.co/papers/2505.16928) (2025)\n* [A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations](https://huggingface.co/papers/2505.14106) (2025)\n* [A Survey of Personalization: From RAG to Agent](https://huggingface.co/papers/2504.10147) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-05-28T01:35:40.450Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7553896307945251},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2505.16348","authors":[{"_id":"6835365d2925bc8bb23a57c7","user":{"_id":"636b529ef796304dd67d139c","avatarUrl":"/avatars/7a64d5095fcb1da558b52ad48177ad76.svg","isPro":false,"fullname":"Taeyoon Kwon","user":"Connoriginal","type":"user"},"name":"Taeyoon Kwon","status":"claimed_verified","statusLastChangedAt":"2025-05-27T07:50:26.210Z","hidden":false},{"_id":"6835365d2925bc8bb23a57c8","user":{"_id":"654c263fbe11400417c93d9f","avatarUrl":"/avatars/eb5778e28091200efee2c6b68589a1a2.svg","isPro":false,"fullname":"choi dongwook","user":"dongwookchoi","type":"user"},"name":"Dongwook Choi","status":"claimed_verified","statusLastChangedAt":"2025-06-16T13:54:37.749Z","hidden":false},{"_id":"6835365d2925bc8bb23a57c9","name":"Sunghwan Kim","hidden":false},{"_id":"6835365d2925bc8bb23a57ca","name":"Hyojun Kim","hidden":false},{"_id":"6835365d2925bc8bb23a57cb","user":{"_id":"6420f4f55bccaa42484496e5","avatarUrl":"/avatars/4996ba26955f8423c946b1ecd3989964.svg","isPro":false,"fullname":"Seung Jun Moon","user":"Lune-Blue","type":"user"},"name":"Seungjun Moon","status":"admin_assigned","statusLastChangedAt":"2025-05-27T08:24:40.306Z","hidden":false},{"_id":"6835365d2925bc8bb23a57cc","user":{"_id":"64b72a408ba7d6c922c73054","avatarUrl":"/avatars/6d9797430bc36f05fb950b84aa6a9374.svg","isPro":false,"fullname":"Beong Woo Kwak","user":"bwookwak","type":"user"},"name":"Beong-woo Kwak","status":"admin_assigned","statusLastChangedAt":"2025-05-27T08:24:46.911Z","hidden":false},{"_id":"6835365d2925bc8bb23a57cd","user":{"_id":"658a57b4126b8d7eae07b983","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/658a57b4126b8d7eae07b983/reZrbaObwq_kNVsuTR_iR.jpeg","isPro":false,"fullname":"Kuan-Hao Huang","user":"kuanhaoh","type":"user"},"name":"Kuan-Hao Huang","status":"admin_assigned","statusLastChangedAt":"2025-05-27T08:24:53.502Z","hidden":false},{"_id":"6835365d2925bc8bb23a57ce","user":{"_id":"682e91865fa8c5df85b3d8e5","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/XWTfZoOjCdMnqaFEBBYWe.png","isPro":false,"fullname":"Jinyoung Yeo","user":"jinyeo","type":"user"},"name":"Jinyoung Yeo","status":"admin_assigned","statusLastChangedAt":"2025-05-27T08:25:01.610Z","hidden":false}],"publishedAt":"2025-05-22T08:00:10.000Z","submittedOnDailyAt":"2025-05-27T02:20:10.067Z","title":"Embodied Agents Meet Personalization: Exploring Memory Utilization for\n Personalized Assistance","submittedOnDailyBy":{"_id":"636b529ef796304dd67d139c","avatarUrl":"/avatars/7a64d5095fcb1da558b52ad48177ad76.svg","isPro":false,"fullname":"Taeyoon Kwon","user":"Connoriginal","type":"user"},"summary":"Embodied agents empowered by large language models (LLMs) have shown strong\nperformance in household object rearrangement tasks. However, these tasks\nprimarily focus on single-turn interactions with simplified instructions, which\ndo not truly reflect the challenges of providing meaningful assistance to\nusers. To provide personalized assistance, embodied agents must understand the\nunique semantics that users assign to the physical world (e.g., favorite cup,\nbreakfast routine) by leveraging prior interaction history to interpret\ndynamic, real-world instructions. Yet, the effectiveness of embodied agents in\nutilizing memory for personalized assistance remains largely underexplored. To\naddress this gap, we present MEMENTO, a personalized embodied agent evaluation\nframework designed to comprehensively assess memory utilization capabilities to\nprovide personalized assistance. Our framework consists of a two-stage memory\nevaluation process design that enables quantifying the impact of memory\nutilization on task performance. This process enables the evaluation of agents'\nunderstanding of personalized knowledge in object rearrangement tasks by\nfocusing on its role in goal interpretation: (1) the ability to identify target\nobjects based on personal meaning (object semantics), and (2) the ability to\ninfer object-location configurations from consistent user patterns, such as\nroutines (user patterns). Our experiments across various LLMs reveal\nsignificant limitations in memory utilization, with even frontier models like\nGPT-4o experiencing a 30.5% performance drop when required to reference\nmultiple memories, particularly in tasks involving user patterns. These\nfindings, along with our detailed analyses and case studies, provide valuable\ninsights for future research in developing more effective personalized embodied\nagents. Project website: https://connoriginal.github.io/MEMENTO","upvotes":52,"discussionId":"683536612925bc8bb23a58e1","projectPage":"https://connoriginal.github.io/MEMENTO/","githubRepo":"https://github.com/Connoriginal/MEMENTO","githubRepoAddedBy":"user","ai_summary":"MEMENTO evaluates personalized memory utilization in embodied agents, revealing limitations in understanding user semantics and routines.","ai_keywords":["embodied agents","large language models (LLMs)","object rearrangement tasks","user semantics","prior interaction history","memory utilization","personalized assistance","goal interpretation","object semantics","user patterns"],"githubStars":17},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64b70a18be30d6567a6baa72","avatarUrl":"/avatars/c1296ca88195332aef82a6a0d89a7de6.svg","isPro":false,"fullname":"minju kim","user":"min942773","type":"user"},{"_id":"642d4f092320338df27ceb15","avatarUrl":"/avatars/b522613d2b3ddfcc2fa3a5a1a2500c86.svg","isPro":false,"fullname":"Wing.D","user":"Wingu","type":"user"},{"_id":"646a0897c37ca1e12308b026","avatarUrl":"/avatars/6d720a9e366db9bec15c8c10878c0c75.svg","isPro":false,"fullname":"Sunghwan Kim","user":"KimSHine","type":"user"},{"_id":"6826114ba47cb2b8762289aa","avatarUrl":"/avatars/a232061498fe967b3ffb26dbbf10d1c0.svg","isPro":false,"fullname":"ToolHaystack","user":"ToolHaystack","type":"user"},{"_id":"636b529ef796304dd67d139c","avatarUrl":"/avatars/7a64d5095fcb1da558b52ad48177ad76.svg","isPro":false,"fullname":"Taeyoon Kwon","user":"Connoriginal","type":"user"},{"_id":"654c263fbe11400417c93d9f","avatarUrl":"/avatars/eb5778e28091200efee2c6b68589a1a2.svg","isPro":false,"fullname":"choi dongwook","user":"dongwookchoi","type":"user"},{"_id":"66bb326534295e9cf08df4e2","avatarUrl":"/avatars/5dc3225be1194467b30691b5d33c7b19.svg","isPro":false,"fullname":"Gyeom hwangbo","user":"aerojohn1223","type":"user"},{"_id":"6811d91633558457e1c2c7e0","avatarUrl":"/avatars/9e28daed4fd12bce6ced81d3e1d295f3.svg","isPro":false,"fullname":"Yeonjun Hwang","user":"hbhhyj","type":"user"},{"_id":"65238893cfbef01dc7c97bb1","avatarUrl":"/avatars/1227009b5010ee0a1823a603fe753caa.svg","isPro":false,"fullname":"iruno","user":"iruno","type":"user"},{"_id":"6819b8eff0afe451dd8b714b","avatarUrl":"/avatars/f05a6b8dc680544c4545fe9cded2954d.svg","isPro":false,"fullname":"SoohyunOh","user":"oceann010315","type":"user"},{"_id":"64c8f4cec547ed5243ebd0a8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64c8f4cec547ed5243ebd0a8/MiOH5YbMg8Gh9KYlQsLmX.jpeg","isPro":false,"fullname":"Hyungjoo Chae","user":"hyungjoochae","type":"user"},{"_id":"64184d05db24526c7c9cbef5","avatarUrl":"/avatars/b71e28a09290ae0929888187485b296a.svg","isPro":false,"fullname":"vive kang","user":"Vive-kang","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

Papers

arxiv:2505.16348

Embodied Agents Meet Personalization: Exploring Memory Utilization for Personalized Assistance

Published on May 22, 2025

· Submitted by

Taeyoon Kwon on May 27, 2025

Upvote

Authors:

Taeyoon Kwon ,

Dongwook Choi ,

Seungjun Moon ,

Beong-woo Kwak ,

Kuan-Hao Huang ,

Jinyoung Yeo

Abstract

MEMENTO evaluates personalized memory utilization in embodied agents, revealing limitations in understanding user semantics and routines.

AI-generated summary

Embodied agents empowered by large language models (LLMs) have shown strong performance in household object rearrangement tasks. However, these tasks primarily focus on single-turn interactions with simplified instructions, which do not truly reflect the challenges of providing meaningful assistance to users. To provide personalized assistance, embodied agents must understand the unique semantics that users assign to the physical world (e.g., favorite cup, breakfast routine) by leveraging prior interaction history to interpret dynamic, real-world instructions. Yet, the effectiveness of embodied agents in utilizing memory for personalized assistance remains largely underexplored. To address this gap, we present MEMENTO, a personalized embodied agent evaluation framework designed to comprehensively assess memory utilization capabilities to provide personalized assistance. Our framework consists of a two-stage memory evaluation process design that enables quantifying the impact of memory utilization on task performance. This process enables the evaluation of agents' understanding of personalized knowledge in object rearrangement tasks by focusing on its role in goal interpretation: (1) the ability to identify target objects based on personal meaning (object semantics), and (2) the ability to infer object-location configurations from consistent user patterns, such as routines (user patterns). Our experiments across various LLMs reveal significant limitations in memory utilization, with even frontier models like GPT-4o experiencing a 30.5% performance drop when required to reference multiple memories, particularly in tasks involving user patterns. These findings, along with our detailed analyses and case studies, provide valuable insights for future research in developing more effective personalized embodied agents. Project website: https://connoriginal.github.io/MEMENTO