https://github.com/facebookresearch/DigiData
Website: https://facebookresearch.github.io/DigiData

\n","updatedAt":"2025-11-11T04:33:20.004Z","author":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","fullname":"taesiri","name":"taesiri","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":235,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.484051913022995},"editors":["taesiri"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg"],"reactions":[],"isReport":false}},{"id":"6913e4d585f8cc585ec5165a","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-11-12T01:37:25.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Hi-Agent: Hierarchical Vision-Language Agents for Mobile Device Control](https://huggingface.co/papers/2510.14388) (2025)\n* [Ferret-UI Lite: Lessons from Building Small On-Device GUI Agents](https://huggingface.co/papers/2509.26539) (2025)\n* [See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Toggles](https://huggingface.co/papers/2509.13615) (2025)\n* [Scaling Synthetic Task Generation for Agents via Exploration](https://huggingface.co/papers/2509.25047) (2025)\n* [LightAgent: Mobile Agentic Foundation Models](https://huggingface.co/papers/2510.22009) (2025)\n* [Retrieval-augmented GUI Agents with Generative Guidelines](https://huggingface.co/papers/2509.24183) (2025)\n* [UIPro: Unleashing Superior Interaction Capability For GUI Agents](https://huggingface.co/papers/2509.17328) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-11-12T01:37:25.189Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7121571898460388},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2511.07413","authors":[{"_id":"6912bc52a644ba07c499c7b0","user":{"_id":"665e28741d30854dbbf62046","avatarUrl":"/avatars/78b7291107eff7180031094103bd0a8b.svg","isPro":false,"fullname":"Yuxuan Sun","user":"yuxuans","type":"user"},"name":"Yuxuan Sun","status":"claimed_verified","statusLastChangedAt":"2025-11-11T19:43:39.339Z","hidden":false},{"_id":"6912bc52a644ba07c499c7b1","name":"Manchen Wang","hidden":false},{"_id":"6912bc52a644ba07c499c7b2","user":{"_id":"6445dc7e219dc6142a6a6861","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6445dc7e219dc6142a6a6861/Ak_YV8V1NZpkF5AQnJVlJ.png","isPro":false,"fullname":"Shengyi Qian","user":"shengyi-qian","type":"user"},"name":"Shengyi Qian","status":"claimed_verified","statusLastChangedAt":"2025-11-11T19:54:33.507Z","hidden":false},{"_id":"6912bc52a644ba07c499c7b3","name":"William R. Wong","hidden":false},{"_id":"6912bc52a644ba07c499c7b4","name":"Eric Gan","hidden":false},{"_id":"6912bc52a644ba07c499c7b5","name":"Pierluca D'Oro","hidden":false},{"_id":"6912bc52a644ba07c499c7b6","name":"Alejandro Castillejo Munoz","hidden":false},{"_id":"6912bc52a644ba07c499c7b7","name":"Sneha Silwal","hidden":false},{"_id":"6912bc52a644ba07c499c7b8","name":"Pedro Matias","hidden":false},{"_id":"6912bc52a644ba07c499c7b9","name":"Nitin Kamra","hidden":false},{"_id":"6912bc52a644ba07c499c7ba","name":"Satwik Kottur","hidden":false},{"_id":"6912bc52a644ba07c499c7bb","name":"Nick Raines","hidden":false},{"_id":"6912bc52a644ba07c499c7bc","name":"Xuanyi Zhao","hidden":false},{"_id":"6912bc52a644ba07c499c7bd","name":"Joy Chen","hidden":false},{"_id":"6912bc52a644ba07c499c7be","name":"Joseph Greer","hidden":false},{"_id":"6912bc52a644ba07c499c7bf","name":"Andrea Madotto","hidden":false},{"_id":"6912bc52a644ba07c499c7c0","name":"Allen Bolourchi","hidden":false},{"_id":"6912bc52a644ba07c499c7c1","name":"James Valori","hidden":false},{"_id":"6912bc52a644ba07c499c7c2","name":"Kevin Carlberg","hidden":false},{"_id":"6912bc52a644ba07c499c7c3","name":"Karl Ridgeway","hidden":false},{"_id":"6912bc52a644ba07c499c7c4","name":"Joseph Tighe","hidden":false}],"publishedAt":"2025-11-10T18:57:35.000Z","submittedOnDailyAt":"2025-11-11T02:03:02.991Z","title":"DigiData: Training and Evaluating General-Purpose Mobile Control Agents","submittedOnDailyBy":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},"summary":"AI agents capable of controlling user interfaces have the potential to\ntransform human interaction with digital devices. To accelerate this\ntransformation, two fundamental building blocks are essential: high-quality\ndatasets that enable agents to achieve complex and human-relevant goals, and\nrobust evaluation methods that allow researchers and practitioners to rapidly\nenhance agent performance. In this paper, we introduce DigiData, a large-scale,\nhigh-quality, diverse, multi-modal dataset designed for training mobile control\nagents. Unlike existing datasets, which derive goals from unstructured\ninteractions, DigiData is meticulously constructed through comprehensive\nexploration of app features, resulting in greater diversity and higher goal\ncomplexity. Additionally, we present DigiData-Bench, a benchmark for evaluating\nmobile control agents on real-world complex tasks. We demonstrate that the\ncommonly used step-accuracy metric falls short in reliably assessing mobile\ncontrol agents and, to address this, we propose dynamic evaluation protocols\nand AI-powered evaluations as rigorous alternatives for agent assessment. Our\ncontributions aim to significantly advance the development of mobile control\nagents, paving the way for more intuitive and effective human-device\ninteractions.","upvotes":6,"discussionId":"6912bc53a644ba07c499c7c5","projectPage":"https://facebookresearch.github.io/DigiData/","githubRepo":"https://github.com/facebookresearch/digidata","githubRepoAddedBy":"user","ai_summary":"DigiData and DigiData-Bench advance mobile control agents by providing a diverse, high-quality dataset and dynamic evaluation protocols, respectively.","ai_keywords":["mobile control agents","DigiData","DigiData-Bench","step-accuracy metric","dynamic evaluation protocols","AI-powered evaluations"],"githubStars":10,"organization":{"_id":"5e63d8713071d5be688861b8","name":"facebook","fullname":"AI at Meta","avatar":"https://cdn-uploads.huggingface.co/production/uploads/1592839207516-noauth.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"665e28741d30854dbbf62046","avatarUrl":"/avatars/78b7291107eff7180031094103bd0a8b.svg","isPro":false,"fullname":"Yuxuan Sun","user":"yuxuans","type":"user"},{"_id":"6445dc7e219dc6142a6a6861","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6445dc7e219dc6142a6a6861/Ak_YV8V1NZpkF5AQnJVlJ.png","isPro":false,"fullname":"Shengyi Qian","user":"shengyi-qian","type":"user"},{"_id":"631e14ac473a6825f285e89d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/631e14ac473a6825f285e89d/K-6QnoeGLg8XFvbTMMdqA.jpeg","isPro":false,"fullname":"Yury Panikov","user":"panikov","type":"user"},{"_id":"686db5d4af2b856fabbf13aa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/6BjMv2LVNoqvbX8fQSTPI.png","isPro":false,"fullname":"V bbbb","user":"Bbbbbnnn","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0,"organization":{"_id":"5e63d8713071d5be688861b8","name":"facebook","fullname":"AI at Meta","avatar":"https://cdn-uploads.huggingface.co/production/uploads/1592839207516-noauth.png"}}">

Papers

arxiv:2511.07413

DigiData: Training and Evaluating General-Purpose Mobile Control Agents

Published on Nov 10, 2025

· Submitted by

taesiri on Nov 11, 2025

AI at Meta

Upvote

Authors:

Yuxuan Sun ,

Shengyi Qian ,

Abstract

DigiData and DigiData-Bench advance mobile control agents by providing a diverse, high-quality dataset and dynamic evaluation protocols, respectively.

AI-generated summary

AI agents capable of controlling user interfaces have the potential to transform human interaction with digital devices. To accelerate this transformation, two fundamental building blocks are essential: high-quality datasets that enable agents to achieve complex and human-relevant goals, and robust evaluation methods that allow researchers and practitioners to rapidly enhance agent performance. In this paper, we introduce DigiData, a large-scale, high-quality, diverse, multi-modal dataset designed for training mobile control agents. Unlike existing datasets, which derive goals from unstructured interactions, DigiData is meticulously constructed through comprehensive exploration of app features, resulting in greater diversity and higher goal complexity. Additionally, we present DigiData-Bench, a benchmark for evaluating mobile control agents on real-world complex tasks. We demonstrate that the commonly used step-accuracy metric falls short in reliably assessing mobile control agents and, to address this, we propose dynamic evaluation protocols and AI-powered evaluations as rigorous alternatives for agent assessment. Our contributions aim to significantly advance the development of mobile control agents, paving the way for more intuitive and effective human-device interactions.

View arXiv page View PDF Project page GitHub 10 Add to collection

Community

taesiri

Paper submitter Nov 11, 2025

DigiData introduces a large, diverse multi-modal dataset and benchmark for training and evaluating mobile control agents with robust dynamic evaluation methods beyond step accuracy.