Librarian Bot. I found the following papers similar to this paper.

\n

The following papers were recommended by the Semantic Scholar API

\n

\n
• Unified Video Action Model (2025)
\n
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy (2025)
\n
• HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model (2025)
\n
• FP3: A 3D Foundation Policy for Robotic Manipulation (2025)
\n
• Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success (2025)
\n
• PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planning (2025)
\n
• CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models (2025)
\n

\n

Please give a thumbs up to this comment if you found it helpful!

\n

If you want recommendations for any Paper on Hugging Face checkout this Space

\n

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-04-10T01:34:22.427Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6323785781860352},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2504.02792","authors":[{"_id":"67f713dbb27f2c52f771fcb5","name":"Chuning Zhu","hidden":false},{"_id":"67f713dbb27f2c52f771fcb6","user":{"_id":"66fc85c98f3b78428e37be61","avatarUrl":"/avatars/4b026e4d13e3d6f910533e84c2de040b.svg","isPro":false,"fullname":"Raymond Yu","user":"raymondyu","type":"user"},"name":"Raymond Yu","status":"claimed_verified","statusLastChangedAt":"2025-04-11T07:26:05.361Z","hidden":false},{"_id":"67f713dbb27f2c52f771fcb7","name":"Siyuan Feng","hidden":false},{"_id":"67f713dbb27f2c52f771fcb8","name":"Benjamin Burchfiel","hidden":false},{"_id":"67f713dbb27f2c52f771fcb9","name":"Paarth Shah","hidden":false},{"_id":"67f713dbb27f2c52f771fcba","name":"Abhishek Gupta","hidden":false}],"publishedAt":"2025-04-03T17:38:59.000Z","submittedOnDailyAt":"2025-04-09T23:12:55.499Z","title":"Unified World Models: Coupling Video and Action Diffusion for\n Pretraining on Large Robotic Datasets","submittedOnDailyBy":{"_id":"6494c76b3f3a5da8043e1198","avatarUrl":"/avatars/3af6620b938dd9dc7079c893b7a23768.svg","isPro":false,"fullname":"Chuning Zhu","user":"zchuning","type":"user"},"summary":"Imitation learning has emerged as a promising approach towards building\ngeneralist robots. However, scaling imitation learning for large robot\nfoundation models remains challenging due to its reliance on high-quality\nexpert demonstrations. Meanwhile, large amounts of video data depicting a wide\nrange of environments and diverse behaviors are readily available. This data\nprovides a rich source of information about real-world dynamics and\nagent-environment interactions. Leveraging this data directly for imitation\nlearning, however, has proven difficult due to the lack of action annotation\nrequired for most contemporary methods. In this work, we present Unified World\nModels (UWM), a framework that allows for leveraging both video and action data\nfor policy learning. Specifically, a UWM integrates an action diffusion process\nand a video diffusion process within a unified transformer architecture, where\nindependent diffusion timesteps govern each modality. We show that by simply\ncontrolling each diffusion timestep, UWM can flexibly represent a policy, a\nforward dynamics, an inverse dynamics, and a video generator. Through simulated\nand real-world experiments, we show that: (1) UWM enables effective pretraining\non large-scale multitask robot datasets with both dynamics and action\npredictions, resulting in more generalizable and robust policies than imitation\nlearning, (2) UWM naturally facilitates learning from action-free video data\nthrough independent control of modality-specific diffusion timesteps, further\nimproving the performance of finetuned policies. Our results suggest that UWM\noffers a promising step toward harnessing large, heterogeneous datasets for\nscalable robot learning, and provides a simple unification between the often\ndisparate paradigms of imitation learning and world modeling. Videos and code\nare available at https://weirdlabuw.github.io/uwm/.","upvotes":4,"discussionId":"67f713ddb27f2c52f771fd3d","projectPage":"https://weirdlabuw.github.io/uwm/","githubRepo":"https://github.com/WEIRDLabUW/unified-world-model","githubRepoAddedBy":"user","ai_summary":"Unified World Models leverage both video and action data within a unified transformer architecture to enable effective pretraining on large-scale multitask robot datasets and improve policy generalization and robustness.","ai_keywords":["Unified World Models (UWM)","action diffusion process","video diffusion process","unified transformer architecture","diffusion timesteps","policy learning","forward dynamics","inverse dynamics","video generator","pretraining","multitask robot datasets","policy generalization","policy robustness"],"githubStars":188},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6494c76b3f3a5da8043e1198","avatarUrl":"/avatars/3af6620b938dd9dc7079c893b7a23768.svg","isPro":false,"fullname":"Chuning Zhu","user":"zchuning","type":"user"},{"_id":"66fc85c98f3b78428e37be61","avatarUrl":"/avatars/4b026e4d13e3d6f910533e84c2de040b.svg","isPro":false,"fullname":"Raymond Yu","user":"raymondyu","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"634aab35dcf125e4dafc87b1","avatarUrl":"/avatars/aa2db84fd423e9eefe3ef3167c9d3999.svg","isPro":false,"fullname":"YangXiuyu","user":"gzzyyxy","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">