https://yuzhou914.github.io/ConceptMaster/

\n","updatedAt":"2025-01-13T08:05:04.293Z","author":{"_id":"63468720dd6d90d82ccf3450","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg","fullname":"YSH","name":"BestWishYsh","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":64,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.4285845160484314},"editors":["BestWishYsh"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg"],"reactions":[],"isReport":false}},{"id":"6785bee6940b81b3526a3076","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-01-14T01:33:26.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [MotionCharacter: Identity-Preserving and Motion Controllable Human Video Generation](https://huggingface.co/papers/2411.18281) (2024)\n* [VividFace: A Diffusion-Based Hybrid Framework for High-Fidelity Video Face Swapping](https://huggingface.co/papers/2412.11279) (2024)\n* [MAKIMA: Tuning-free Multi-Attribute Open-domain Video Editing via Mask-Guided Attention Modulation](https://huggingface.co/papers/2412.19978) (2024)\n* [VIRES: Video Instance Repainting with Sketch and Text Guidance](https://huggingface.co/papers/2411.16199) (2024)\n* [DIVE: Taming DINO for Subject-Driven Video Editing](https://huggingface.co/papers/2412.03347) (2024)\n* [VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM](https://huggingface.co/papers/2501.00599) (2024)\n* [VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Control](https://huggingface.co/papers/2501.01427) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-01-14T01:33:26.192Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6870246529579163},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2501.04698","authors":[{"_id":"678479ad0e7d90facee7ebfe","user":{"_id":"6625114c11772517e5ebf349","avatarUrl":"/avatars/9ef727d5f2abacad6668a1996bbb2043.svg","isPro":false,"fullname":"Yuzhou Huang","user":"Yuzhou914","type":"user"},"name":"Yuzhou Huang","status":"admin_assigned","statusLastChangedAt":"2025-01-13T09:37:29.216Z","hidden":false},{"_id":"678479ad0e7d90facee7ebff","user":{"_id":"66c15837b186e4f6a0dac80c","avatarUrl":"/avatars/a3b75d6945f1608e64a2fcff887a5024.svg","isPro":false,"fullname":"Ziyang Yuan","user":"ziyangy","type":"user"},"name":"Ziyang Yuan","status":"admin_assigned","statusLastChangedAt":"2025-01-13T09:37:35.767Z","hidden":false},{"_id":"678479ad0e7d90facee7ec00","name":"Quande Liu","hidden":false},{"_id":"678479ad0e7d90facee7ec01","user":{"_id":"6347aca88af48bc2fb5416c5","avatarUrl":"/avatars/ee56e7985599314c6c7076300dbde884.svg","isPro":false,"fullname":"Wang Qiulin","user":"Qiulin","type":"user"},"name":"Qiulin Wang","status":"admin_assigned","statusLastChangedAt":"2025-01-13T09:37:54.749Z","hidden":false},{"_id":"678479ad0e7d90facee7ec02","user":{"_id":"60e272ca6c78a8c122b12127","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg","isPro":false,"fullname":"Xintao Wang","user":"Xintao","type":"user"},"name":"Xintao Wang","status":"admin_assigned","statusLastChangedAt":"2025-01-13T09:38:01.080Z","hidden":false},{"_id":"678479ad0e7d90facee7ec03","user":{"_id":"65324848220f490dbaa3b8c9","avatarUrl":"/avatars/cec7dcadf4657bb4f61e2c9a8075521b.svg","isPro":false,"fullname":"Zhang","user":"Ruimao","type":"user"},"name":"Ruimao Zhang","status":"admin_assigned","statusLastChangedAt":"2025-01-13T09:38:09.610Z","hidden":false},{"_id":"678479ad0e7d90facee7ec04","user":{"_id":"662f93942510ef5735d7ad00","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/662f93942510ef5735d7ad00/ZIDIPm63sncIHFTT5b0uR.png","isPro":false,"fullname":"magicwpf","user":"magicwpf","type":"user"},"name":"Pengfei Wan","status":"claimed_verified","statusLastChangedAt":"2025-03-17T14:42:55.038Z","hidden":false},{"_id":"678479ad0e7d90facee7ec05","user":{"_id":"64bce15bafd1e46c5504ad38","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64bce15bafd1e46c5504ad38/vkEjiu-mIagKlrXzDH75o.png","isPro":false,"fullname":"Di Zhang","user":"di-zhang-fdu","type":"user"},"name":"Di Zhang","status":"admin_assigned","statusLastChangedAt":"2025-01-13T09:38:41.879Z","hidden":true},{"_id":"678479ad0e7d90facee7ec06","name":"Kun Gai","hidden":false}],"publishedAt":"2025-01-08T18:59:01.000Z","submittedOnDailyAt":"2025-01-13T05:35:04.282Z","title":"ConceptMaster: Multi-Concept Video Customization on Diffusion\n Transformer Models Without Test-Time Tuning","submittedOnDailyBy":{"_id":"63468720dd6d90d82ccf3450","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg","isPro":false,"fullname":"YSH","user":"BestWishYsh","type":"user"},"summary":"Text-to-video generation has made remarkable advancements through diffusion\nmodels. However, Multi-Concept Video Customization (MCVC) remains a significant\nchallenge. We identify two key challenges in this task: 1) the identity\ndecoupling problem, where directly adopting existing customization methods\ninevitably mix attributes when handling multiple concepts simultaneously, and\n2) the scarcity of high-quality video-entity pairs, which is crucial for\ntraining such a model that represents and decouples various concepts well. To\naddress these challenges, we introduce ConceptMaster, an innovative framework\nthat effectively tackles the critical issues of identity decoupling while\nmaintaining concept fidelity in customized videos. Specifically, we introduce a\nnovel strategy of learning decoupled multi-concept embeddings that are injected\ninto the diffusion models in a standalone manner, which effectively guarantees\nthe quality of customized videos with multiple identities, even for highly\nsimilar visual concepts. To further overcome the scarcity of high-quality MCVC\ndata, we carefully establish a data construction pipeline, which enables\nsystematic collection of precise multi-concept video-entity data across diverse\nconcepts. A comprehensive benchmark is designed to validate the effectiveness\nof our model from three critical dimensions: concept fidelity, identity\ndecoupling ability, and video generation quality across six different concept\ncomposition scenarios. Extensive experiments demonstrate that our ConceptMaster\nsignificantly outperforms previous approaches for this task, paving the way for\ngenerating personalized and semantically accurate videos across multiple\nconcepts.","upvotes":15,"discussionId":"678479b00e7d90facee7ecfe","projectPage":"https://yuzhou914.github.io/ConceptMaster/","ai_summary":"ConceptMaster addresses identity decoupling and data scarcity in text-to-video generation using decoupled multi-concept embeddings and a comprehensive data pipeline, outperforming previous methods.","ai_keywords":["diffusion models","Multi-Concept Video Customization (MCVC)","identity decoupling","concept fidelity","multi-concept embeddings","high-quality video-entity pairs","data construction pipeline","concept composition","video generation quality"]},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"63468720dd6d90d82ccf3450","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg","isPro":false,"fullname":"YSH","user":"BestWishYsh","type":"user"},{"_id":"648eb1eb59c4e5c87dc116e0","avatarUrl":"/avatars/c636cea39c2c0937f01398c94ead5dad.svg","isPro":false,"fullname":"fdsqefsgergd","user":"T-representer","type":"user"},{"_id":"656ee8008bb9f4f8d95bd8f7","avatarUrl":"/avatars/4069d70f1279d928da521211c495d638.svg","isPro":false,"fullname":"Hyeonho Jeong","user":"hyeonho-jeong-video","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"65a4567e212d6aca9a3e8f5a","avatarUrl":"/avatars/ed944797230b5460381209bf76e4a0e4.svg","isPro":false,"fullname":"Catherine Liu","user":"Liu12uiL","type":"user"},{"_id":"64e567c9ddbefb63095a9662","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/F2BwrOU0XpzVI5nd-TL54.png","isPro":false,"fullname":"Bullard ","user":"Charletta1","type":"user"},{"_id":"6322cae4212b17c7728e7387","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6322cae4212b17c7728e7387/p-hJt0795EO_wlUVY39rL.jpeg","isPro":false,"fullname":"Chang Liu","user":"AlonzoLeeeooo","type":"user"},{"_id":"6683fc5344a65be1aab25dc0","avatarUrl":"/avatars/e13cde3f87b59e418838d702807df3b5.svg","isPro":false,"fullname":"hjkim","user":"hojie11","type":"user"},{"_id":"67136093d2e50f1e8c9fad52","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0q49MyGuav8lJ9CIeyLhu.png","isPro":false,"fullname":"Donghao Zhou","user":"donghao-zhou","type":"user"},{"_id":"64f955c582673b2a07fbf0ad","avatarUrl":"/avatars/1c98c8be61f6580c1e4ee698fa5c0716.svg","isPro":false,"fullname":"hongyu","user":"learn12138","type":"user"},{"_id":"663ccbff3a74a20189d4aa2e","avatarUrl":"/avatars/83a54455e0157480f65c498cd9057cf2.svg","isPro":false,"fullname":"Nguyen Van Thanh","user":"NguyenVanThanhHust","type":"user"},{"_id":"6773bc1c3cd7faf7e803ed15","avatarUrl":"/avatars/9978f7e8741fe0c24a96d7595a651190.svg","isPro":false,"fullname":"Yuan Xu","user":"TEXTHEn","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

Papers

arxiv:2501.04698

ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning

Published on Jan 8, 2025

· Submitted by

YSH on Jan 13, 2025

Upvote

Authors:

Yuzhou Huang ,

Ziyang Yuan ,

Qiulin Wang ,

Xintao Wang ,

Ruimao Zhang ,

Pengfei Wan ,

Di Zhang ,

Abstract

ConceptMaster addresses identity decoupling and data scarcity in text-to-video generation using decoupled multi-concept embeddings and a comprehensive data pipeline, outperforming previous methods.

AI-generated summary

Text-to-video generation has made remarkable advancements through diffusion models. However, Multi-Concept Video Customization (MCVC) remains a significant challenge. We identify two key challenges in this task: 1) the identity decoupling problem, where directly adopting existing customization methods inevitably mix attributes when handling multiple concepts simultaneously, and 2) the scarcity of high-quality video-entity pairs, which is crucial for training such a model that represents and decouples various concepts well. To address these challenges, we introduce ConceptMaster, an innovative framework that effectively tackles the critical issues of identity decoupling while maintaining concept fidelity in customized videos. Specifically, we introduce a novel strategy of learning decoupled multi-concept embeddings that are injected into the diffusion models in a standalone manner, which effectively guarantees the quality of customized videos with multiple identities, even for highly similar visual concepts. To further overcome the scarcity of high-quality MCVC data, we carefully establish a data construction pipeline, which enables systematic collection of precise multi-concept video-entity data across diverse concepts. A comprehensive benchmark is designed to validate the effectiveness of our model from three critical dimensions: concept fidelity, identity decoupling ability, and video generation quality across six different concept composition scenarios. Extensive experiments demonstrate that our ConceptMaster significantly outperforms previous approaches for this task, paving the way for generating personalized and semantically accurate videos across multiple concepts.