Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /home/zhenxiangba/zhenxiangba.com/public_html/phproxy-improved-master/index.php on line 456 Paper page - End-to-End Video Character Replacement without Structural Guidance
Please give a thumbs up to this comment if you found it helpful!
\n
If you want recommendations for any Paper on Hugging Face checkout this Space
\n
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
\n","updatedAt":"2026-01-15T01:37:00.198Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.679844081401825},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2601.08587","authors":[{"_id":"696705cec5e371f6b235d0f4","name":"Zhengbo Xu","hidden":false},{"_id":"696705cec5e371f6b235d0f5","user":{"_id":"641ab8b0a5f876fe30c1ff87","avatarUrl":"/avatars/a95544619ef05814d58e2b2b8581f223.svg","isPro":false,"fullname":"jiema","user":"unrealMJ","type":"user"},"name":"Jie Ma","status":"claimed_verified","statusLastChangedAt":"2026-01-14T09:51:30.285Z","hidden":false},{"_id":"696705cec5e371f6b235d0f6","name":"Ziheng Wang","hidden":false},{"_id":"696705cec5e371f6b235d0f7","name":"Zhan Peng","hidden":false},{"_id":"696705cec5e371f6b235d0f8","user":{"_id":"6455afeabda0fbba412d4922","avatarUrl":"/avatars/367731ce1c71d1e19ff415a52ae4067d.svg","isPro":false,"fullname":"Jun Liang","user":"utopiar","type":"user"},"name":"Jun Liang","status":"claimed_verified","statusLastChangedAt":"2026-01-14T09:51:28.253Z","hidden":false},{"_id":"696705cec5e371f6b235d0f9","name":"Jing Li","hidden":false}],"publishedAt":"2026-01-13T14:10:34.000Z","submittedOnDailyAt":"2026-01-14T00:27:48.852Z","title":"End-to-End Video Character Replacement without Structural Guidance","submittedOnDailyBy":{"_id":"640d3eaa3623f6a56dde856d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1678589663024-640d3eaa3623f6a56dde856d.jpeg","isPro":true,"fullname":"vansin","user":"vansin","type":"user"},"summary":"Controllable video character replacement with a user-provided identity remains a challenging problem due to the lack of paired video data. Prior works have predominantly relied on a reconstruction-based paradigm that requires per-frame segmentation masks and explicit structural guidance (e.g., skeleton, depth). This reliance, however, severely limits their generalizability in complex scenarios involving occlusions, character-object interactions, unusual poses, or challenging illumination, often leading to visual artifacts and temporal inconsistencies. In this paper, we propose MoCha, a pioneering framework that bypasses these limitations by requiring only a single arbitrary frame mask. To effectively adapt the multi-modal input condition and enhance facial identity, we introduce a condition-aware RoPE and employ an RL-based post-training stage. Furthermore, to overcome the scarcity of qualified paired-training data, we propose a comprehensive data construction pipeline. Specifically, we design three specialized datasets: a high-fidelity rendered dataset built with Unreal Engine 5 (UE5), an expression-driven dataset synthesized by current portrait animation techniques, and an augmented dataset derived from existing video-mask pairs. Extensive experiments demonstrate that our method substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research. Please refer to our project page for more details: orange-3dv-team.github.io/MoCha","upvotes":8,"discussionId":"696705cfc5e371f6b235d0fa","projectPage":"https://orange-3dv-team.github.io/MoCha/","githubRepo":"https://github.com/Orange-3DV-Team/MoCha","githubRepoAddedBy":"user","ai_summary":"MoCha enables controllable video character replacement using a single frame mask through condition-aware RoPE and a comprehensive data construction pipeline with specialized datasets.","ai_keywords":["controllable video character replacement","reconstruction-based paradigm","per-frame segmentation masks","structural guidance","facial identity","condition-aware RoPE","RL-based post-training","data construction pipeline","high-fidelity rendered dataset","expression-driven dataset","augmented dataset"],"githubStars":640,"organization":{"_id":"69670c472ad4f2c0e892575c","name":"Orange-Team","fullname":"Orange Team","avatar":"https://cdn-uploads.huggingface.co/production/uploads/6455afeabda0fbba412d4922/Sy7zZn0kb-Q1SCSUwOn-9.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"641ab8b0a5f876fe30c1ff87","avatarUrl":"/avatars/a95544619ef05814d58e2b2b8581f223.svg","isPro":false,"fullname":"jiema","user":"unrealMJ","type":"user"},{"_id":"63039a1c0547362a22a384df","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63039a1c0547362a22a384df/FywK_6NACqVlf9CP7Ty-d.png","isPro":true,"fullname":"Dopamine","user":"CCP6","type":"user"},{"_id":"691f436d6101ec369272dc4d","avatarUrl":"/avatars/8dfb7931694202a35757ca576439f493.svg","isPro":false,"fullname":"bao","user":"jijibaojiji123","type":"user"},{"_id":"68f5dbffcd4766046036f33d","avatarUrl":"/avatars/51309c996abd0520fe95bd4fa8901659.svg","isPro":false,"fullname":"Orange-3DV-Team","user":"Orange-3DV-Team","type":"user"},{"_id":"6351e5bb3734c6e8a5c1bec1","avatarUrl":"/avatars/a784a51b369b197398575c3afbd5ceab.svg","isPro":false,"fullname":"Han-Bit Kang","user":"hbkang","type":"user"},{"_id":"62f087ed019f0a3ba2c45fe5","avatarUrl":"/avatars/d1fc9d4f804696d8d605420b2ea55137.svg","isPro":false,"fullname":"ouasdg","user":"ouasdg","type":"user"},{"_id":"6455afeabda0fbba412d4922","avatarUrl":"/avatars/367731ce1c71d1e19ff415a52ae4067d.svg","isPro":false,"fullname":"Jun Liang","user":"utopiar","type":"user"},{"_id":"686db5d4af2b856fabbf13aa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/6BjMv2LVNoqvbX8fQSTPI.png","isPro":false,"fullname":"V bbbb","user":"Bbbbbnnn","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0,"organization":{"_id":"69670c472ad4f2c0e892575c","name":"Orange-Team","fullname":"Orange Team","avatar":"https://cdn-uploads.huggingface.co/production/uploads/6455afeabda0fbba412d4922/Sy7zZn0kb-Q1SCSUwOn-9.png"}}">
MoCha enables controllable video character replacement using a single frame mask through condition-aware RoPE and a comprehensive data construction pipeline with specialized datasets.
AI-generated summary
Controllable video character replacement with a user-provided identity remains a challenging problem due to the lack of paired video data. Prior works have predominantly relied on a reconstruction-based paradigm that requires per-frame segmentation masks and explicit structural guidance (e.g., skeleton, depth). This reliance, however, severely limits their generalizability in complex scenarios involving occlusions, character-object interactions, unusual poses, or challenging illumination, often leading to visual artifacts and temporal inconsistencies. In this paper, we propose MoCha, a pioneering framework that bypasses these limitations by requiring only a single arbitrary frame mask. To effectively adapt the multi-modal input condition and enhance facial identity, we introduce a condition-aware RoPE and employ an RL-based post-training stage. Furthermore, to overcome the scarcity of qualified paired-training data, we propose a comprehensive data construction pipeline. Specifically, we design three specialized datasets: a high-fidelity rendered dataset built with Unreal Engine 5 (UE5), an expression-driven dataset synthesized by current portrait animation techniques, and an augmented dataset derived from existing video-mask pairs. Extensive experiments demonstrate that our method substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research. Please refer to our project page for more details: orange-3dv-team.github.io/MoCha