https://arxiv.org/abs/2503.01774
Website: https://research.nvidia.com/labs/toronto-ai/difix3d

\n","updatedAt":"2025-03-04T05:53:48.756Z","author":{"_id":"633aaf695df91da9cea92960","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633aaf695df91da9cea92960/h1oOWUudbdlsd63Q5w0hM.png","fullname":"Jay Wu","name":"jayw","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":15,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.42840439081192017},"editors":["jayw"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/633aaf695df91da9cea92960/h1oOWUudbdlsd63Q5w0hM.png"],"reactions":[],"isReport":false}},{"id":"67c8fc54b53ec5f03f352a2c","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-03-06T01:37:24.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting](https://huggingface.co/papers/2501.06714) (2025)\n* [Pointmap-Conditioned Diffusion for Consistent Novel View Synthesis](https://huggingface.co/papers/2501.02913) (2025)\n* [Enhancing Monocular 3D Scene Completion with Diffusion Model](https://huggingface.co/papers/2503.00726) (2025)\n* [Kiss3DGen: Repurposing Image Diffusion Models for 3D Asset Generation](https://huggingface.co/papers/2503.01370) (2025)\n* [Instructive3D: Editing Large Reconstruction Models with Text Instructions](https://huggingface.co/papers/2501.04374) (2025)\n* [MEt3R: Measuring Multi-View Consistency in Generated Images](https://huggingface.co/papers/2501.06336) (2025)\n* [Synthesizing Consistent Novel Views via 3D Epipolar Attention without Re-Training](https://huggingface.co/papers/2502.18219) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-03-06T01:37:24.955Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7007511854171753},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2503.01774","authors":[{"_id":"67c694febdab31ec59fea175","user":{"_id":"633aaf695df91da9cea92960","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633aaf695df91da9cea92960/h1oOWUudbdlsd63Q5w0hM.png","isPro":false,"fullname":"Jay Wu","user":"jayw","type":"user"},"name":"Jay Zhangjie Wu","status":"claimed_verified","statusLastChangedAt":"2025-03-04T08:34:53.874Z","hidden":false},{"_id":"67c694febdab31ec59fea176","name":"Yuxuan Zhang","hidden":false},{"_id":"67c694febdab31ec59fea177","user":{"_id":"656e000253703dd78fd072a9","avatarUrl":"/avatars/6702ba8fabe3d08884aa757f90cea333.svg","isPro":false,"fullname":"Haithem Turki","user":"hturki","type":"user"},"name":"Haithem Turki","status":"admin_assigned","statusLastChangedAt":"2025-03-04T09:13:26.878Z","hidden":false},{"_id":"67c694febdab31ec59fea178","user":{"_id":"658529d61c461dfe88afe8e8","avatarUrl":"/avatars/a22c1b07d28c2662833c462c6537d835.svg","isPro":false,"fullname":"Xuanchi Ren","user":"xrenaa","type":"user"},"name":"Xuanchi Ren","status":"admin_assigned","statusLastChangedAt":"2025-03-04T09:13:33.467Z","hidden":false},{"_id":"67c694febdab31ec59fea179","name":"Jun Gao","hidden":false},{"_id":"67c694febdab31ec59fea17a","user":{"_id":"661ab3da2b14565c7acccf5c","avatarUrl":"/avatars/fa4fc03664803e02aede4d4c3d50b393.svg","isPro":false,"fullname":"Mike Zheng Shou","user":"AnalMom","type":"user"},"name":"Mike Zheng Shou","status":"admin_assigned","statusLastChangedAt":"2025-03-04T09:27:21.825Z","hidden":false},{"_id":"67c694febdab31ec59fea17b","name":"Sanja Fidler","hidden":false},{"_id":"67c694febdab31ec59fea17c","user":{"_id":"6366cda3361a96184dc22139","avatarUrl":"/avatars/d8a88c84cb5f69e69dd038674a29be89.svg","isPro":false,"fullname":"Zan Gojcic","user":"zgojcic","type":"user"},"name":"Zan Gojcic","status":"admin_assigned","statusLastChangedAt":"2025-03-04T09:27:34.034Z","hidden":false},{"_id":"67c694febdab31ec59fea17d","name":"Huan Ling","hidden":false}],"publishedAt":"2025-03-03T17:58:33.000Z","submittedOnDailyAt":"2025-03-04T03:22:22.204Z","title":"Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models","submittedOnDailyBy":{"_id":"633aaf695df91da9cea92960","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633aaf695df91da9cea92960/h1oOWUudbdlsd63Q5w0hM.png","isPro":false,"fullname":"Jay Wu","user":"jayw","type":"user"},"summary":"Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D\nreconstruction and novel-view synthesis task. However, achieving photorealistic\nrendering from extreme novel viewpoints remains challenging, as artifacts\npersist across representations. In this work, we introduce Difix3D+, a novel\npipeline designed to enhance 3D reconstruction and novel-view synthesis through\nsingle-step diffusion models. At the core of our approach is Difix, a\nsingle-step image diffusion model trained to enhance and remove artifacts in\nrendered novel views caused by underconstrained regions of the 3D\nrepresentation. Difix serves two critical roles in our pipeline. First, it is\nused during the reconstruction phase to clean up pseudo-training views that are\nrendered from the reconstruction and then distilled back into 3D. This greatly\nenhances underconstrained regions and improves the overall 3D representation\nquality. More importantly, Difix also acts as a neural enhancer during\ninference, effectively removing residual artifacts arising from imperfect 3D\nsupervision and the limited capacity of current reconstruction models. Difix3D+\nis a general solution, a single model compatible with both NeRF and 3DGS\nrepresentations, and it achieves an average 2times improvement in FID score\nover baselines while maintaining 3D consistency.","upvotes":44,"discussionId":"67c69500bdab31ec59fea24d","projectPage":"https://research.nvidia.com/labs/toronto-ai/difix3d","ai_summary":"Difix3D+, a single-step diffusion model pipeline, enhances 3D reconstruction and novel-view synthesis by improving photorealism and reducing artifacts from extreme viewpoints.","ai_keywords":["Neural Radiance Fields","3D Gaussian Splatting","diffusion models","3D reconstruction","novel-view synthesis","photorealistic rendering","underconstrained regions","pseudo-training views","neural enhancer","FID score","3D consistency"]},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"633aaf695df91da9cea92960","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633aaf695df91da9cea92960/h1oOWUudbdlsd63Q5w0hM.png","isPro":false,"fullname":"Jay Wu","user":"jayw","type":"user"},{"_id":"64440be5af034cdfd69ca3a7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64440be5af034cdfd69ca3a7/qmx24QiDFT29vleCxL9TX.jpeg","isPro":false,"fullname":"Qinghong (Kevin) Lin","user":"KevinQHLin","type":"user"},{"_id":"6462ebcdd9b4902188120ac2","avatarUrl":"/avatars/750ae44428713d3622f3cad95cd7f63c.svg","isPro":false,"fullname":"Daniel Gao","user":"maybelu9","type":"user"},{"_id":"66f2432d5405e6677d430bbf","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/gxQWA3WTHGEQUL10fiIFh.png","isPro":false,"fullname":"GUO XIANGWU","user":"SamuelGuo","type":"user"},{"_id":"642435a1a3adbc7142c3b0a6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/642435a1a3adbc7142c3b0a6/wgLT_w9jNWRU3O0jU0646.jpeg","isPro":false,"fullname":"Joya Chen","user":"chenjoya","type":"user"},{"_id":"634e2217c1ce28f1de921708","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/634e2217c1ce28f1de921708/XTMB6alYUM0KAUptM98kP.jpeg","isPro":false,"fullname":"yyyang404","user":"yyyang","type":"user"},{"_id":"652b83b73b5997ed71a310f2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/652b83b73b5997ed71a310f2/ipCpdeHUp4-0OmRz5z8IW.png","isPro":false,"fullname":"Rui Zhao","user":"ruizhaocv","type":"user"},{"_id":"623461fccd8a0462e55b3666","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1647600114080-noauth.jpeg","isPro":true,"fullname":"Guian Fang","user":"Enderfga","type":"user"},{"_id":"66c45954ab8f09b10b7ab6a8","avatarUrl":"/avatars/f9946c775c4d70b8e044865ac34ef121.svg","isPro":false,"fullname":"Zhu","user":"ZaynZhu","type":"user"},{"_id":"64731de50da364bd0dfb7cd0","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/6DTmh7MOemA6xYMWGY9Ce.jpeg","isPro":false,"fullname":"Jinheng Xie","user":"Sierkinhane","type":"user"},{"_id":"63021630a35b21bd8a53305a","avatarUrl":"/avatars/7a7e8b39749eda61e57d8a1908726558.svg","isPro":true,"fullname":"Gu Yuchao","user":"guyuchao","type":"user"},{"_id":"6468855763a564ba347e1efa","avatarUrl":"/avatars/caa99c3318c1d6d21be44d2d32795a62.svg","isPro":false,"fullname":"David Junhao ZHANG","user":"Junhao233","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":3}">

Papers

arxiv:2503.01774

Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models

Published on Mar 3, 2025

· Submitted by

Jay Wu on Mar 4, 2025

#3 Paper of the day

Upvote

Authors:

Jay Zhangjie Wu ,

Haithem Turki ,

Xuanchi Ren ,

Mike Zheng Shou ,

Zan Gojcic ,

Abstract

Difix3D+, a single-step diffusion model pipeline, enhances 3D reconstruction and novel-view synthesis by improving photorealism and reducing artifacts from extreme viewpoints.

AI-generated summary

Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D reconstruction and novel-view synthesis task. However, achieving photorealistic rendering from extreme novel viewpoints remains challenging, as artifacts persist across representations. In this work, we introduce Difix3D+, a novel pipeline designed to enhance 3D reconstruction and novel-view synthesis through single-step diffusion models. At the core of our approach is Difix, a single-step image diffusion model trained to enhance and remove artifacts in rendered novel views caused by underconstrained regions of the 3D representation. Difix serves two critical roles in our pipeline. First, it is used during the reconstruction phase to clean up pseudo-training views that are rendered from the reconstruction and then distilled back into 3D. This greatly enhances underconstrained regions and improves the overall 3D representation quality. More importantly, Difix also acts as a neural enhancer during inference, effectively removing residual artifacts arising from imperfect 3D supervision and the limited capacity of current reconstruction models. Difix3D+ is a general solution, a single model compatible with both NeRF and 3DGS representations, and it achieves an average 2times improvement in FID score over baselines while maintaining 3D consistency.