https://noisehypernetworks.github.io/

\n","updatedAt":"2025-08-14T03:59:12.262Z","author":{"_id":"6254599b6e36fe62e141c8f9","avatarUrl":"/avatars/08d6b68b92c7bfd0a8be022ba9f2f289.svg","fullname":"Shyamgopal Karthik","name":"shyamgopal","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.647847592830658},"editors":["shyamgopal"],"editorAvatarUrls":["/avatars/08d6b68b92c7bfd0a8be022ba9f2f289.svg"],"reactions":[],"isReport":false}},{"id":"689e8f53a002f7b6c850ead6","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-08-15T01:37:23.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models](https://huggingface.co/papers/2507.11554) (2025)\n* [Provable Maximum Entropy Manifold Exploration via Diffusion Models](https://huggingface.co/papers/2506.15385) (2025)\n* [Fine-Tuning Text-to-Speech Diffusion Models Using Reinforcement Learning with Human Feedback](https://huggingface.co/papers/2508.03123) (2025)\n* [ShortFT: Diffusion Model Alignment via Shortcut-based Fine-Tuning](https://huggingface.co/papers/2507.22604) (2025)\n* [Divergence Minimization Preference Optimization for Diffusion Model Alignment](https://huggingface.co/papers/2507.07510) (2025)\n* [Diffusion Tree Sampling: Scalable inference-time alignment of diffusion models](https://huggingface.co/papers/2506.20701) (2025)\n* [DiffusionBlocks: Blockwise Training for Generative Models via Score-Based Diffusion](https://huggingface.co/papers/2506.14202) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-08-15T01:37:23.799Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.665181577205658},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2508.09968","authors":[{"_id":"689d5ebbb083e610d741ea45","user":{"_id":"64edae322d5805cb998c7047","avatarUrl":"/avatars/6f1e887842d339a6c80b990491da1950.svg","isPro":false,"fullname":"Luca Eyring","user":"lucaeyring","type":"user"},"name":"Luca Eyring","status":"claimed_verified","statusLastChangedAt":"2025-08-18T06:59:07.512Z","hidden":false},{"_id":"689d5ebbb083e610d741ea46","user":{"_id":"6254599b6e36fe62e141c8f9","avatarUrl":"/avatars/08d6b68b92c7bfd0a8be022ba9f2f289.svg","isPro":false,"fullname":"Shyamgopal Karthik","user":"shyamgopal","type":"user"},"name":"Shyamgopal Karthik","status":"claimed_verified","statusLastChangedAt":"2025-08-14T13:37:25.592Z","hidden":false},{"_id":"689d5ebbb083e610d741ea47","name":"Alexey Dosovitskiy","hidden":false},{"_id":"689d5ebbb083e610d741ea48","user":{"_id":"6318c6e6cb116eab31d70a16","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6318c6e6cb116eab31d70a16/yAw5XtvqaPUkPj2B6JOzz.jpeg","isPro":false,"fullname":"Nataniel","user":"flavoredquark","type":"user"},"name":"Nataniel Ruiz","status":"claimed_verified","statusLastChangedAt":"2025-08-20T19:13:35.230Z","hidden":false},{"_id":"689d5ebbb083e610d741ea49","name":"Zeynep Akata","hidden":false}],"publishedAt":"2025-08-13T17:33:37.000Z","submittedOnDailyAt":"2025-08-14T02:29:12.254Z","title":"Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models","submittedOnDailyBy":{"_id":"6254599b6e36fe62e141c8f9","avatarUrl":"/avatars/08d6b68b92c7bfd0a8be022ba9f2f289.svg","isPro":false,"fullname":"Shyamgopal Karthik","user":"shyamgopal","type":"user"},"summary":"The new paradigm of test-time scaling has yielded remarkable breakthroughs in\nLarge Language Models (LLMs) (e.g. reasoning models) and in generative vision\nmodels, allowing models to allocate additional computation during inference to\neffectively tackle increasingly complex problems. Despite the improvements of\nthis approach, an important limitation emerges: the substantial increase in\ncomputation time makes the process slow and impractical for many applications.\nGiven the success of this paradigm and its growing usage, we seek to preserve\nits benefits while eschewing the inference overhead. In this work we propose\none solution to the critical problem of integrating test-time scaling knowledge\ninto a model during post-training. Specifically, we replace reward guided\ntest-time noise optimization in diffusion models with a Noise Hypernetwork that\nmodulates initial input noise. We propose a theoretically grounded framework\nfor learning this reward-tilted distribution for distilled generators, through\na tractable noise-space objective that maintains fidelity to the base model\nwhile optimizing for desired characteristics. We show that our approach\nrecovers a substantial portion of the quality gains from explicit test-time\noptimization at a fraction of the computational cost. Code is available at\nhttps://github.com/ExplainableML/HyperNoise","upvotes":15,"discussionId":"689d5ebcb083e610d741ea4a","projectPage":"https://noisehypernetworks.github.io/","githubRepo":"https://github.com/ExplainableML/HyperNoise","githubRepoAddedBy":"user","ai_summary":"A Noise Hypernetwork is introduced to integrate test-time scaling knowledge into diffusion models, reducing computational cost while maintaining quality.","ai_keywords":["test-time scaling","Large Language Models","generative vision models","diffusion models","reward guided test-time noise optimization","Noise Hypernetwork","noise-space objective","distilled generators"],"githubStars":70},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6254599b6e36fe62e141c8f9","avatarUrl":"/avatars/08d6b68b92c7bfd0a8be022ba9f2f289.svg","isPro":false,"fullname":"Shyamgopal Karthik","user":"shyamgopal","type":"user"},{"_id":"63bbf972d8d676a2299cdb44","avatarUrl":"/avatars/366d6ca7a4e19e42d2ec236a38d74ebd.svg","isPro":false,"fullname":"Sangwon","user":"agwmon","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"62d81b2b14bc83f0febefc2e","avatarUrl":"/avatars/d6520e85d1cead2249d29becaf311e1d.svg","isPro":false,"fullname":"Felix Tuma","user":"floom","type":"user"},{"_id":"651f8133dbf879b8c58f5136","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/651f8133dbf879b8c58f5136/0L8Ecgi5Ietkm_DchJwE-.png","isPro":false,"fullname":"Zikai Zhou","user":"Klayand","type":"user"},{"_id":"64edae322d5805cb998c7047","avatarUrl":"/avatars/6f1e887842d339a6c80b990491da1950.svg","isPro":false,"fullname":"Luca Eyring","user":"lucaeyring","type":"user"},{"_id":"663ccbff3a74a20189d4aa2e","avatarUrl":"/avatars/83a54455e0157480f65c498cd9057cf2.svg","isPro":false,"fullname":"Nguyen Van Thanh","user":"NguyenVanThanhHust","type":"user"},{"_id":"64137e2150358a805203cbac","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64137e2150358a805203cbac/vU4OjyvOlu2g5PEOOik-t.jpeg","isPro":false,"fullname":"euclaise","user":"euclaise","type":"user"},{"_id":"66c0ac2cd6719dab3eee9670","avatarUrl":"/avatars/397def36891d4ee123929d44c51c646d.svg","isPro":false,"fullname":"Cedric Perauer","user":"CedricPerauer","type":"user"},{"_id":"62e1403f926f4892a4c545f8","avatarUrl":"/avatars/1f9d09bba8dd2d8657619536078f9ec2.svg","isPro":true,"fullname":"Basit mustafa","user":"BasitMustafa","type":"user"},{"_id":"64df3ad6a9bcacc18bc0606a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/s3kpJyOf7NwO-tHEpRcok.png","isPro":false,"fullname":"Carlos","user":"Carlosvirella100","type":"user"},{"_id":"64ea59beb36ed038b6638ece","avatarUrl":"/avatars/a74b3c8b63b5ca8ebb3a00455f6f803f.svg","isPro":false,"fullname":"Slava","user":"wertlon","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

Papers

arxiv:2508.09968

Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models

Published on Aug 13, 2025

· Submitted by

Shyamgopal Karthik on Aug 14, 2025

Upvote

Authors:

Luca Eyring ,

Shyamgopal Karthik ,

Nataniel Ruiz ,

Abstract

A Noise Hypernetwork is introduced to integrate test-time scaling knowledge into diffusion models, reducing computational cost while maintaining quality.

AI-generated summary

The new paradigm of test-time scaling has yielded remarkable breakthroughs in Large Language Models (LLMs) (e.g. reasoning models) and in generative vision models, allowing models to allocate additional computation during inference to effectively tackle increasingly complex problems. Despite the improvements of this approach, an important limitation emerges: the substantial increase in computation time makes the process slow and impractical for many applications. Given the success of this paradigm and its growing usage, we seek to preserve its benefits while eschewing the inference overhead. In this work we propose one solution to the critical problem of integrating test-time scaling knowledge into a model during post-training. Specifically, we replace reward guided test-time noise optimization in diffusion models with a Noise Hypernetwork that modulates initial input noise. We propose a theoretically grounded framework for learning this reward-tilted distribution for distilled generators, through a tractable noise-space objective that maintains fidelity to the base model while optimizing for desired characteristics. We show that our approach recovers a substantial portion of the quality gains from explicit test-time optimization at a fraction of the computational cost. Code is available at https://github.com/ExplainableML/HyperNoise