The following papers were recommended by the Semantic Scholar API
\n- \n
- Token Pruning for In-Context Generation in Diffusion Transformers (2026) \n
- GRAN-TED: Generating Robust, Aligned, and Nuanced Text Embedding for Diffusion Models (2025) \n
- Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing (2025) \n
- Exploring MLLM-Diffusion Information Transfer with MetaCanvas (2025) \n
- PokeFusion Attention: Enhancing Reference-Free Style-Conditioned Generation (2026) \n
- RePack then Refine: Efficient Diffusion Transformer with Vision Foundation Model (2025) \n
- Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs (2026) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
arXivLens breakdown of this paper 👉 https://arxivlens.com/PaperView/Details/semantic-routing-exploring-multi-layer-llm-feature-weighting-for-diffusion-transformers-115-95f137b5
\n- \n
- Executive Summary \n
- Detailed Breakdown \n
- Practical Applications \n
Semantic Routing: Exploring Multi-Layer LLM Feature Weighting for Diffusion Transformers
Abstract
Text conditioning in DiT-based models is enhanced through a unified normalized convex fusion framework that optimizes multi-layer LLM hidden states via depth-wise semantic routing, improving text-image alignment and compositional generation.
Recent DiT-based text-to-image models increasingly adopt LLMs as text encoders, yet text conditioning remains largely static and often utilizes only a single LLM layer, despite pronounced semantic hierarchy across LLM layers and non-stationary denoising dynamics over both diffusion time and network depth. To better match the dynamic process of DiT generation and thereby enhance the diffusion model's generative capability, we introduce a unified normalized convex fusion framework equipped with lightweight gates to systematically organize multi-layer LLM hidden states via time-wise, depth-wise, and joint fusion. Experiments establish Depth-wise Semantic Routing as the superior conditioning strategy, consistently improving text-image alignment and compositional generation (e.g., +9.97 on the GenAI-Bench Counting task). Conversely, we find that purely time-wise fusion can paradoxically degrade visual generation fidelity. We attribute this to a train-inference trajectory mismatch: under classifier-free guidance, nominal timesteps fail to track the effective SNR, causing semantically mistimed feature injection during inference. Overall, our results position depth-wise routing as a strong and effective baseline and highlight the critical need for trajectory-aware signals to enable robust time-dependent conditioning.
Community
Recent DiT-based text-to-image models increasingly adopt LLMs as text encoders, yet text conditioning remains largely static and often utilizes only a single LLM layer, despite pronounced semantic hierarchy across LLM layers and non-stationary denoising dynamics over both diffusion time and network depth. To better match the dynamic process of DiT generation and thereby enhance the diffusion model's generative capability, we introduce a unified normalized convex fusion framework equipped with lightweight gates to systematically organize multi-layer LLM hidden states via time-wise, depth-wise, and joint fusion. Experiments establish Depth-wise Semantic Routing as the superior conditioning strategy, consistently improving text–image alignment and compositional generation (e.g., +9.97 on the GenAI-Bench Counting task). Conversely, we find that purely time-wise fusion can paradoxically degrade visual generation fidelity. We attribute this to a train–inference trajectory mismatch: under classifier-free guidance, nominal timesteps fail to track the effective SNR, causing semantically mistimed feature injection during inference. Overall, our results position depth-wise routing as a strong and effective baseline and highlight the critical need for trajectory-aware signals to enable robust time-dependent conditioning.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Token Pruning for In-Context Generation in Diffusion Transformers (2026)
- GRAN-TED: Generating Robust, Aligned, and Nuanced Text Embedding for Diffusion Models (2025)
- Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing (2025)
- Exploring MLLM-Diffusion Information Transfer with MetaCanvas (2025)
- PokeFusion Attention: Enhancing Reference-Free Style-Conditioned Generation (2026)
- RePack then Refine: Efficient Diffusion Transformer with Vision Foundation Model (2025)
- Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
arXivLens breakdown of this paper 👉 https://arxivlens.com/PaperView/Details/semantic-routing-exploring-multi-layer-llm-feature-weighting-for-diffusion-transformers-115-95f137b5
- Executive Summary
- Detailed Breakdown
- Practical Applications
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper