💻 Code: https://github.com/SEACrowd/itlc\n","updatedAt":"2025-06-17T03:28:16.217Z","author":{"_id":"61728a033edf4cc38a81237a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1652231681579-61728a033edf4cc38a81237a.jpeg","fullname":"Joanito Agili Lopo","name":"joanitolopo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7757159471511841},"editors":["joanitolopo"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1652231681579-61728a033edf4cc38a81237a.jpeg"],"reactions":[{"reaction":"🔥","users":["joanitolopo","samuelcahyawijaya","gentaiscool"],"count":3}],"isReport":false}},{"id":"6852188222479d7f67412983","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-06-18T01:38:10.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners](https://huggingface.co/papers/2505.15257) (2025)\n* [Improving Multilingual Language Models by Aligning Representations through Steering](https://huggingface.co/papers/2505.12584) (2025)\n* [Multilingual Test-Time Scaling via Initial Thought Transfer](https://huggingface.co/papers/2505.15508) (2025)\n* [Refusal Direction is Universal Across Safety-Aligned Languages](https://huggingface.co/papers/2505.17306) (2025)\n* [Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models](https://huggingface.co/papers/2505.16538) (2025)\n* [CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning](https://huggingface.co/papers/2506.00875) (2025)\n* [Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfer](https://huggingface.co/papers/2505.10945) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners (2025) \n
- Improving Multilingual Language Models by Aligning Representations through Steering (2025) \n
- Multilingual Test-Time Scaling via Initial Thought Transfer (2025) \n
- Refusal Direction is Universal Across Safety-Aligned Languages (2025) \n
- Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models (2025) \n
- CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning (2025) \n
- Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfer (2025) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Language Surgery in Multilingual Large Language Models
Abstract
Research confirms natural representation alignment in large language models and introduces Inference-Time Language Control to enhance cross-lingual performance.
Large Language Models (LLMs) have demonstrated remarkable generalization capabilities across tasks and languages, revolutionizing natural language processing. This paper investigates the naturally emerging representation alignment in LLMs, particularly in the middle layers, and its implications for disentangling language-specific and language-agnostic information. We empirically confirm the existence of this alignment, analyze its behavior in comparison to explicitly designed alignment models, and demonstrate its potential for language-specific manipulation without semantic degradation. Building on these findings, we propose Inference-Time Language Control (ITLC), a novel method that leverages latent injection to enable precise cross-lingual language control and mitigate language confusion in LLMs. Our experiments highlight ITLC's strong cross-lingual control capabilities while preserving semantic integrity in target languages. Furthermore, we demonstrate its effectiveness in alleviating the cross-lingual language confusion problem, which persists even in current large-scale LLMs, leading to inconsistent language generation. This work advances our understanding of representation alignment in LLMs and introduces a practical solution for enhancing their cross-lingual performance.
Community
We’re excited to share our latest work, "Language Surgery in Multilingual Large Language Models". We proposed a method, named Inference-Time Language Control (ITLC), designed to enhance cross-lingual language control and mitigate language confusion in Large Language Models (LLMs). ITLC leverages latent injection to enable precise manipulation of language-specific information during inference, while preserving semantic integrity. By exploiting representation alignment in LLMs’ middle layers, ITLC achieves zero-shot cross-lingual generation (10.70 average BLEU), mitigates language confusion (2.7x better LCPR, 4x better LPR), and allows language-specific manipulation without compromising meaning. Key contributions include confirming representation alignment via cosine similarity analysis and providing a practical solution for cross-lingual tasks. ITLC’s applications include enabling zero-shot cross-lingual generation and ensuring consistent language output.
📖 Paper: http://arxiv.org/abs/2506.12450
💻 Code: https://github.com/SEACrowd/itlc
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners (2025)
- Improving Multilingual Language Models by Aligning Representations through Steering (2025)
- Multilingual Test-Time Scaling via Initial Thought Transfer (2025)
- Refusal Direction is Universal Across Safety-Aligned Languages (2025)
- Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models (2025)
- CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning (2025)
- Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfer (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper