Get Started
Home
Topics
Search
Library
Research questionDo multilingual attention heads that retrieve context also control transitions into the target language during reasoning?In multilingual Transformers, retrieving information from context and producing reasoning in a specified target language may involve different internal components. It is unclear whether the same attention heads perform both roles.
AI
Evaluation & Benchmarks
Mechanistic Interpretability
Natural Language Processing
Reasoning
Latest papersRecent research connected to this question, newest first.Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition HeadsThe evidence concerns multilingual and cross-lingual Transformer settings, comparing retrieval heads with heads associated with target-language transitions during chain-of-thought reasoning. Experiments use Qwen-2.5 and Llama-3.1 across MMLU-ProX, MGSM, MLQA, and XQuaD, with masking used to assess their effects.research paper · Sep 2, 2026
Related questions
How do attention and recurrent state divide factual retrieval from output style in hybrid language models?How should multilingual models choose the amount of English context used for reasoning in lower-resource languages?How can language-model attention remain reliable beyond its training context?How can attention-head contributions be measured in prompt-injection classifiers across circuit and output scales?