Get Started
Home
Topics
Search
Library
Research questionCan multilingual LLMs maintain mathematical reasoning when equivalent inputs use different word order or voice?A mathematically equivalent prompt can be expressed through reordered constituents or active-passive voice. Models that rely on surface form may change their answers even when the underlying entity-quantity relations remain unchanged.
AI
Evaluation & Benchmarks
Natural Language Processing
Reasoning
Latest papersRecent research connected to this question, newest first.Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving PerturbationsThe evidence covers Hindi and Malayalam, GSM8K-derived mathematical problems, constrained constituent reordering and active-passive transformations, six state-of-the-art LLMs, and multiple prompting strategies. Qualitative analysis and residual-stream activation patching examine the resulting errors, with findings implicating entity-quantity alignment and intermediate transformer layers. The evidence does not establish behavior across all languages, tasks, or models.research paper · Sep 3, 2026
Related questions
How can we tell whether LLMs follow coherent, human-like prerequisite relationships in mathematical reasoning?How can we tell whether LLM hidden-state geometry reflects reasoning operations rather than lexical or positional cues?What determines whether an LLM transfers arithmetic reasoning across numeric and verbal prompts?How can LLMs interleave reasoning with reliable step-level self-critique without a separate verifier?