Get Started
Home
Topics
Search
Library
Research questionHow can multimodal models rely on images or audio rather than language shortcuts?A model can choose a plausible answer from wording alone while overlooking the image or audio. The challenge is making its answer depend on what it actually sees or hears.
AI
Audio & Speech
Computer Vision
Evaluation & Benchmarks
Image & Video Processing
Inference Optimization
LLM Pretraining & Post-training
Machine Learning
Mechanistic Interpretability
Multimodal Models
Natural Language Processing
Reasoning
Reinforcement Learning
Research Paper
Latest papersRecent research connected to this question, newest first.From Vision to Language: Investigating Causal Information Flow in Multimodal Decision-MakingThe evidence concerns vision-language models in a video-based generative multiple-choice-like setting. The study applies layer-wise causal interventions to video-text attention pathways and reports that visual integration concentrates around candidate answer options, with nouns acting as semantic anchors and verbs being more relevant to temporal relations. It also reports fragile temporal reasoning, while noting that linguistic biases in temporal expressions may contribute.research paper · Sep 4, 2026GSM8K-V: Can Vision Language Models Solve Grade School Math Word Problems in Visual ContextsThe evidence comes from GSM8K-V, a benchmark of 1,319 human-verified multi-image problems derived from GSM8K with semantic equivalence preserved. It evaluates 34 vision-language models; the reported best model reaches 59% accuracy versus 91% for humans, with implicit visual inference identified as the main bottleneck.research paper · Sep 4, 2026Tracing Audio Grounding and Answer Selection in Audio LLMsThe study compares pretrained and trained Audio LLMs using absent or unrelated-audio interventions, then traces effects on answer-choice representations, final predictions, and layer-specific learned weights. Evidence is limited to the studied answer-selection tasks and models, so broader multimodal behaviors are not established.research paper · Sep 4, 2026BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QAThe source evaluates a training-free decoding approach that contrasts multimodal and text-only output preferences for multiple-choice answers. Its evidence covers five vision-language models and five multimodal MCQA benchmarks, not open-ended question answering or other multimodal tasks.research paper · Sep 4, 2026IchthyoNoma: Nomenclature and Context Sensitivity of Zero-Shot Biological Vision--Language Models for Bangladeshi Freshwater Fish RecognitionThe evidence covers CLIP, BioCLIP, BioCLIP2, and multilingual Jina CLIP v2 on 10,321 images across seven freshwater-fish categories from two Bangladeshi sources. It compares English common names, scientific names, and Bengali prompts, alongside blur, grayscale, and masking interventions; the findings are limited to these datasets, models, and evaluation conditions.research paper · Sep 3, 2026CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary ReasoningThe evidence concerns multimodal models evaluated on 4,870 items in 10 languages and 18 regions. Tasks combine final-dish and step-by-step cooking images with ingredients, procedural text, and regional labels, including process-grounded cultural attribution; results cover 12 models and show performance gaps between standard recognition and culturally specific attribution.research paper · Sep 3, 2026Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVRThe source describes multimodal RLVR training that uses annotated evidence regions as privileged training labels while inference uses only the original image and question. Reported experiments cover Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B across perception, hallucination, visual-math, and multimodal-reasoning benchmarks; the evidence is limited to the reported scores, diagnostics, and ablations.research paper · Sep 3, 2026
Related questions
How can multimodal models integrate evidence across deeply interleaved text and images?How can multimodal models reason about fine-grained interpersonal relationships from conversational and visual cues?How can multimodal models infer directorial intent from audiovisual choices rather than merely recognize events?How can multimodal models integrate narrative context with chart evidence to answer multi-step questions?