Get Started
Home
Topics
Search
Library
Research questionWhen does an LLM’s verbal confidence reliably reflect its underlying uncertainty?An LLM can sound certain without its confidence tracking whether its answer is likely to be correct or semantically stable. Prompting and tuning can also shift the distribution of stated scores, making apparent agreement difficult to interpret.
AI
Alignment & Safety
Evaluation & Benchmarks
Natural Language Processing
Statistical Machine Learning
Latest papersRecent research connected to this question, newest first.When Linguistic and Internal Confidence Diverge in Large Language ModelsThe source compares verbal confidence with logits-based measures for classification and semantic-entropy estimates for generation across the tested tasks, models, and prompts. These comparisons depend on access to the corresponding model-derived signals and do not establish reliability for other settings.research paper · Sep 4, 2026
Related questions
How can LLMs produce reliable confidence estimates for deciding when to defer outputs to humans?How should multilingual LLMs estimate uncertainty and calibrate abstention across languages and model sizes?How can we reliably assess whether conversational LLMs clarify ambiguity and recover user intent?How can LLMs distinguish ambiguous inputs from gaps in their knowledge when estimating uncertainty?