Get Started
Topic · 57 recaps
Natural Language Processing
Language understanding and generation — covering classical NLP tasks, language models, dialogue systems, machine translation, and text processing at large.
Play all
...
Posts
Questions
Home
Topics
Search
Library
Questions researchers are working on
Follow a question through Rcap’s explanations and the latest papers addressing it.
Search
Can automatic metrics and LLM judges reliably reflect human judgments of multilingual summary quality?
Automatic evaluation makes it practical to compare summaries, but its scores may not capture how people judge summary quality. This makes it difficult to know which evaluators can be trusted across languages and criteria.
Can black-box LLM judges provide reproducible measurements on shared endpoints?
The same request to the same model name may produce different rankings across repeated or later calls on shared infrastructure. This instability can make filtering, scoring, and pass/fail decisions irreproducible even when execution records are complete.
Can language agents maintain hidden state consistently across dialogue branches using only public conversation history?
A chat interface exposes conversation history but provides no separate channel for state that must remain hidden. When dialogue branches, the agent must preserve the same secret and answer consistently without revealing or reconstructing it from public text.
Can language models infer a verb’s intended semantic frame from context?
The same verb can evoke different semantic frames—and different implied knowledge—depending on its context. It is unclear whether language models make this kind of implicit enrichment reliably and in a human-like way.
Can language models infer others’ mental states as social interactions evolve under unreliable information?
Socially grounded tasks require models to use interaction history, infer what participants know or intend, and distinguish reliable from unreliable information. Existing evaluations often isolate these demands, making performance in changing social environments difficult to characterize.
Can language models infer the intended pragmatic function of naturally occurring indirect Chinese comments from conversational context?
Indirect and playful Chinese comments can support multiple plausible readings, with their intended social function depending on the surrounding exchange. Models may recognize broad irony or playfulness while misidentifying the particular interactional move.
Can large language models reliably perform Arabic morphosyntactic tagging and dependency parsing despite morphological and orthographic ambiguity?
Arabic’s rich morphology and orthographic ambiguity make morphological and syntactic interpretation closely interdependent. Performance can also vary with how text is represented and whether relevant annotated examples are available as demonstrations.
Can multilingual LLMs maintain mathematical reasoning when equivalent inputs use different word order or voice?
A mathematically equivalent prompt can be expressed through reordered constituents or active-passive voice. Models that rely on surface form may change their answers even when the underlying entity-quantity relations remain unchanged.
Can user feedback reliably guide LLM revisions if LLM judges overlook the resulting improvements?
User interactions may reveal issues that an LLM cannot detect on its own, but the feedback can be noisy and improvements may be difficult to measure. Evaluation becomes especially problematic when judges prefer a baseline response even after feedback has corrected the targeted issue.
Do language models generalize phonological rules or rely on memorized forms when selecting allomorphs?
Tokenization obscures sound structure, yet models often select morpheme forms according to phonological context. It is difficult to determine whether this behavior reflects memorized forms or a rule-like representation that causally influences selection.
Do multilingual attention heads that retrieve context also control transitions into the target language during reasoning?
In multilingual Transformers, retrieving information from context and producing reasoning in a specified target language may involve different internal components. It is unclear whether the same attention heads perform both roles.
Do pretrained language models encode a reusable truthfulness signal for detecting misinformation without external evidence?
Misinformation detectors can rely on wording patterns rather than whether a claim is true, while evidence-retrieval pipelines depend on outside sources. The central issue is whether pretrained models contain an internal signal that supports transferable truthfulness judgments.
Do refusal representations transfer across language-model architectures, and where should safety interventions read them?
Architectures differ in how they mix and update token information, so a refusal signal identified in one model may not be directly usable in another. Safety tooling must determine both whether the representation transfers and where the relevant computation is exposed.
Do source labels bias human and LLM judgments of logical fallacies differently?
Labels about who produced a comment can influence judgments of its credibility and logical quality independently of the argument itself. It remains unclear whether humans and language models are affected by these source cues in the same way.
Does Cantonese-specific adaptation improve language-model prediction of naturalistic Cantonese reading consistently across information-theoretic measures?
Language models estimate information-theoretic expectations that may help explain human reading, but it is unclear whether Cantonese-specific adaptation improves this alignment. Different measures can rank models differently, making conclusions sensitive to how predictive fit is quantified.
Does reusing Transformer layers improve language-model quality when parameter, compute, and KV-cache budgets are matched?
Layer looping increases effective computation by revisiting shared parameters, but comparisons can mistake extra computation or memory for an architectural improvement. The central difficulty is isolating the effect of reuse while holding major training and inference budgets constant.
How can a model select the correct entity-specific parametric memory for graph queries without query-time text or semantic similarity?
Graph facts stored in separate parameter memories avoid repeating retrieved subgraphs in each prompt. However, a query’s answer may reside in one specific entity’s memory even when another entity is semantically similar, making ordinary similarity retrieval unreliable.
How can a single search agent improve multi-hop web research without sub-agents or test-time verification?
Multi-hop web research requires an agent to connect evidence across several pages while retaining the information needed for later steps. Long search trajectories can overwhelm the available context, making both training and reliable performance difficult without additional agents or verification passes.
How can activation steering represent and control multidimensional authorship style without dedicated training?
Authorship style combines interacting rhetorical dimensions, so a single steering direction can blur distinct stylistic signals or cause interference. The representation must preserve the target author’s style without simply copying source exemplars.
How can adaptive English-learning materials personalize practice while preserving curriculum structure and reducing teacher workload?
Fixed English materials give learners the same sequence and leave teachers to diagnose needs and correct work manually. Adaptive systems must tailor tasks and feedback without making curriculum progress or classroom oversight harder to manage.
Previous
1 / 19
Next