Get Started
Research questionCan language models infer the intended pragmatic function of naturally occurring indirect Chinese comments from conversational context?Indirect and playful Chinese comments can support multiple plausible readings, with their intended social function depending on the surrounding exchange. Models may recognize broad irony or playfulness while misidentifying the particular interactional move.
AI
Evaluation & Benchmarks
Natural Language Processing
Latest papersRecent research connected to this question, newest first.You Really Didn't Get That? Benchmarking Social Pragmatic Inference for Indirect and Playful Chinese Online CommentsThe benchmark contains 4,735 human-validated items drawn from more than 200,000 public Chinese social-media interaction records. Each item pairs a target comment with reconstructed preceding context and plausible misreadings. Eight LLMs were evaluated in a cross-writer setting: mean leave-writer-out accuracy was 68.70%, the strongest model reached 81.42%, and human accuracy was 90.8%.research paper · Sep 3, 2026
Related questions
Can language models infer a verb’s intended semantic frame from context?Can language models infer others’ mental states as social interactions evolve under unreliable information?How can we compare language models’ conditional behavior and predict the effects of prompt changes?How can language models detect subtle mental manipulation in conversations without missing covert tactics?
Home
Topics
Search
Library