Get Started
Home
Topics
Search
Library
Research questionHow can LLM-based automated program repair detect patches and artifacts unsupported by repair evidence?A repair can pass available tests while still being causally unsupported or based on incorrect intermediate analysis. Such artifacts include triggering-test identification, line-coverage predictions, and generated tests that misrepresent the bug or expected behavior.
AI
AI Agents
Code Generation & Program Synthesis
Evaluation & Benchmarks
Research Paper
Technology
Latest papersRecent research connected to this question, newest first.Better Understanding, Better Fixes? A Study of Hallucination in LLM-based Automated Program RepairThe evidence covers three representative LLMs evaluated on 832 Defects4J bugs using automatic evaluation and manual analysis. It examines repair hallucinations in final patches and understanding hallucinations in intermediate artifacts; the findings do not establish behavior beyond these models, bugs, and evaluation evidence.research paper · Sep 4, 2026PatchBench: Evaluating AI Agents for Vulnerability PatchingThe study evaluates 11 vulnerability-patching agents, including the three leading AIxCC agents. Its evidence covers C/C++ tasks, finding substantial similarity to historical developer patches in 25% of agent patches and an average 1.83× inflation in solve rates under proof-of-concept-only validation.research paper · Sep 3, 2026
Related questions
How can autonomous LLM agents detect attacks whose evidence accumulates across loop iterations?How can we detect when LLM-decompiled code diverges from original behavior or erases disclosed vulnerabilities?How can sampling-only diagnosis identify whether an individual LLM hallucination comes from retrieval, training, or autoregressive commitment?How can we reliably detect when an LLM response is unsupported by its reference documents?