Get Started
Research questionHow can we generate executable tests that are both sound and discriminative for code-generating LLMs?Executable tests must distinguish incorrect programs without rejecting correct ones. Such tests are scarce because they need to be both sound with respect to the intended solution and effective at revealing model-specific failures.
AI
Code Generation & Program Synthesis
Evaluation & Benchmarks
Reinforcement Learning
Latest papersRecent research connected to this question, newest first.Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMsThe study examines reinforcement-learning-based test generation for code LLMs. It uses reference-consistent tests followed by counterexamples targeting current failure modes, with results reported on TACO and LiveCodeBench and for inference-time selection among generated answers.research paper · Sep 3, 2026
Related questions
How can LLMs generate reliable, adaptive tests that expose one another’s model-specific weaknesses?How can LLMs generate functionally correct code without introducing security vulnerabilities?How can LLMs synthesize executable machines and improve their designs through physical simulation feedback?How can LLMs generate correct routines when correctness depends on their joint runtime behavior?
Home
Topics
Search
Library