DisCo distills GitHub repositories and papers into verified, on-demand skills that an ML research agent loads as operating context, lifting the same Codex agent harness agent’s MLE-bench score from 31.1% to 72.9% without changing the model or harness.
You’ve shipped an autonomous coding agent that reproduces ML papers or tunes models on new datasets. Today, on every task, it re-discovers by trial and error which library version fits, how the data loader wants inputs shaped, and which config flags silently break training. Each failed run burns your compute budget before any real measurement lands.
The field’s response so far has been to make the model stronger (better backbones) or the harness smarter (better planning, memory, verification, e.g. Codex agent harness or Claude Code). This paper argues that a whole layer is still missing between them: the domain-specific know-how that turns knowing a method into making it work. The authors call it operational knowledge and package it as reusable Agent Skills.
A skill here is a small folder with three parts: a SKILL.md that says what the skill is for and when to use it, a references/ directory with deeper docs loaded only on demand (Progressive disclosure), and a scripts/ directory with executable wrappers. The agent scans one-line summaries of thousands of skills but only opens the few a task needs.
Skills come from DisCo, which runs the same four-stage pipeline (scope → ground → construct → verify) in two modes:
•
Task-agnostic: anchor is a repository or paper. DisCo inspects source, docs, tests, and examples, then emits a skill graph rooted at an entry skill that routes to component skills (data prep, training, inference, troubleshooting).
•
Task-oriented: anchor is a concrete task. DisCo decomposes it, finds gaps in what the agent already knows, searches for supporting material, and produces skills tailored to that class of problem.
Crucially, nothing enters the library without verification: assertion-backed usability cases, native tests or smoke scripts, and static gates. Failures trigger local repair; unresolved gaps are recorded, not hidden. Applied across 1,000 popular ML repos, this yields the AREX-Skill Library: 5,353 skills organized under 20 areas and 178 capability families, with a router that narrows a request from area → family → repo graph.
def solve(task, library):
router = library.router
area, family = router.route(task) # progressive disclosure
graph = router.open_repo_graph(area, family)
skill = graph.entry_skill # SKILL.md only, ~summary
while not done(task):
step = agent.plan(task, context=skill)
if step.needs_detail:
skill = graph.follow_link(skill, step) # load reference/script
obs = env.execute(step)
The prevailing view of agent progress is that you either upgrade the backbone or engineer a smarter control loop around it. This paper shows the opposite. The bottleneck is not reasoning power or orchestration but a missing third layer: verified, version-specific operational knowledge that the agent can load on demand, produced automatically from the same repos and papers a human expert would read. The load-bearing evidence is that the same Codex agent harness + GPT-5.5 setup, with only distilled skills added, beats the strongest public MLE-bench leaderboard entries that use stronger backbones like Gemini-3-Pro-Preview.
The finding that makes the thesis true: with backbone, harness, and per-task running budget held fixed, adding skills is the only variable at run time, and it moves every benchmark in the same direction.
•
MLE-bench (75 competitions, Any-Medal %): 31.11 → 72.89 (+41.78 pp, +134.3% relative). Gains grow with difficulty: +43.94 pp on Low, +37.72 pp on Medium, +48.89 pp on High (13.33 → 62.22).
•
Beats stronger public agents on their own leaderboard: same Codex agent harness + GPT-5.5 with AREX-Skill scores 72.89 vs 64.44 for the best public entry (which uses Gemini-3-Pro-Preview).
•
PaperBench (20 papers, replication score): 29.45 → 39.59 (+10.14, +34.4%). Improves 18 of 20 papers; biggest jumps are on low-baseline papers (e.g. one task goes 1.50 → 17.17, another 7.94 → 48.51). Two tasks regress, consistent with a retrieval-precision trade-off.
•
FrontierCS Agent Track (188 tasks): 70.63 → 77.14 (+6.51, +9.2%). Improves 74 tasks, unchanged on 66. On tasks scoring below 50 without skills, mean rises from 19.43 to 45.99. Score gains are essentially uncorrelated with token or step usage (Spearman ρ ≈ 0.01), so extra compute isn’t doing the work. Codex+AREX-Skill Pareto-dominates the Claude Code entries on score, tokens, steps, and tool calls.
•
PassNet (graph-compiler passes): AS Score 1.343 → 1.5313 (+14.0%); correctness 81.35% → 90.76%; failed samples drop from 14 to 5.
Reach for this if you’re building an agent that ships code against a library-heavy stack, i.e. reproducing papers, running Kaggle-style ML pipelines, or generating compiler passes. Instead of letting the agent rediscover on each run that a given package needs its data laid out just so, you pre-distill the repos it depends on into skill graphs. The agent still plans and executes as before; it just reads a scoped SKILL.md at the start of a step instead of grepping source.
The code and the AREX-Skill Library (5,000+ verified skills over 1,000 repos, 20 areas, 178 families, with the area/family router) are released at GitHub under CC BY-NC-SA 4.0 (non-commercial). Distillation is expensive up front (about $40 per repo with GPT-5.5 and xhigh reasoning), but that cost is paid once and amortized across every task that touches that repo.
Before making the agent smarter, give it a curated, verified memory of the software it will actually touch. Most of what looks like reasoning failure in ML research agents is really missing operational knowledge about specific libraries and papers. Distill that knowledge once, verify it against real executions, and load only the branch a task needs.
•
The strong headline gain is on a baseline (vanilla Codex agent harness + GPT-5.5, 31.1% on MLE-bench) that is well below the public leaderboard’s best; the absolute lead over the strongest public agent is a more modest +8.45 pp, and public entries used different (sometimes stronger) backbones.
•
Skills can hurt when retrieval is imprecise: on PaperBench, two tasks with above-average no-skill scores regressed by 4–5 points because retrieved guidance pulled the agent off an idiosyncratic solution it would have found unaided.
•
The distillation cost (~$40/repo with a frontier model at xhigh reasoning, plus verification loops) is real; the economics work only because the resulting skills are reused across many downstream runs. A one-shot user gets little benefit from this pipeline.