ResearchStudio-Idea mines 1,947 ICLR/ICML/NeurIPS papers (including rejects) to induce 15 reusable Ideation pattern cards that guide an LLM through bottleneck-diagnosis, pattern-guided drafting, and prior-art audit, winning 88 of 100 blind quality rankings against bare-model and generic-skill baselines.
You’ve built an internal “research copilot” that turns a Slack prompt like “give me a novel idea for long-context RL” into a proposal your team can actually critique. Today it emits a fluent paragraph, no one can tell if it’s already been published, and the “novelty” is mostly that it’s vague enough not to collide with anything. The dominant baseline here is AI Scientist-style end-to-end agents that try to write the whole paper, and separate novelty-checkers that grade ideas after the fact. This paper instead scopes the tool to just the first mile (problem → one auditable idea card) and grounds it in what actually got accepted or rejected at top venues.
The authors collect 1,014 Oral, 260 high-citation, and 722 Reject papers from OpenReview (2021–2025). For each paper, Claude Sonnet 4.6 extracts a 12-field “innovation signature” describing how the paper innovates. A second pass rewrites the strategy-bearing fields into domain-agnostic language (stripping words like “diffusion” or “Transformer”) so clustering groups papers by research move rather than topic. Those rewrites get embedded and clustered with UMAP plus HDBSCAN, yielding 31 fine-grained sub-patterns which Claude Opus 4.7 then abstracts into 15 higher-level ideation patterns (e.g., “Audit and Pivot an Assumption”, “Reframe as a Solvable Object”). Each pattern becomes a card with success conditions from Oral papers and failure modes from Rejects.
At inference time, the IdeaSpark skill runs a fixed pipeline rather than free-form reasoning:
lit = phase0_retrieve(user_direction) # arXiv, OpenReview, OpenAlex, S2
fulltext = phase0_fetch_pdfs(lit) # hard-gated: no cache, no bottleneck
bottleneck = phase1_diagnose(lit, fulltext) # builds method-lineage tree
pattern = phase2_select(bottleneck, cards) # 1-3 patterns by structural fit
candidate = phase2_generate(pattern, sub_pattern_card)
hits = phase3_collision_search(candidate) # mechanism-level, not area-level
verdict = phase3_audit(candidate, hits, failure_modes) # advance/revise/abandon
if verdict == "advance": return render_idea_card(candidate)
The audit runs four corpus-anchored checks: reject-lesson match, whether the mechanism actually performs the cited sub-pattern’s move (not just its parent), substantive delivery of anti-pattern mitigations, and paper-pointed prior-art collision. Deterministic validators enforce that the “kill-switch” fields (a falsification prediction and compute budget) survive byte-identical from generation through rendering, so the model can’t quietly soften the testable claim.
The common assumption when building idea-generation systems is that you should train or prompt an LLM to produce novel ideas, then score them with a separate novelty judge. This paper shows the opposite. The same corpus-derived pattern card should both guide generation and audit the result, because rejected papers occupy the same strategy space as accepted ones. What separates them is execution against known failure modes, not choice of move. The load-bearing evidence is the reject-only re-clustering, which maps every rejected cluster back into the 15-pattern taxonomy with no out-of-taxonomy bucket.
The finding that makes the thesis true: when the Reject pool is clustered in isolation, all 13 resulting clusters land within cosine 0.897–0.986 of an existing accepted-paper pattern centroid. Rejection is not explained by choice of strategy. The Oral-vs-Reject spread at the pattern level is only ±2.9 pp. The wider signal is a nearly orthogonal program-committee-vs-community axis reaching ±13 pp: “Audit and Pivot an Assumption” is disproportionately elevated to Oral (+13.1 pp vs high-citation share), while cross-modal unification is disproportionately cited but rarely made Oral.
•
Composition is the norm. Paper-level Paper-level multi-label tagging shows k=2 patterns as the mode across Oral, HC, and Reject, with a 33.6% tail at k≥3. Class means cluster at ~2.3, so composition size doesn’t discriminate acceptance; composition content does.
•
Endpoint quality. On 100 held-out ICLR 2026 Oral seeds rewritten to be method-agnostic, IdeaSpark scores 3.87/4 on blind listwise quality ranking, winning 88/100 seeds. Claude Opus 4.8 bare and a generic auto-authored skill on the same backbone both score ~2.57. GPT-5.5 bare scores highest on novelty (3.73) but lowest on quality (1.00), because its vague generic template evades prior-art collision. This is the “novel-but-empty” failure mode single-axis novelty scores miss.
•
Ablation. Embedding the domain-agnostic rewrites instead of raw abstracts prevents cluster collapse: at min_cluster_size=15, the raw-abstract variant degenerates to k=2 topical mega-clusters. SPECTER2 loses to a general-purpose embedder by +0.15 silhouette because its citation prior anchors on topic, not strategy.
Reach for this design when you’re shipping a research or product-ideation copilot that needs to justify why a proposal is defensible, not just emit one. The move to steal: derive a small library of “move cards” from your own outcome corpus (accepted proposals, launched features, killed projects), then use the same card to both guide generation and run a checklist audit against known failure modes. The kill-switch pattern (byte-preserving a falsification prediction from generation through rendering) is directly transferable to any agent that needs a testable claim to survive editing.
The project page is at aka.ms/ResearchStudio. The paper mentions a GitHub artifact release containing the 15 patterns and 31 sub-pattern cards but I can’t verify the exact URL from the text. Directly reusable pieces include the 1,947-paper corpus with signatures, the domain-by-pattern matrix, and the Scoop-Check and Paper-Search skills as standalone components. The endpoint evaluation uses only automated LLM judges, so treat the 88/100 win rate as agreement-with-structured-reviewers, not human program-committee preference.
When you build a generator, build the auditor from the same object. If your rejected examples cluster the same way as your accepted ones, the difference lives in execution details, and those details belong in a failure-mode checklist the generator has to survive, not in a separate ranker bolted on afterward.
•
The corpus is three ML conferences over five years, and the held-out seeds are ICLR 2026 Orals. The induced patterns may not transfer to other fields or to benchmark/systems contributions, which the quality rubric explicitly under-rewards.
•
Both endpoint judges are LLM skills, and the quality judge shares a model family with three of the four systems being scored. Human reviewer agreement is deferred to future work, so “quality” here means “agreement with a structured automated reviewer.”
•
The 15-pattern count comes from a single induction call with Claude Opus 4.7. The authors expect a comparable run to land at 12–18 patterns, but inter-seed and inter-model stability is untested. If your downstream use depends on the exact taxonomy, that fragility matters.