Get Started
Home
Topics
Search
Library
Agents · LLM Training · Jul 27, 2026

From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search

Source: research paper via Hugging Face Daily Papers
When distilling a black-box frontier model into a small agent, fine-tuning on its raw traces backfires — students mimic prose, not skill. MAPD routes teacher supervision through a strict JSON plan-and-evidence schema used only as privileged context, beating raw-text distillation by 7 points on Qwen3-1.7B.
TL;DR
MAPD distills proprietary LLMs into open-source agentic search models by routing supervision through a style-normalized JSON protocol instead of raw teacher text, keeping the student’s own token distribution intact while a same-model privileged branch supplies dense per-token guidance.
Why It Matters
You’ve shipped a small open-source retrieval agent (say, a Qwen-class 1.7B or 4B model) that plans a query, calls a search tool a few times, and answers. You want it to reason more like Claude or GPT does on hard multi-hop questions, but you can’t touch their logits and their tokenizers don’t match yours. The obvious fallback, fine-tuning on transcripts of the big model’s chain-of-thought, is known to backfire: the student picks up the teacher’s verbose phrasing and confident tone without the underlying skill, a failure mode often called false promise imitation. Meanwhile, the standard training recipe for these agents, outcome-only RL with verifiable rewards, only tells the model “final answer right/wrong” across a 20-step trajectory, which is very thin feedback.
How It Works
MAPD has two stages. Offline, a proprietary model (Claude-Opus-4.6, GPT-5.5, or Gemini-3.1-Pro in the paper) drives a small Multi-Agent System: an Orchestrator decomposes the question into sub-tasks with dependencies, parallel Searchers hit a local Wikipedia index, a Repair agent diagnoses failures using the gold answer as a hint (the gold answer is never written into the trace itself), and a Protocolizer compresses everything into a JSON object with five fields: task type, reasoning plan, extractive grounding facts, partial findings, and a verified answer. A quality gate checks schema, exact-match on the answer, that every grounding fact is a verbatim substring of a retrieved passage, and that no oracle info leaked into the plan. Only ~99% clean protocols survive.
Online, the trick is On-Policy Self-Distillation. The same student weights are run twice per token: a student branch sees only the question and its own prior tokens, a privileged branch also sees the JSON protocol. The student is trained to match the privileged branch’s next-token distribution via reverse KL, and because both branches share weights and tokenizer, there is no cross-vocabulary alignment problem. This dense loss is added to a standard Group Relative Policy Optimization (GRPO) outcome-reward objective, weighted by a coefficient the paper calls lambda.
for question, gold in train_batch: protocol = cached_mas_protocol[question] # offline JSON rollouts = student.sample(question, n=8) # for GRPO for t in tokens(rollouts): p_student = student(question, prefix_up_to(t)) p_priv = student(question, protocol, prefix_up_to(t)) loss_opsd = KL(p_student || stop_grad(p_priv)) loss_rl = grpo(rollouts, reward=exact_match(gold)) update(student, loss_rl + 0.05 * loss_opsd)
Core Insight
The common move when distilling a black-box teacher into an open student is to fine-tune on the teacher’s natural-language reasoning traces. This paper argues the opposite. The teacher’s surface language is the part you must throw away; only the abstract plan and the grounded facts should cross the gap, and they should cross in a schema the student never has to imitate at inference. The load-bearing evidence is the ablation where distilling proprietary raw text actually underperforms using no proprietary teacher at all.
What They Found
•
The ablation that carries the thesis: distilling from a proprietary model’s raw natural-language trajectory (w/o SP&MAS) is worse than a baseline that uses no proprietary teacher (GRPO+OPSD on self-rollouts). Swapping raw text for the JSON protocol under a single-teacher setup adds +7.0 points on 1.7B and +5.6 on 4B. Structure, not teacher strength, is what unlocks the transfer.
•
Headline: on seven QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultihopQA, MuSiQue, Bamboogle), MAPD averages 39.4% on Qwen3-1.7B and 44.4% on Qwen3-4B, beating the strongest prior hybrid SDAR by 4.8% and 3.3% relative.
•
Multi-hop gains are roughly 3x the single-hop gains (7.9% vs 2.3% relative on 1.7B), which lines up with what a decomposition-heavy MAS should transfer.
•
Pure On-Policy Self-Distillation without an external protocol collapses: 5.9% on 1.7B, with response-length clipping jumping from 5% to ~74% as the model degenerates into runaway outputs.
•
The distillation weight has a real sweet spot at lambda=0.05. Push it to 0.1 and the 4B model shortcuts to “retrieve without reasoning”: mean response drops from 135 to ~42 tokens, tool calls saturate at 3 per episode, single-hop scores tick up but multi-hop scores fall sharply (2WikiMultihopQA 45.7% -> 38.4%).
•
Teacher swap is nearly free: Claude, GPT, and Gemini as the offline teacher land within 2 points of each other on both student sizes. Offline synthesis cost was about $1,454 one-time for 25.6K training instances with Gemini.
What’s Useful
Reach for this when you’re training a small open-source agent (retrieval, code search, tool-using QA) and you already pay for a frontier API but can’t get logits out of it. Instead of SFT on the frontier model’s transcripts, run it in an offline planner/searcher/repair loop, dump each solved episode to a strict JSON schema (plan + extracted evidence + verified answer), and use that JSON only as privileged context in a same-model self-distillation branch alongside your existing RL loop. Your production agent never sees the JSON at inference, so there’s no serving overhead or format lock-in, and your dispatcher can swap teacher APIs without retraining.
Code is released at GitHub. The training data is the standard NQ + HotpotQA splits from Search-R1, retrieval runs against the wiki-18 corpus, and the students are Qwen3-1.7B and Qwen3-4B base checkpoints. The paper does not release the pre-synthesized protocol cache, so you would regenerate it yourself.
Takeaway
When you can’t touch a teacher’s logits, distill its plan, not its prose. A rigid JSON schema for “what to do and what facts support it” transfers the useful part of a frontier model’s reasoning without dragging its writing style into your smaller model’s distribution, and it lets you keep the teacher entirely offline.
Caveats
•
The whole approach depends on there being a verifier (exact-match against a gold answer) strong enough to drive both the offline Repair agent and the online RL reward. Open-ended tasks without an EM-style check don’t obviously fit.
•
The lambda=0.1 failure is a warning: too much protocol pressure teaches the model to skip reasoning and just retrieve. The good operating point is narrow and had to be tuned per model scale.
•
Evaluation is Wikipedia QA at 4 turns and 512-token responses with a top-3 retriever. Longer-horizon agents (code, browsing, multi-tool) may need a richer protocol schema than the five fields used here, and the paper doesn’t test that.
Topics
Don't miss new content
Log in to follow topics and personalize your feed.
By content type
Research Paper171 episodes
AI171 episodes